The psychology behind surveys: How to design questions that get results

Ever made an online purchase because there’s “Only one 1 left in stock?” Then you’ve experienced what’s called “the scarcity effect.”

And virtually every teenager has fallen prey to the bandwagon effect, begging their parents to do something because “everyone else is doing it.” 

Anyone who’s taken a college psych class is likely familiar with these principles of human behavior. But social psychology isn’t the only field where those influences come into play.

Marketers tap into cognitive biases every day to appeal to customers and understand their audiences. Few people understand this better than Tammy Duggan-Herd. As Senior Marketing Manager at HubSpot with a Ph.D. in Applied Social Psychology, Tammy is a fount of knowledge on all things marketing surveys. She’s used surveys throughout her marketing career to craft better messaging, recommend new product features, and drive value for platform users.

What does it take to craft a great survey? Turns out, it’s a healthy mix of marketing and psychology. To get the right answers, researchers need to ask the right questions. With Tammy’s help and expertise, we’ll explore how to do just that. But first, make sure you don’t fall prey to these common survey mishaps.

Avoid these 4 survey mistakes we’ve all made

Before Thomas Edison discovered how to let there be light, he found a thousand ways not to make a lightbulb—and we’ve all been there. Marketers have found plenty of wrong ways to survey.

Tammy’s seen it all, and she notes four major missteps that marketing teams often take in the survey process.

1. The “why” behind the survey is missing

Marketers who want to convince an audience to complete a survey should start by clarifying why they’re conducting the research and what they’ll do with the results.

Tammy says it best: “We don’t want to waste our customer’s time. If we don’t have that clear objective and share how we are going to use the information we gather in a useful way, we’re setting ourselves up for failure.”

A survey’s purpose informs what kinds of questions to ask. When marketers start with the end in mind, they ask questions that allow them to act after the fact. A clear sense of purpose is essential to weed out any non-essential questions. So, marketers, your job is to make sure respondents know why every answer they give matters.

2. Long surveys with too many questions

Never-ending surveys are a no-win scenario for audiences and brands. Respondents get tired and bored, while brands get fewer responses and less data. “Every single question you add just increases your drop-off rate,” Tammy says.

Companies might think they need answers to all 50 questions they’ve put together. But the answers people give when they’re tapped out typically aren’t quality answers—Tammy calls this “satisficing.” “They may work their way through the survey, but it gets to a certain point where they're just clicking. They're not really thinking that hard about their answers,” she says.

Instead of building never-ending surveys, Tammy recommends asking, “Do I really need this information?” about every question. Researchers need to be ruthless, separating the non-essentials from the must-haves. “With attention spans as low as they are, there’s just not a lot of room for those nice-to-haves.” A good rule of thumb we often share with our customers: Never ask more than 10 questions in a survey.

3. No clear incentive

Respondents need a compelling reason to share their time and opinions with brands—and researchers need to make that reason crystal clear right away. They do so through both non-monetary, and monetary incentives.

Non-monetary incentives are big-picture benefits that someone will gain for completing a survey. Respondents might go for it if you offer them

• Advanced tips, tricks, or product insights 

• Product discounts or limited-time offers

• Admittance to a webinar, gated content, or other virtual event

Brands need to share the “what’s in it for me?” hook respondents crave upfront to prove that a survey is worth their time.

Some audiences need some extra cashflow in their pocket before carving out time for a survey. “Even a small incentive is better than no incentive,” Tammy says. She also notes that a “sure thing” incentive—like a $5 gift card—is much more effective than a chance to win a larger prize.

Without a clear incentive, most surveys won’t bring in much of a data set at all.

4. Siloed survey creation

Most marketers know that they need a second set of eyes on their creative work before publishing—and surveys are no exception.

Tammy has encountered plenty of surveys with obvious errors. “I look at surveys sometimes, and I think, ‘Did the person creating this not have anyone else review the survey?’” When a survey maker is the only person who reviews the form, they’re too close to the content to catch every mistake that might trip up respondents. 

An internal reviewer who completes the survey can point out glaring errors in the form’s logic, poor answer choices, formatting issues, or even grammar mistakes. Those corrections translate to an all-around better survey experience.

14 essential principles for better survey design

Professor Don Dillman became research royalty in 2000 when he published his book, Mail and Internet Surveys. Since then, his method for survey design has been the industry standard for researchers everywhere. “A lot of people, even if they don’t have this whole book, will fall back on what are known as Dillman's principles,” Tammy explains.

The following 14 best practices are some of Tammy’s go-to guidelines from Dillman’s list. They offer a solid foundation for choosing great survey questions and designing research that yields actionable results.

1. Choose simple over specialized words

When someone agrees to fill out a survey, they usually aren’t up for a vocab quiz or an IQ test. Complex questions bog down audiences—so resist the urge to sound smart or impressive in survey questions. 

For example, instead of “demonstrate,” try “show.” Rather than “utilize,” you might go with “use.” As a rule of thumb, look out for words over six or seven letters—and try a simpler, shorter word instead. 

“The main goal of any survey question that you develop is that all people who read that question are going to interpret it the same way,” Tammy explains. “If you start using jargon or complicated words that people don’t understand, you’re risking that.”

Cut down on confusion by using words that people hear and say in their daily lives.

2. Use as few words as possible

Keep question wording concise. “If the question goes on and on and on, you’ve lost me,” Tammy says. Long questions can confuse respondents so much that they pick a random answer and move on—which leads to unhelpful data.

If a question is getting too long, consider breaking it up into two questions. Alternatively, offer a scenario or instructions about the topic, and then pop the question.

3. Ask questions in complete sentences

No one enjoys a curt customer service interaction—the ones where a gum-chewing, headset-wearing agent mutters, “Name? Age? Last four digits of your social?” Make your surveys friendly and engaging by asking survey questions in complete sentences. 

“What’s your email address?” feels far more conversational and less transactional than “Email address?” This small touch puts respondents at ease and gives them a better survey experience.

4. Avoid double-barreled questions

Imagine a yes or no survey question that asks, “Do you like burgers and Brussels sprouts?” Laying aside this odd-sounding dinner combo, sprouts fans who don’t like red meat would have to answer no—but their answer would only tell part of the story. 

Tammy calls this a double-barreled question. It’s one where a survey asks someone to answer two questions at once. Well-crafted surveys always steer clear of these confusing questions.

To learn about how customers feel about a product and its pricing, ask about each topic in its own separate question. (“How do you feel about our product, X?” then “How do you feel about X’s pricing?”) Break down multi-faceted topics to get the most accurate results.

5. Offer answer choices to remove the burden of recall

Some surveys ask people to provide information they won’t know off the top of their heads. Do most people know precisely how much money they spent on hair care products in February or the exact number of movies they saw in theaters last year? Probably not.

A well-crafted survey question takes some of this burden off the audience by offering a range of options to choose from. Respondents feel less overwhelmed when they don’t have to do the work of remembering a specific number.

Alternatively, ask for an opinion. Instead of asking how much someone spent, ask if the item was very expensive, a little expensive, or inexpensive. “That’s something they may have a ready-made answer for,” Tammy notes. “But if you’re expecting them to have an exact dollar amount, that is a lot of cognitive energy, and you’re likely going to get an inaccurate answer.”

6. Don’t ask respondents to say yes to mean no 

“Do you agree or disagree that marketers shouldn’t avoid clear wording in surveys?” Imagine encountering this question in a survey. It feels a little like you need an algebra textbook to decode it, right?

Surveys shouldn’t force the audience to do mental math to answer a question. Confusing wording makes respondents scratch their heads (at best) or pick a random answer (at worst).

Steer clear of double negatives, and ask straightforward questions. “Do you agree or disagree with the following statement: Marketers should use clear wording in surveys.” reads much better than the earlier option—and hopefully everyone would answer “agree.”

7. Turn open-ended questions into closed-ended questions

Think back to high school. Were multiple-choice quizzes or essay exams more mentally taxing? Most test takers prefer to answer closed-ended, multiple-choice questions. The same holds true in the great debate between open-ended and closed-ended questions.

Tammy says that open-ended questions in a survey—rather than a phone or face-to-face interview—present a problem. “[Open-ended questions] depend on the extent to which the respondent is willing to think hard about that question and write a complete answer.” 

In contrast to multiple-choice questions, open-ended questions are the most cognitively taxing, Tammy explains. “These questions are the least likely to be answered, and the more you add, the more you increase your drop-off rate.” The best practice is to turn open-ended questions into closed-ended ones by providing answer options.

One caveat though: What if you really want the qualitatively rich responses that come from open-ended questions? Ask a closed-ended question, then follow up with an open-ended one. Ideally, leave the open-ended questions optional to reduce drop-off.

8. Include both ends of a response scale in the question

The best survey results are unbiased survey results. Avoid swaying the audience in one direction or another by including the far ends of a response scale in the question. 

• How satisfied or dissatisfied were you with your service? 

• Do you agree or disagree that this product was priced fairly?

Offering both ends of the spectrum reminds respondents of the available options and prompts them to reflect honestly on their answers.

9. Keep comparisons equitable

Naturally, every marketer wants their product to look and sound good. But if the goal of a survey is to gather accurate data, a survey isn’t the time and place to drop hints about how great the product is. So avoid unequal comparisons. 

Don’t skew negatively in questions about the competition (“What’s the biggest problem with [Competitor Name]’s software?”) while only trying to make your product sound amazing (“What’s the greatest thing about our software?”). 

As Tammy says, “You set them up to think negatively about that other thing versus yours.” Instead, when gathering information about your product and a competitor’s, ask the same questions about both to avoid biased results.

10. Balance positive and negative answer choices

When using a scale to gauge sentiment (from “very satisfied” to “very dissatisfied,” let’s say), build a scale that offers an equal number of positive and negative responses. “Otherwise, you’re kind of biasing their answer,” Tammy points out.

Likert scales often use an odd number of values to gauge sentiment—such as one to five—with a neutral response in the middle. Tammy recommends labeling what each value represents, to lower the cognitive energy people might spend deciding the difference between, say, a two and a three.

11. Create mutually exclusive categories 

Confused respondents provide poor survey results. Make sure when you’re offering numerical ranges for people to choose from that there’s no overlap in the answer choices.

For instance, let’s say a question asks someone how much they spent on lunch yesterday. Answer choices shouldn’t include the same number in multiple ranges like: 

1. $0-10

2. $10-20

3. $20-30+

If someone spent exactly $10, both A and B are technically correct. (Likewise, if they spent $20, B or C is equally valid.) They’ll have to just pick an answer based on how they feel. 

Instead, offer mutually exclusive ranges, such as:

1. $0-10

2. $10.01-20

3. $20.01-30+

These distinct ranges limit the cognitive energy people will spend picking an answer and ultimately yield more accurate results.

12. Make your answers exhaustive

The audience should always see their answer on a list of choices, so be sure to account for all possible responses. When asking how much people spend on streaming services per month, don’t just include $0-20, $20.01-40, and $40.01-60—they might have spent more than that.

Include a “$60+” answer that accounts for any answers above the expected ranges so that anyone answering the question finds their place in the survey question.

13. Avoid vague qualifiers when you can offer more specific options

Words like “rarely” and “often” show up in everyday conversation. But their definitions vary from person to person. Someone might use the word “rarely” to describe both eating at McDonald’s once a year and going to the dentist once per decade.

For accurate answers, define terms as clearly as possible. Rather than “rarely,” specify “once per year.” Instead of “often,” say, “once a month,” “weekly,” or “daily.” Give respondents specific answer choices instead of subjective ones.

14. Write questions that don’t sound like your marketing team

When people browse your website or follow you on social, a strong brand voice helps them connect with you. But Tammy cautions against projecting that same brand persona in survey questions. 

The goal of a survey question is for everyone to understand it the same way, but Tammy cautions that using brand language can lead to varied interpretations of the question. On the other hand, asking questions in a more professional or “scientific” voice can make people feel like the survey is a neutral third party they can be honest with.

Of course, companies should make it clear that they are the ones conducting the research by including their logo in the survey, for example. But using brand language or lingo in survey questions may not make people more likely to share their information. “I think it might do the opposite,” Tammy says.

Best practices for high completion rates

Once every survey question has been crafted with care, it’s time to get the survey into people’s hands and convince them to complete it. Tammy shares a few tips on this as well.

1. Stay transparent

Convincing someone to complete a survey can be a hard sell. There’s story after story of how brands have misused or breached important customer information.

“Mistrust is at an all-time high,” Tammy says. “People are far more wary of sharing their personal information and of what companies are going to do with the data they share.”

This means brands need to be upfront with potential survey respondents, sharing details like:

• What the data will be used for

• Who will have access to the data

• Where and how the data will be stored—and for how long

• Why the data is being collected

 Companies need to prove they’re trustworthy for audiences to offer up their time and information. Transparency and authenticity encourage people to place a vote of confidence in brands they believe in.

2. Make them care

We’ve already talked about how important it is to offer your respondents an incentive to complete your survey, and it bears repeating: If someone is going to spend their precious time and energy answering your questions, they need to care about the results on some level. 

Before and after a respondent clicks into the survey, tell them why it’s worth their time. Will their answers strengthen their experience with a product or service? Will the survey lead to more targeted messaging or maybe even strengthen their industry as a whole? Let them know the ‘why’ behind the research so they know they’re contributing to something meaningful. 

Of course, making people care also typically means offering a tangible incentive like a gift card, discount, or waiver for a free item.

3. Ask the right people

This one’s huge. Brands need to be thoughtful about their sampling and make sure they’re reaching out to the right people with the right expertise and experience. Surveys should provide context early on, including an explanation of who the survey is designed for. This allows anyone who isn’t part of the target group to self-select out right away.

Researchers can also start the survey with screening questions that confirm whether someone is right for the survey. This saves time for irrelevant audiences and keeps the brand from receiving skewed, inaccurate responses.

The survey tool itself also plays a pivotal role in customizing the audience’s experience. If Question 1 asks if the respondent has a dog and they answer “No,” they shouldn’t then have to answer, “What brand of food do you purchase for your dog?” If that happens, Tammy says, “I feel like you aren’t listening to me. You’re wasting my time.” 

Instead, use logic or branching to keep respondents on a targeted, personalized journey so they want to finish the survey.

4. Time surveys well

Another way to keep surveys relevant—and audiences engaged—is by sending a survey that’s directly related to an action that the person recently took. 

Did they just have a customer service interaction? Send a survey about the elements of that experience shortly after—when a survey is timely, it feels way more relevant. 

Another example: If you want to learn about customers’ onboarding experience, don’t wait a while to send a survey—ask right after they onboard. “If you ask them 12 months later, they may not have that knowledge ready or remember it in a way that's going to motivate them to answer that question.”

To tackle survey strategy, go back to psych 101

Marketers don’t have to have a Ph.D. in psychology to get stellar survey results.

But they should stay curious about how people think and what legitimately engages audiences. It’s this  knowledge that fuels them to ask questions which capture the golden nuggets of data they’re after. 

According to Tammy, crafting questions is far from the first step of designing a survey, though. She takes several key steps before she’s ready to actually ask anything or send off a survey. She first:

• Defines the survey's objective

• Confirms whether a survey is the best method to capture the data

• Decides on which tool to use (because how you ask matters)

• Maps out what to do with the knowledge gained

• Determines who the survey is for

• Outlines the flow of the survey

Then—and only then—does Tammy think about which questions will measure what she wants to learn, and she keeps the core principles of great survey design in mind at every step. This kind of big-picture prep work keeps researchers crafting survey questions stay focused, avoid bias, and limit the cognitive energy respondents have to spend. And that’s a recipe for great results.

Tap into the psychology of surveys today to turn research from a shot in the dark to a surefire way to create better marketing.