How to write a great survey

Learning how to write a great survey is one of the most important aspects of conducting research. Your questionnaire defines your inputs and the resulting data you end up working with. A poor questionnaire can lead to inaccurate data and insights. Here are five tips to avoid this outcome.

1. Clarity

Keep your question wording simple. Use everyday language. Avoid long words, acronyms and business jargon. Everyone should be able to understand the question you are asking.

Common mistakes that can add confusion:

  • Be careful using ‘and’ to ensure you are not asking about multiple things. For example, this question is problematic: “How satisfied are you with the product and the customer service?”
  • Be specific. Asking, “How happy are you?” is far too vague. Be more specific like “How happy are you with your commute to work?”
  • Add context. “How likely are you to buy an iPhone?”. To make accurate predictions of purchase behaviour we need to add context to the question such as “… the next time you upgrade your smartphone” or “… in the next six months”

2. Avoid leading questions

This is obvious and most people try to avoid this. However, this mistake is usually an unintentional one. The person writing the question must aim to clear their mind of their own biases and place themselves in the shoes of the intended respondent.

Common mistakes to avoid leading your respondents:

  • Do not lead with a belief, for example: “We have made lots of innovative improvements to our brand, how would you rate them?”. Instead try: “We have made some changes to our brand, how would you rate them?”
  • Even when capturing more generic attitudinal data, make sure the question is balanced. For example, ask “Do you agree or disagree …” rather than “Do you agree…”
  • Be careful using hard and soft words. For example, you will get a different response to this question: “How important would you rate the prohibition of smoking in public places?”, if you replaced “prohibition” with “minimisation”
  • if your answer list is not ordinal (i.e. there’s no a natural order to your answer list) then randomise your answer list so each respondent sees the question with the answers in a different order. Subconsciously, you may have put the answer you prefer at the top of the list, so randomisation removes any order bias.

3. Ensure your answer list is relevant to all

So far we’ve just talked about how the right questions can help you write a great survey, but the answer list you provide is equally important. Without a balanced set of answers the data you get back will be skewed.

Top tips:

  • If in doubt, assume that your answer list is not exhaustive and that you have probably missed something. To cover this add options like “other” or “don’t know”.
  • However, do not add too many answers to your question otherwise the respondent will be overwhelmed and may not answer the question properly. We’d recommend a maximum answer list of 10-12 options.
  • Always consider adding a null option, for example, with this question…

Which brand of car do you prefer?

  1. Audi
  2. Ford
  3. BMW
  4. Toyota
  5. Tesla
  6. Other

The ‘other’ option compensates for the short list (there are lots more brands), but the question does not work for people who do not like cars. To allow for this, add “if any” to the question text and ‘None of the above’ to the answer list, like this:

Which, if any, brand of car do you prefer?

  1. Audi
  2. Ford
  3. BMW
  4. Toyota
  5. Tesla
  6. Other
  7. None of the above

4. Balancing Rating Scales

Rating scales that capture metrics – like satisfaction, importance, or agreement – need to be balanced. This means including both positive and negative ends of the scale. You might have a question like this: “How satisfied are you with Product X?”.

This is a bad example of a response scale:

  1. Somewhat satisfied
  2. Very satisfied
  3. Extremely satisfied

A more appropriate question would be less leading and the response scale would include negative options, like this:

How satisfied or dissatisfied are you with Product X?

  1. Extremely dissatisfied
  2. Quite dissatisfied
  3. Neither satisfied nor dissatisfied
  4. Quite satisfied
  5. Extremely satisfied

A reason for choosing the first scale would be that a product might operate in market with very low levels of dissatisfaction. That’s fine and very common, especially in mature markets. For example, most people are satisfied with their smartphone. To accommodate for this we would recommend a top weighted scale to help measure the varying the degrees of satisfaction, but you still need a negative and neutral option, like this:

  1. Dissatisfied
  2. Neither satisfied nor dissatisfied
  3. Quite satisfied
  4. Very satisfied
  5. Extremely satisfied

5. Length and order of your survey

Do not over burden your respondents with too many questions. The longer your survey the less attention they will pay to reading and answering your questions properly. Any survey that takes longer than 10 minutes is at severe risk of this happening.

Regarding ordering your questions, try to start with the more generic and broad questions first and the specific questions later. This will make the survey easy to start with and warm the respondent up to answering more specific question that will require more thought.

Also avoid giving away too much too early on in the survey. The questions and answer lists you ask at the start of the survey will ‘prime’ or influence the respondent in how they answer subsequent questions.

Mistakes like the ones listed in this article are easy to make. Writing a good survey takes practice, but even if you are a seasoned professional, we would always recommend getting someone to check it before you publish it

Ryan Garner

Ryan has provided insights to global brands, helping shape products and services around real customer needs. He has spent the past 3 years helping businesses think about the opportunities in the personal information economy. He now brings this experience to CitizenMe. As much as Ryan loves technology, he likes to escape by growing fruit and veg on an allotment with his kids. And if you have not already heard, Ryan is a very smug Leicester City supporter right now!

All stories by: Ryan Garner

Leave a Reply

Your email address will not be published.