But what is sometimes overlooked are the questions themselves. What makes a good survey question?  What are the inherent tradeoffs when drafting a question?

A good survey question should accurately and efficiently collect the data you need. The best way to accomplish this is by sticking to the TL;DR Insights CORE principles:

• Clear        

• Objective

• Relevant

• Engaging

CLEAR

The first and most important consideration is to ensure your questions are clear, both in what you are asking and in how you ask it.

Being clear in what you are asking means avoiding ambiguity - you want each person that reads your question to have the same interpretation of what you want to know.

You can do this by not asking multiple questions at once, such as "How satisfied are you with the functionality and value of feature A?". If you could reasonably answer differently for "functionality" and "value" then you should avoid mixing the two; otherwise your results may be unreliable.

Next, focus on being clear in how you ask the question. Researchers will often strive to write questions that leverage optimal grammar and proper vocabulary to communicate specific and complex ideas. But surveys are neither the time, nor the place to for such haughty pursuits.

Rather than aiming to show off your Inner-English Teacher, extensive vocabulary, and impeccable grammar, focus on an entirely different lesson from your writing classes and Know Your Audience.

Survey participants are often rushed, distracted, and trying to complete your survey as quickly as they reasonably can. Clarity is of utmost importance to ensure they answer accurately. Wherever possible:

• Keep questions simple

• Use as few words as possible

• Emphasize key phrases using bold or underline to draw attention

• Use imperfect grammar if it improves comprehension

• Favor common words over jargon

One thing to call out is the potential conflict between eliminating ambiguity and simplicity. Do you keep your questions short?  Or do you spell out additional details to further clarify your point? This is where the art of survey writing comes into play. More on that in a future post.

OBJECTIVE

When drafting survey questions, you will likely have a deep understanding of your research topic and a hypothesis as to what data a specific question will generate. While difficult, you must not let this information produce a biased or leading question. If you show respondents the way, they will likely take it. If you give them a way to satisfice or spare your feelings, they likely will.

Here are some tips for writing unbiased questions:

Make sure your question is neutral. Do not ask a respondent how wonderful they think your product is; ask them what they think about it.

Provide the right options and the spectrum of responses. For example, if you are gauging reactions that could span both positive and negative, make sure to include balanced responses from both sides.

Avoid asking confirmatory questions (i.e. yes / no), or other questions with an obvious answer (i.e. How likely would you be to litter?)

RELEVANT

You likely have several different research and business questions that you would like to answer with your survey, possibly from several different audiences or segments of the market. However, it may also be the case that not all your respondents have the necessary expertise or life experience to answer all your questions, especially true in B2B studies.

It is important to keep the following in mind:

Each audience can only give you the information they know. If you ask them a question that they are not capable of answering, they will likely still answer it to the best of their abilities. For example, it may make sense to ask a CTO about their overall cloud strategy, while asking a Junior Developer the same question may produce illogical responses due to their lack of knowledge.

Confirm that topic areas are relevant to your audience before asking them a more specific question. For instance, before asking your audience what type of smartphone they have, you should confirm that they own a smartphone (1 in 5 Americans do not).

ENGAGING

Survey respondents have busy lives and may even multitask while completing your survey. Do not take offense, take it as a challenge! Try to keep it interesting and engaging. Long stretches of explanatory text, instructions, or jargon-laden questions will encourage respondents to tune out. The best-case scenario is that they drop out of your survey or are caught in the data quality checks. The worst-case scenario is that your results are muddied by respondents randomly answering questions they cannot bring themselves to read.