Writing an effective questionnaire is not a task for novices. At the very least it requires an understanding of four basics. These are:
1. Considering the differences that exist when writing a questionnaire that respondent’s will fill out themselves as opposed to when a professional interviewer administers the questionnaire to the respondent.
2. Knowing what questions should be asked early on in the questionnaire,
in the middle or toward the end.
3. Understanding how to phrase questions
4. Being sensitive to questionnaire length.
I am fully aware that it boarders on the absurd to address as broad a topic as questionnaire construction in a blog piece—or even in two, three, ten or twenty pieces for that matter. There are dozens of books on the subject not to mention thousands of technical papers. But in discussing certain givens here it might, in some small way, help in your next market research study. So, here goes:
1. Self Administered vs. Interviewer Administered Questionnaires
There are some basic differences in how the questionnaire should be constructed if it is to be filled out personally by the respondent or if an interviewer is going to administer it. These are:
Self-administered questionnaires should be simple, straightforward and logical. Question 2 should follow question 1. Question 3 should follow question 2, and so forth. Further, the going-in assumption with self-administered questionnaires should be that respondents will not complete a questionnaire when there are complex skip patterns, when pages are crowded or hard to read or when instructions for completion are overly complex.
It has been estimated that as many as 50% of respondents who start a self-administered questionnaire will not complete it because they become irritated and annoyed at the way it is constructed. When writing a self-administered questionnaire, then, every care must be taken to ensure that it is easy to complete in that it almost answers itself. Know that:
Self-administered questionnaires should be written with an eight grade mentality in mind while interviewer-administered questionnaire can be quite complex. Because interviewers are trained in the flow of the questionnaires they administer and will conduct a number of practice interviews prior to confronting a respondent, developing a complex questionnaire that is interviewer-administered does not present a problem for the respondent.
Interviewer-administered questionnaires can easily accommodate skip patterns that jump a respondent from one section of the questionnaire to another based on their responses. Sometimes, particularly with face-to-face interviews, the questioning process might also involve showing respondents certain products or exhibits during the interview or having the respondents read concepts or ideas based on how they respond to various questions.
2. Knowing what questions should be asked early on in the questionnaire, in the middle or toward the end
Questionnaires have a flow to them, usually from the general to the specific. But because the types of studies you can do vary so greatly, as do the goals of every questionnaire, there are no hard and fast rules for the flow or sequence in which questions should be asked. Nevertheless, I would suggest following certain guidelines:
Keep the respondent in one mind-set at a time. If at all possible, complete all your questions about one topic before moving on to the next. For example, don’t ask about a favorite place to shop, then about brands used and then go back to additional questioning on favorite place to shop.
Ask the easy questions first. Simple questions regarding behavior, such as frequency of buying, brands purchased or places shopped, are easy for respondents to answer because these don’t require a lot of thinking or pondering. As such, respondents quickly get comfortable with the interview.
Ask more involved or introspective questions toward the middle of the questionnaire. Be prepared to transition to questions that require thought and consideration after the respondent is a few minutes into the interview and once the easy questions are out of the way. Respondents don’t mind giving more thought to complex questions once they are comfortable with the interview format or process.
If it’s important to tell respondents who the study is for, do so at the last possible moment. Sometimes you’ll have to identify the sponsor at the beginning, but when this isn’t necessary, keep the respondent in the dark. Once they know who is doing the research, every answer they give will be with that knowledge and will present an informed bias.
Save sensitive questions for the end. Again, this might not always be possible, but when it doesn’t matter, be aware that sensitive questions such as race or income can alienate respondents and turn them off to the entire interview process. If asked at the end, respondents are more likely to answer as they are wholly invested in the questionnaire.
3. Understanding how to phrase questions
It takes very little thought to write out a bunch of questions on a piece of paper and call it a questionnaire. It takes considerable thought to write questions that are good ones and produce a meaningful questionnaire. Consider the questions below as examples good and bad questioning techniques:
Biased question: What do you like about the last airline flight you took? Assumption here is that respondent liked something and the question tends to push for a positive response.
Unbiased question. What, if anything, do you like the last airline flight you took? By simply using if anything as part of the question phrasing, the respondent is not put on the spot to find something to like.
Dual-thought question: What, if anything, do you like or dislike about your internist? With such a question, respondents tend to focus first on the strongest likes or dislikes. If it happens to be something they like, they will give less thought to what they might not like, and vice versa. Here, it would be much better to ask two questions, one focusing only on likes and the other only on dislikes.
Multiple-thought question: With 10 being more important and 1 being lest important, how important is it for a bank to have friendly and knowledgeable employees? Friendly is one thought here; knowledgeable is another. When you ask a dual-thought, you are stuck interpreting both issues at once. It’s probably a good idea that a bank should have both friendly and knowledgeable employees, but it might not be necessary. Being just the friendliest bank could set it apart, as could just being the bank with the most knowledgeable people. Keeping your questions to a single thought is most always the best approach.
4. Being sensitive to questionnaire length
Questionnaire length is dictated by the study objectives. Some questionnaires take five minutes worth of questions to address the objectives, others will take an hour. Two simple rules of thumb should be kept in mind:
Respondents will sit still for 40-page self-administered questionnaire if it is well-constructed and easy to complete. They will toss two-page questionnaire if it is cluttered and complex.
When conducting telephone interviews, it’s relatively easy to keep respondents on the phone and answering questions for 15, 20 or 25 minutes if the questionnaire has a good flow and is thoughtfully written. But try keeping a respondent on the phone for 3 minutes with a questionnaire that is the least bit confusing, seens redundant or is insensitive to sensitive issues.
Suffice it to say here that questionnaire length is less an issue than the other three that were discussed. Questioning the respondent in a professional and considerate manner is where your focus should be.
If some of these quidelines are helpful to you, please let me know. Also, if there are other questionnaire development issues that you'd like me to address, shout 'em out.