There are 2 types of multiple-choice questions in survey research.

• Click one of the following. (known as radio buttons)
• Check all that apply. (known as check boxes)

These 2 question types are effective if properly used, but they are not interchangeable. When misused, they turn an otherwise good research question into an unanswerable mess.

Google Consumer Surveys invites questions to be misused by DIY researchers who are in over their heads. Consider:

Taken clockwise from top-left:

1. When we eat junk food, we feel bloated and depressed and guilty. The single-answer format does not permit us to answer this question properly. Throw away the data collected from that question, or rewrite the question to read: “When I eat junk food, the one feeling i have more than any other is…”
2. If you ask people “which of the following” they have scanned, naturally it is possible they will have scanned more than one. The question as constructed does not permit more than one answer. Throw away the data collected from that question. Separately, if you include “if any” in the question, you must include “none” in the possible answer choices.
3. If you ask, “which of the following do you use most-often” (and especially if you put MOST OFTEN in all capital letters, like you are shouting), you are signaling to a research respondent that you are looking for one and only one answer. Yet the question is written using check boxes, which permit — encourage — more than one answer. Throw away the data from everyone who checked more than one box.
4. If you ask what would be the maximum that you are willing to pay, you are signalling to a respondent that you are looking for one (and only one) answer: the maximum. Yet the question as formatted permits multiple answers. Again: throw away the data from anyone who checked more than one box.

To a DIY researcher, writing his or her first questionnaire, the differences between “click one” and “check many” may seem so subtle as to be unimportant. But every word said to a respondent must have clear meaning if every answer taken back from a respondent is to have clear meaning.

Don’t fall into the trap so many fall into with Google Consumer Surveys: “this looks so easy even a child could do it.” Yes, it’s easy. It’s easy to do it exactly backwards, as all 4 of the above DIY researchers did. It’s not so easy to get right.

If you are in doubt, hire a professional researcher. Professional researchers write questions for a living. A professional researcher can help you save money in the long run, by getting your research set up correctly the first time.