During our ongoing study of the DIY research tool, Google Consumer Surveys, we were today presented with a straightforward question. How many medals would the USA win in the Olympics? But when we went to answer the question, we found that the number 80 had been pre-populated into the answer field. We thought this was odd, and surely would steer respondents to think that 80 was the “correct” answer.
To see how this might have happened, we created our own Google Consumer Surveys project, that asked, “What percent of Americans cannot read?”
Sure enough, in order to “launch” the survey, Google Consumer Surveys required us to pre-populate the answer field with some value. Otherwise, we could not get off the page, and could not continue. This is what a DIY researcher trying to build a question would see:
The “Go!” button, far right, is grayed out and remains grayed out until a value is entered into the field (at left) that says “enter placeholder text.” Google should turn off this “feature” tomorrow.
To make matters more confusing, for any DIY researcher trying to do data analysis with the results of Google Consumer Surveys, Firefox displays these questions one way, and Internet Explorer displays these questions another way. Firefox displays the pre-populated text; Internet Explorer does not. So that means, in the Olympic medal example above, respondents using Firefox (version 14.0.1) would have seen “80″ in the answer window, and respondents using Internet Explorer (version 9.0.8) would have seen nothing.
We would ask the DIY researcher who fielded the Olympic medal research to share with us whether his/her answers clustered around 80. If so, you should re-run your research, quickly, before the Olympics is over. The results you have are invalid.
It is unclear what “placeholder text” Google is looking for a researcher to enter into this empty rectangle. We can think of nothing helpful, and plenty unhelpful. Consider the following example, where the DIY researcher tried ever-so-hard to put something constructive in the box. Did the DIY researcher help respondents by displaying brackets and ellipses?
We doubt it. We would ask the DIY researcher who fielded this research to share with us whether Google accepted answers that were entered with brackets and/or whether Google accepted answers that were entered with percent signs, or only accepted answers with numerals.
To see everything we have written about Google Consumer Surveys, click here.