Train-Wreck on Random Street: How the Google-Matic Answer-Choice Randomizer Ruins Google Consumer Surveys
Some things in life cannot be randomized.
- The Space Shuttle does not launch in 7 … 4 … 8 … 9 … 0 … 3 … 10 … 6 … 1 … 5 … 2 … ignition!
- David Letterman does not read the Top Ten list in random order.
So, it is frustrating to see that some mad scientist at the Google Plex has persuaded the business managers that DIY Google Consumer Surveys (#gsurveys) would have more validity (and be more embraced by the research community, and produce more meaningful results) if the answer choices to all questions were, by default, random. Here’s what happens when DIY researchers are eaten alive by the Google-Matic Randomizer:
The reason why Google does not turn off the Randomizer by default, is that Google is preoccupied at the moment with “junk clicks.” Once Google dimensions how much of a problem junk clicks are, it will turn the default on the Randomizer to Off.
Here is an example of somebody in the playing-card business who had the good sense to override the Google-Matic Randomizer, and produced an actual list that looked familiar, and made sense, to human beings:
If you are thinking about conducting DIY research using Google Consumer Surveys (#gsurveys), or any other DIY research tool, be so careful not to get eaten alive by the tool. If you are comfortable doing it yourself, wonderful. Knock yourself out. But if you have any concerns, think about talking to a professional before you start. A professional researcher will not make the kind of mistakes we see here. Above and beyond turning the Randomizer off, a professional researcher would have taken the ESL (English as Second Language) draft that appears above rewritten the coffee question above to say:
How many cups of coffee do you drink a day?
(15 total words in the rewrite, compared to 27 total words in the original, a 44% improvement).
A professional researcher would have talked to somebody in the fine-art business before creating a scale that tops out at $1,000.
A professional researcher can help you gather the data you need, without falling through one of the hundred possible trap doors that a DIY tool makes possible.
What else have you seen the Google-Matic Randomizer do to ruin an otherwise well-intentioned piece of #gsurveys research?