Surveyed to Death

So far in May I got requests to fill in surveys for Consumer Reports, industry research in IT management, a hotel I recently stayed in, new Lyric Opera dining options and at Northwestern: Internal Communications, International Office, course management system, library space planning, research computing needs and dealing with prospective grad students. The Internet, particularly sites like Surveymonkey make surveys very simple to create and distribute. Each survey promises to take only a small amount of my time and in some cases (like Lyric Opera Dining) I actually care about the outcome. But since I get constant requests, I tend to skip nearly all such surveys.

If you ask the average person on the street which is more accurate: a random sampling of 1200 people or an online survey open to all, most will (incorrectly) say the latter. On-line surveys suffer from statistical skewing—they only measure people who take the time to fill out surveys. And as people like me get inundated with requests, the only ones to fill out surveys are people with strong opinions about the topic or those with too much time on their hands and the results of these survey will be a quite poor reflection of reality.

If every survey writer only sent their surveys to a small randomly selected group of people, then each of us would have very few surveys to fill out and could take the time to do so. But we can't expect surveyors to act so responsibly, nearly all surveys will suffer. So don't bother with the surveys. Open up an on-line suggestion box, a message board or a blog and get the discussion going. Use words instead of meaningless statistics to guide your decisions.