Unless you’ve been stranded on a desert island with no access to the internet, you’ve probably seen headlines over the last two election cycles bemoaning the accuracy of survey research. And you might be wondering, “Why does it seem that pollsters keep getting it all wrong?”
Well, there’s a very simple explanation but it’s not one that the “survey research experts” want you to know. These days, repeating the same old methods leads to repeated failures.
- Most pollsters are using a communication technology from the 1970s.
The methodology that most pollsters are still using today consists of calling residents. This practice became prevalent in the 1970s as home-based landline telephones became the primary form of communication. And, even though landline use has plummeted over the years, many pollsters have simply supplemented landlines with cellphones. For example, in 2004 more than 90% of households had an operational landline. Now that number is less than 40%. But, rather than changing their methods, pollsters decided to continue calling residents since today 97% of U.S. adults report owning a cellphone. The problem is that this method is increasingly ineffective and extremely inefficient. Stop the dialing!
- People prefer texting to talking.
Just because almost everyone has a mobile phone, it doesn’t mean that people are more inclined to speak with each other. In fact, research has found that people prefer texting to talking. This means that reaching them via text message as opposed to calling them is much more efficient and effective. As Spencer Kimball, the Director of Emerson College Polling recently stated in a New York Times article, the use of text messaging in survey research is “the future of polling.”
- Live caller surveys are extremely expensive.
Among all of the methodologies, live caller surveys are the most expensive – a lot more expensive! For example, a nationwide survey generally requires a sample size of 1,000 completes in order to obtain a confidence level of 95%. A live caller survey to this universe in three languages is estimated to cost about $30,000. We can conduct this same survey with far greater speed, accuracy and efficiency for half the cost.
- Live caller surveys take too much time to complete.
Traditional live caller surveys can take between five days and two weeks to complete. This is because obtaining a representative sample using this method is so time consuming. This is exceptionally problematic since surveys are supposed to represent a snapshot in time. If it takes you up to two weeks to complete your survey, you might as well throw out the results because the dynamics of your circumstances might have changed. By contrast, we can turn around a survey within 48 hours.
- People don’t want to answer a survey during dinner.
If you’re conducting a live caller survey, the best time to reach people is when they least like to be reached. That is, when they’re having dinner. With text message surveys, you can give respondents a window of time in which to respond, thereby allowing them to complete your survey at their convenience.
- Live caller surveys can be skewed by interview bias and social desirability effects.
Extensive research has shown that live caller surveys can be biased in light of the interaction between the interviewer and the respondent. This could be due to a social desirability effect that causes the respondent to provide answers that are seen as more socially acceptable as opposed to their actual opinions. This also allows for more candid responses through more detailed and open-ended questions.
- Text messages remove demographic and language barriers.
If you want to communicate with someone in a different language with a live caller survey, you need to hire callers who speak that language. With text message surveys, you can have your surveys instantly translated with a click of a button. This makes it much easier to contact key demographics that are hard to reach and results in more accurate and comprehensive data.
- Online polling samples are too small.
There are many companies and organizations that utilize online polling. However, even the largest among them is only able to access 15% of the population which is a non-probability sample. This means that their results are generally not representative of the entire population.
- Online samples are skewed younger by age.
In addition to the challenges presented by its limited reach, the samples from online polling are skewed by its younger and higher educated demographics. Only 32% of Americans have earned college degrees according to the latest census data and college graduates can vary widely from general population data. By contrast, text message surveys offer a better distribution by age given its extensive and comprehensive reach.
- Mixed-method polling helps ensure better results.
A mixed-method approach in survey research helps ensure better results by reaching a broader and more representative cross-section of the population. For instance, by using both online panels and text messaging, you can ensure a more representative sample, and you can compare and contrast the results to see if there are any anomalies.
Pollsters failing to adapt and innovate in a changing world are doomed to fail.
Survey research is still an invaluable tool with a myriad of powerful strategic uses when done correctly. It takes experience to assess project goals, build and target the right survey tool and generate accurate and actionable results.