Access leading survey analysis tools and market research at mTab.
A notable number of survey respondents are lying on your customer surveys – or not – depending on whom you ask. As some survey data experts and you’ll hear that up to 50 percent of people in any given sample will provide dishonest responses on any given survey. Ask another group of experts and the number drops to “a small minority” of respondents who will either purposely or inadvertently fill your survey with a few mistruths.
While gathering any amount of lies on your survey can be disheartening, there is some good news in the mix. If you know the reasons behind the respondents’ lies, you can more easily recognize and even dissuade them. There are also a few other strategies you can employ to ensure your survey responses are as truthful and accurate as possible.
What Makes Survey Respondents Lie
Your first step is to understand why people lie on surveys, with reasons ranging from innocent forgetfulness to downright maliciousness.
People boast about their behavior
Survey respondents tend to be drawn toward making themselves out better than, more than or somehow superior to others. A case in point comes from a question CBS News asked prior to the 2008 presidential election. The news station wanted to know if survey respondents, and the country, were ready for an African-American president.Most people who answered said they were more ready than the country was, perhaps showcasing how they were more open-minded, accepting of change and all for equality than the nation as a whole.
Other surveys given to Cornell students back in the 1960s found the students consistently provided higher SAT scores on the surveys than those they actually received.
People become defensive
Survey questions that unwittingly, or even purposely, elicit a defensive response may also be answered by a batch of lies. Here respondents may be unwilling to disclose something about their beliefs or nature they don’t want others to know. Some may even be embarrassed by the question, finding it easier to lie than admit a truth they may find shameful.
Survey questions that ask if people read tabloid newspapers fall into this category. Despite few daring to admit they read the tabloids, tabloid sales are mysteriously yet consistently in the millions every week.
People want to be socially accepted
Similar to becoming defensive, people are prone to lying on surveys when they're afraid their honest opinion is not necessarily mainstream or politically correct, a concept known as social desirability bias.An example of this phenomenon arises when Americans are asked if they own guns. A full 50 percent of American families own at least one firearm, but only 25 percent of families will admit it. The 25 percent that are lying would rather tell a mistruth than be demonized for owning a gun.
People are trying to be polite
Perhaps you created a survey asking your customers about your new website design, with the results showing the vast majority really liked it. Or maybe you've launched a new software product and want to know if people found it helpful and user-friendly. Again, the majority happened to say yes.In both cases the responses could be lies, just because people are trying to be polite – or even avoid their own embarrassment. In other words, they are telling the survey sponsors what they think the sponsors want to hear.
In the case of your new web design, regular customers may want to make you feel the enormous effort was worth it, even if the color scheme is garish and navigation options clunky.In the case of your new software product, people who can’t get it to work may be unwilling to admit it, even if the flaw is with the software instructions or design. They may be more inclined to blame themselves for not being savvy enough to catch on, simply telling you it works just fine so they don’t have to feel stupid.
People don’t want to appear old-fashioned
Most people generally want to appear with it, hip or cool, and that desire can skew your survey results. An example in this category revolves around unisex bathrooms, which were popping up at several college campuses in the 1990s.A student at Williams College wrote an essay outlining how both male and female students readily agreed to the idea for a freshmen dorm, although they quickly found it awkward and uncomfortable once the unisex bathroom was established.
Students voted to return to the previous separate bathroom arrangement, with most of them noting they had been against the idea from the get-go. They had only agreed so they wouldn’t appear old-fashioned or uncool.
People can be hurried, annoyed or downright mischievous
Survey respondents may also lie because they’re in a hurry, annoyed or are bent on sabotaging the results. Pop-up surveys, such as those that won’t let online viewers read an article until viewers answer a question or three, can be prone to inaccuracies if viewers are in a hurry to simply get to the article. Other surveys that bar someone from getting to a certain web page or resource can be likewise riddled with random answers that double as lies.
And you may also find a few respondents who deliberately falsify their age, gender, race, income, employment status or any other fact. They may be attempting to be funny, or they may simply have the burning desire to do what they can to mess up the results.
People can’t always predict their future actions
Lies may be part of the fabric when it comes to surveys that ask people why they bought a particular product, or if they intend to purchase specific products or engage in specific behaviors in the future. Here people may be telling the honest truth, although what they say may not necessarily match up with their eventual actions, a phenomenon known as hypothetical bias.
An example in this comes from surveys that ask why consumers purchased personal computers. For more than a decade, the majority of respondents listed personal finance management as one of the top three reasons they decided to buy a personal PC. When it comes down to how many consumers actually ended up using a personal finance manager on their computer, the number is somewhere around 2 percent.
Here people honestly want to use their PCs for personal finance management – or any of the other multiple reasons they say swayed their purchase. But once they find out how tedious and time-consuming personal finance management can be, even with a computer, the idea pretty much dies on the vine.
People simply forget
Ask people about a product, service or experience too far after the fact and you may end up with inaccurate responses simply because respondents don’t remember the truth. This came up in a survey that discovered a 40 percent underreporting of single-day hospital stays, with the underreporting due to people honestly forgetting.
How to Minimize Lies and Maximize Accuracy
Knowing why people lie on surveys is your first step toward developing a carefully designed survey that prompts truthful, accurate responses. A number of reasons behind the lies can be addressed head-on, such as:
Correcting Hypothetical Bias
You can try specific fixes that address hypothetical bias, which refers to respondents incorrectly predicting their future actions, or not doing what they say they will do. One tactic is to ask people how certain they are about their answer.
Those who indicate a high level of uncertainty can have their “yes” answers changed to “no” when you’re tallying results. It can be tough, however, to set a certainty scale across the board, and the strategy only works on straightforward yes and no questions.
Another fix for hypothetical bias is to explain the phenomenon somewhere on the survey, then specifically ask people to do their best to avoid it. This strategy may not work for everyone, and it also poses the risk of introducing a new bias into the survey.
Knowing the Trigger Questions
Questions that are most likely to prompt lying are those that involve the three Bs:
These triggers were evident in several of the reasons for lying, with people boasting, wanting to appear superior, wanting to fit with mainstream thought or wanting to be part of the in crowd. Pay special attention to any questions involving these three aspects to ensure they are worded precisely and designed to be fool-proof.
One more tip is to realize your respondents are much more likely to tell the truth if they have a connection to your company. Your regular customers are more likely than total strangers to provide honest answers. And no matter how honest you believe the responses to be, always assume you’ll have even a slight level of inaccuracy without further in-depth research to back up and fortify the truth.