When working with data it’s prudent to approach with some skepticism. Ask questions like, “Who collected the data?” “How were the questions asked?” and “Does this data give the full picture, or is there something missing?” The answers to these questions can help you identify cognitive biases that may affect the data or the results.
A cognitive bias is a type of mental shortcut that can happen when processing information or making decisions—so that’s pretty often. While not all cognitive biases are bad, there are many that can affect how people answer interview or survey questions.
Keep the following three biases in mind when wording survey or interview questions and interpreting or presenting results:
Social Desirability Bias
Even before likes and views were used to validate our daily activities, the need to be viewed favorably by others affected our behavior and responses. We tend to over-report the good, under-report the bad, and hide the really really bad. If asked a sensitive question, a respondent may not be completely honest if honesty means voicing an unpopular opinion or possibly upsetting the interviewer. One way to avoid this when conducting a survey is to have the respondent complete the survey by themselves, whether on paper or digitally.
This is the tendency to overvalue recent events. Imagine you went to the theater to see a newly released movie. The beginning of the movie was predictable, the middle of the movie was a snooze-fest, but at the end of the movie, there was a huge twist that you never saw coming. You emerge from the theater and your friend asks, “How did you like the movie?” At that moment, still reeling from the ending, you would be more likely to give it a better review. Keep this in mind when asking people about recent events.
In a room full of people, it can be difficult to voice a dissenting opinion or even offer a different idea. This is group think—the tendency to set aside personal opinions and adopt the opinion of the rest of the group. This can happen in group discussions, focus groups, or even in a regular meeting. This doesn’t mean that group settings shouldn’t be used to gather information, but watch out for this inherent pressure to conform and address it early on.
Be skeptical of data. When analyzing a dataset or reviewing a data visualization, your first step should always be validating the data. Look into the data source for yourself, see what questions surveyors ask and how, and see if the data holds up against these and other cognitive biases.
Graphics by Parul Agarwal