When is eleven scarier than twelve?

About 23 years ago, a team of psychology researchers played an elaborate trick on their students, convincing them that an 11% risk of a disease was more likely than a 12% risk. How did this happen?

When is eleven scarier than twelve?

About 23 years ago in Iowa, Paul Windschitl and his team of psychology researchers played an elaborate (if mundane) trick on their students, convincing them that an 11% risk of a disease was more likely than a 12% risk. 

Of course, eleven is less than twelve, so how did this happen?

The students were first given three fake facts about three fake diseases:

  • Stomach churn discomfort (SCD) involves embarrassing stomach noises after a sugar binge. It affects 4% of men and 11% of women.
  • Wine-breath disease (WBD) makes your breath smell like you’ve just had a boozy brunch. It affects 1% of men and 2% of women
  • Gluten bloat condition (GBC) causes excessive bloating after a gluten-loaded meal and affects 20% of men and 12% of women.

To put yourself in their shoes, pretend you’ve spent a few minutes reviewing these factoids. Then you go watch cat videos on TikTok for a few more minutes. When you come back, the researcher pops the big question: 

"Are women more vulnerable to stomach churn discomfort (SCD) or gluten bloat condition (GBC)?"

With the data in front of you, it’s hopefully clear that GBC’s 12% risk is greater than SCD’s 11% risk. But these absolute risk numbers aren't what sticks in our minds. Instead, when we try to apply this information to judgments like “which disease is worse?” our brains work from gists like:

  • Gross stomach thinger (SCD) impacts more women than men. 
  • Gluten bloaty thinger (GBC) impacts more men than women. 

These gists have a quirk though: Instead of capturing the absolute risk values, they focus on women's risk relative to men's risk. 

This is an example of a contrast effect. In the same way that lukewarm water feels hot after holding an ice cube, risk for one group can feel exaggerated when compared to another group with lower risk.  

Selected results from Windschitl et al's "Context and the Interpretation of Likelihood Information: The Role of Intergroup Comparisons on Perceived Vulnerability," showing how responses in the third experiment diverged away from the correct answer due to a contrast effect.
After reading that Stomach Churn Disorder (SCD) affects 11% of women (vs 4% of men), while Gluten Bloat Condition (GBC) affects 12% of women (vs 20% of men), participants in Windschitl 2002 reported that women are more vulnerable to SCD’s 11% risk than GBC’s 12% risk, on a 9-point scale from -4 “much more vulnerable to stomach churn discomfort” and +4 “much more vulnerable to gluten bloat condition.” This shows the 95% confidence estimates around participants’ average response.

Windschitl's experiment showed that these exaggerated, relative gists were strong enough to override the absolute risk values included in the factoids, tricking his students into believing that SCD's 11% risk was more likely than GBC's 12% risk. 

In this experiment, Windschitl only used text conditions, not charts, but later work from his lab showed similar effects where people mistake relative risk for absolute risk in bar charts. Frameworks like Valerie Reyna’s Fuzzy Trace Theory suggest these “gist” takeaways are quite common and might actually be more important for influencing decisions and behavior than more precise reads of the data.

What does this mean for dataviz design?

  1. We probably shouldn’t fuss too much about how precisely our charts convey specific values. In many cases, as long as readers take away the right gists, the chart is doing its job. 
  2. Context matters. Twelve is always greater than eleven, but depending on which other comparisons you’re making for context, it might not be remembered that way. 
  3. Twelve is always greater than eleven.

Subscribe to Effect & Affect

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.