Simpson’s Paradox
Simpson’s Paradox is a statistical phenomenon that occurs when a trend that is seen in multiple groups disappears or even reverses when the groups are combined. Likewise, it can occur when a trend that is seen in a group disappears or reverses when the group is split into smaller ones.
Simpson's paradox is a veridical paradox because its proposition is counterintuitive but true.
The Kidney Conundrum
A study examined 700 patients who had received one of two types of treatments for kidney stones. Let's call the treatments Method A and Method B.
Method B showed a higher success rate in the group as a whole.
Then the 700 people were sorted into two groups—people with small kidney stones and people with large kidney stones.
Method A showed a higher success rate in both of these smaller groups.
How is this possible?
The Data
Whole Group
Total Treatments | Successful Treatments | Success Rate | |
---|---|---|---|
Method A | 350 | 273 | 78% |
Method B | 350 | 289 | 83% |
From this data, it appears that Method B is more effective. Now let's look at the split groups.
Small Kidney Stones
Total Treatments | Successful Treatments | Success Rate | |
---|---|---|---|
Method A | 87 | 81 | 93% |
Method B | 270 | 234 | 87% |
This data shows that Method A is more effective at treating small kidney stones.
Large Kidney Stones
Total Treatments | Successful Treatments | Success Rate | |
---|---|---|---|
Method A | 263 | 192 | 73% |
Method B | 80 | 55 | 69% |
This data shows that Method A is also more effective at treating large kidney stones!
Most of the people who were treated with Method A had large kidney stones, whereas most of the people who were treated with Method B had small kidney stones.
So Which Treatment is More Effective?
Large kidney stones are harder to treat than small ones. This is why Treatment A looked less effective in the large group—it was typically used for kidney stones that were harder to treat. However, when we account for this confounding variable of kidney stone size by splitting up the groups, we can see that Treatment A is more effective.
References
Charig CR, Webb DR, Payne SR, Wickham JEA. Comparison of treatment of renal calculi by open surgery, percutaneous nephrolithotomy, and extracorporeal shockwave lithotripsy. British Medical Journal. 1986;292:879–82.
Courses
Fallacy Detectors Part 1
Develop the skills to tackle logical fallacies through a series of 10 science-fiction videos with activities. Recommended for ages 8 and up.
A Statistical Odyssey
Learn about common mistakes in data analysis with an interactive space adventure. Recommended for ages 12 and up.
Symbolic Logic for Teens Part 1
Learn how to make sense of complicated arguments with 14 video lessons and activities. Recommended for ages 13 and up.
Emotional Intelligence
Learn to recognize, understand, and manage your emotions. Designed by child psychologist Ronald Crouch, Ph.D. Recommended for ages 5 and up.
Worksheets
Symbolic Logic Worksheets
Worksheets covering the basics of symbolic logic for children ages 12 and up.
Elementary School Worksheets and Lesson Plans
These lesson plans and worksheets teach students in grades 2-5 about superstitions, different perspectives, facts and opinions, the false dilemma fallacy, and probability.
Middle School Worksheets and Lesson Plans
These lesson plans and worksheets teach students in grades 5-8 about false memories, confirmation bias, Occam's razor, the strawman fallacy, and pareidolia.
High School Worksheets and Lesson Plans
These lesson plans and worksheets teach students in grades 8-12 about critical thinking, the appeal to nature fallacy, correlation versus causation, the placebo effect, and weasel words.
Statistical Shenanigans Worksheets and Lesson Plans
These lesson plans and worksheets teach students in grades 9 and up the statistical principles they need to analyze data rationally.