Simpson’s Paradox

Simpson’s Paradox is a statistical phenomenon that occurs when a trend that is seen in multiple groups disappears or even reverses when the groups are combined. Likewise, it can occur when a trend that is seen in a group disappears or reverses when the group is split into smaller ones.

Simpson's paradox is a veridical paradox because its proposition is counterintuitive but true.

The Kidney Conundrum

A study examined 700 patients who had received one of two types of treatments for kidney stones. Let's call the treatments Method A and Method B.

Method B showed a higher success rate in the group as a whole.

Then the 700 people were sorted into two groups—people with small kidney stones and people with large kidney stones.

Method A showed a higher success rate in both of these smaller groups.

How is this possible?

The Data

Whole Group

	Total Treatments	Successful Treatments	Success Rate
Method A	350	273	78%
Method B	350	289	83%

From this data, it appears that Method B is more effective. Now let's look at the split groups.

Small Kidney Stones

	Total Treatments	Successful Treatments	Success Rate
Method A	87	81	93%
Method B	270	234	87%

This data shows that Method A is more effective at treating small kidney stones.

Large Kidney Stones

	Total Treatments	Successful Treatments	Success Rate
Method A	263	192	73%
Method B	80	55	69%

This data shows that Method A is also more effective at treating large kidney stones!

Most of the people who were treated with Method A had large kidney stones, whereas most of the people who were treated with Method B had small kidney stones.

So Which Treatment is More Effective?

Large kidney stones are harder to treat than small ones. This is why Treatment A looked less effective in the large group—it was typically used for kidney stones that were harder to treat. However, when we account for this confounding variable of kidney stone size by splitting up the groups, we can see that Treatment A is more effective.

References

Charig CR, Webb DR, Payne SR, Wickham JEA. Comparison of treatment of renal calculi by open surgery, percutaneous nephrolithotomy, and extracorporeal shockwave lithotripsy. British Medical Journal. 1986;292:879–82.