Clustering Illusion

If you flip a coin a hundred times, you will likely see heads come up several times a row at some point. Likewise, if you set a music playlist to play songs randomly, you might hear two songs by the same artist in a row. Clusters are a natural part of randomness.

However, our brains are wired to look for patterns. This can sometimes cause us to see these clusters as non-random. The clustering illusion is the tendency to give meaning to clusters in data even if they can be explained by chance.

In data analysis, falling for the clustering illusion can cause analysts to identify trends or patterns that aren't actually significant. This can lead to wasted resources investigating something that's not meaningful. To avoid this, it's important to use statistical tests to determine whether observed patterns are meaningful or if they're likely to have occurred randomly.

Back to the Data Analysis Handbook