A Modern Guide to Thinking, Fast and Slow

Return to Guide

Part II - Heuristics and Biases

10. The Law of Small Numbers
11. Anchors
12. The Science of Availability

The rest of the chapters will be released soon.

Chapter 10: The Law of Small Numbers

Overview
Small samples are less reliable than large ones and often produce more extreme results purely by chance. For instance, both the lowest and highest cancer rates are found in small rural counties. A common mistake is to assume that small samples are just as trustworthy as large ones. Kahneman calls this the belief in the law of small numbers. Closely related is the clustering illusion: our tendency to see streaks or clusters in random data as meaningful rather than inevitable outcomes of small samples.

Replication & Reliability

• Belief in the law of small numbers (Tversky & Kahneman, 1971): A 2023 replication of the 1971 study (still in preprint) replicated most findings except for the finding that people’s tendency to underestimate the required sample size in a replication.
• The “hot hand” illusion (Gilovich, Vallone, & Tversky, 1985): The conclusion that "hot hands" are an illusion has been challenged numerous times. Many recent analyses, such as a 2021 study based on 34 years of NBA data, show evidence of real streaks. However, the effects do tend to be smaller than players and fans believe. A fascinating 2016 paper demonstrates that Gilovich used a biased estimation procedure.

Recommendation

This chapter is important. The belief in the law of small numbers explains why flashy findings in small studies are often misleading. While the “hot hand” effect is controversial, the broad point that we underestimate how much variability naturally arises when working with small samples holds up.

Chapter 11: Anchors

Overview
The anchoring effect is a cognitive bias in which people place too much weight on the first piece of information they encounter (the “anchor”) when making judgments or decisions. This can happen even if the anchor is random and obviously irrelevant to the decision or judgement.

Replication & Reliability

• Classic anchoring: Numerous studies since the 1970s confirm robust anchoring effects across contexts. A 2023 literature review of anchoring studies found that “the anchoring effect is a pervasive judgment bias when people are in an uncertain decision-making environment.”
• Adjustment mechanism (Epley & Gilovich): A 2022 preregistered replication of three Epley & Gilovich studies found that anchoring effects were robust, but found no evidence that need for cognition, cognitive load, and forewarning moderate anchoring: “In line with recent replication efforts, we found that anchoring effects were robust, but the findings on moderators of anchoring effects should be treated with caution.” Note: It is still in preprint.

I haven’t found replications of the real-estate agents valuation study or the German judges & loaded dice sentencing study, nor direct replications of the Strack & Mussweiler experiments. If you know of any, please email me.

Recommendation

The anchoring effect itself is supported by robust evidence, but some of the striking real-world illustrations from this chapter (real-estate agents, judges) have not been extensively studied and should be treated as provisional.

Chapter 12: The Science of Availability

Overview
When judging how common or likely something is, people often answer a substitute question: “How easily do examples come to mind?” This is called the availability heuristic. Vivid news, personal anecdotes, and memorable events impact our judgment more than base rates. How easily examples come to mind is more influential than how many examples one can think of.

Replication & Reliability

• Marriage contributions: This part comes from a 1979 study by Ross & Sicoly. They found that individuals remember their own contributions to a shared product more easily (not only in terms of marriage, but on basketball teams and in laboratory groups, too). While I have not found an exact replication of this study, numerous later studies support the general principle that people tend to accept more responsibility for a group's outcome than other members attribute to them. Interestingly, a 2005 study found people’s tendency to do this can be attenuated “when they ‘unpack’ their collaborators, conceptualizing them as separate individuals, rather than as ‘the rest of the group.’” A 2016 study found that the effect increases with group size.
• Schwarz and colleagues’ 1991 study about recalling assertiveness: This study found that the ease of recalling examples mattered more than the number recalled, but it was had a small sample size and statistically weak results. The problems with this study are explained in more depth here. However, there have been many studies that support this basic effect. A 2017 meta-analysis confirmed the effect is real, but the average size is smaller and more variable than early studies suggested. It also found evidence of publication bias.

Recommendation
The studies Kahneman cites point to real psychological effects: people do overestimate their own role in groups, and the ease of recalling examples shapes judgments. But these effects are smaller than the book suggests.

Return to Guide