The Undoing Project: A Friendship That Changed Our Minds by Michael Lewis, W. W. Norton & Company, December 2016, ISBN 978-0393254594
Forty years ago, the ground-breaking research of Israeli psychologists Daniel Kahneman and Amos Tversky on the decision-making process undid our assumptions about the decision-making process. Their research showed the ways in which the human mind errs, systematically, when forced to make judgments in uncertain situations. They created the field of behavioral economics, revolutionized Big Data studies, and advanced evidence-based medicine. Kahneman and Tversky have done more than anybody else for the powerful trend to mistrust human intuition and defer to algorithms. In The Undoing Project, Michael Lewis says it was a compelling collaboration between two men who have the dimensions of great literary figures. They were one of the greatest partnerships in the history of science, working together very closely. In the process, they changed mankind’s view of its own mind.
Lewis says their work teased out the implications of a single mental error that people commonly made – even when those people were trained statisticians. People mistook even a very small part of a thing for the whole. Even statisticians tended to leap to conclusions from inconclusively small amounts of evidence. They did this, Amos and Danny argued, because they believed – even if they did not acknowledge the belief – that any given sample of a large population was more representative of that population than it actually was.”
The power of the belief could be seen in the way people thought of totally random patterns – like those created by a flipped coin. People knew that a flipped coin was equally likely to come up heads as it was tails. But, Lewis says, they also thought that the tendency for a coin flipped a great many times to land on heads half time would express itself if it were flipped only a few times – an error he calls “the gambler’s fallacy.” People seemed to believe that if a flipped coin landed on heads a few times in a row it was more likely, on the next flip, to land on tails – as if the coin itself could even things out. Lewis writes, “Even the fairest coin, however, given the limitations of its memory and moral sense, cannot be as fair as the gambler expects it to be,” they wrote. In an academic journal that line counted as a splendid joke.”
They then went on to show that trained scientists — experimental psychologists were prone to the same mental error. For instance, the psychologists who were asked to guess the mean IQ of the sample of kids, in which the first kid was found to have an IQ of 150, often guess that it was 100, or the mean of the larger population of eight graders. They assumed that the kid with the high IQ was an outlier with an extremely low IQ – that every heads would be followed by a tails. But the correct answer – as produced by Bayee’s theorem – was 101. Lewis says even people trained in statistics and probability theory failed to intuit how much more variable a small sample could be than the general population — and that the smaller the sample, the lower the likelihood that it would mirror the broader population.
The Undoing Project is the best-told story of two great friends who revolutionized behavioral economics. Like a true storyteller, Lewis has told this story like nobody else could. He has explained the theories of Kahneman and Tversky in such a way that even a layman would easily understand. Reviewed by Jonathan T. Rich