Superforecasting: The Art and Science of Prediction

When someone says "the dollar is about to skyrocket" or "that team is getting relegated," they're making a prediction. We do this all the time, at work, in politics, in sports, with our wallets. Philip Tetlock and Dan Gardner open the book with an uncomfortable question: does anyone check whether these predictions come true?

Most of the time, no. Commentators and analysts speak with confidence, nobody comes back later to hold them accountable, and the audience forgets. Since the bill never comes due, people who get it wrong keep getting invited back, and the ones who speak most dramatically tend to show up the most.

The book wants to change that. It shows that forecasting is a skill that can be measured, trained and improved, as long as you accept two things: turning opinion into numbers and checking the result. The thesis is optimistic, but it has limits that we'll see in the critique.

The origin of the book is an earlier study by Tetlock, begun in the 1980s, that followed hundreds of experts in politics and economics and gathered tens of thousands of predictions over about twenty years. Then he compared what they said with what actually happened.

The result made headlines: on average, the experts did only slightly better than chance, and the famous image of the "dart-throwing chimpanzee" caught on. It was a metaphor, not an experiment with animals, but it illustrates the point: plenty of respected people forecast only a little better than a coin toss. The more famous the expert, the worse they tended to be.

There's an important nuance the authors keep repeating: the average hides differences. Some did much better than others, and the question became what they were doing differently. That question is where the second part of the story begins.

The starting point of the answer comes from an essay by the philosopher Isaiah Berlin: the fox knows many small things, and the hedgehog knows one big thing. Tetlock used the image to describe two styles of thinking. The hedgehog sees the world through a single theory. The fox mixes several.

The hedgehogs were more confident, clearer and more often invited on TV, because a strong thesis makes a good soundbite. The foxes were more cautious, said "on one hand, on the other," and gave less flashy answers. But when it came time to check, the foxes were right more often.

Think of the economist who explains everything with the same factor, or the pundit who thinks every election comes down to one issue. When a single explanation works for everything, be suspicious. The book defends the humility of the fox, even if it sells worse on the campaign stage.