The Signal and the Noise, Nate Silver.
Penguin Press, 2012
In the months before the 2012 election, conservatives launched an all-out assault against the polls. Despite a sluggish economy and a disastrous first debate, polls showed President Barack Obama maintaining a consistent, if narrow, lead over Mitt Romney. Such numbers contradicted the Right’s conviction that the election was a referendum on Obama’s failed presidency.
So they revolted. Insisting pollsters were oversampling Democrats, right-wing pundits predicted a Romney landslide. Even sober analysts like George Will of the Washington Post declared Romney would win by more than 100 electoral votes. On election night, campaign analyst Karl Rove attempted to explain to Fox News viewers why Romney would win, despite ominous early results. Anchor Megyn Kelly responded with the question of the year: “Is this just math that you do as a Republican to make yourself feel better, or is this real?”
Nate Silver’s book The Signal and the Noise is about precisely that: the math we do to make ourselves feel better. Like Karl Rove’s, our therapeutic math may not be real, but it has real consequences. From blown poker bets to the global financial collapse, our inability to set aside our biases when predicting the future can have catastrophic effects.
Which is where Silver comes in. Nate Silver became a household name in 2008 thanks to FiveThirtyEight.com. The website aggregates and analyses polls, then generates the probability a candidate will win. In 2008, Silver correctly predicted the winner of the presidential race in 49 of 50 states.
In 2012, Silver’s model continued to forecast an Obama win even in early October, when polls tightened and conventional wisdom held the race was tied. This led pundits across the political spectrum to question his objectivity. Yet Silver stuck with his model, and bettered his 2008 record by accurately predicting the results in all 50 states.
The Signal and the Noise, however, is not an end-zone dance (it came out before the 2012 election). It is instead a nuanced study of the role predictions play in our lives. As he ranges from climate change to basketball tournaments to swine flu, Silver uncovers a noisy world full of false leads and meaningless patterns that hide the signal — the reality — we are trying to track.
Moreover, he argues, the world is getting noisier. While “the era of Big Data” will in time enhance our predictive capabilities, at the moment it generates massive volumes of noise. Sorting through that noise — separating good information from bad — is the challenge of our age.
Complicating that task is our tendency to seek out data that confirms our biases. Daniel Patrick Moynihan famously said, “Everyone is entitled to his own opinion, but not his own facts.” In the era of Big Data, though, that’s no longer the case. With 2.5 quintillion bytes of new information produced daily, it is not hard to pull together numbers that support our arguments. Republicans had no trouble compiling evidence of a hidden Romney majority. All they had to do was treat certain polls as bellwethers rather than outliers, et voila! A Romney landslide was imminent.
Silver accepts that we bring biases to our predictions. “Pure objectivity,” he writes, “is desirable but unobtainable in this world.” What he has little time for are peddlers of predictions who play on these biases. In an early chapter on pundits, Silver notes television experts get predictions right about half the time — the same as flipping a coin.
Punditry, it turns out, is bad for accuracy. Drawing from a 2006 Philip Tetlock study, Silver delivers bad news for academics crossing over into commentary. Tetlock found the more academics appeared in media, the worse their predictions became. The reason? Punditry values entertainment, boldness, and contrariness far more than accuracy. If an expert happens to call it right, that’s great, but accuracy is the icing, not the cake.
Weather forecasters face the same problems. Meteorology is the big success story in Silver’s book. The combination of scientific models and human adjustments has made weather predictions the gold standard for probability-based forecasts. But local forecasters and weather websites introduce inaccuracies to boost ratings and (ironically) credibility. Thus the “wet bias”: over-predicting rain because viewers are more forgiving of showers when there’s a 30 per cent chance rather than a 10 per cent chance.
But a botched call on election night or a rained-out picnic pales in comparison to other inaccurate predictions. Over-predicting the effects of climate change damages scientists’ credibility and weakens public commitment to policy interventions. Under-predicting the risk of mortgage defaults triggered a global financial collapse. So figuring out the biases that skew our predictions is a matter of some urgency.
Silver dedicates the second half of his book to improving forecasts. It is this project that transforms The Signal and the Noise from a study of predictions into a roadmap for change. Because Silver believes in probability modelling — that is, determining the chance something will happen rather than a simple yes-it-will or no-it-won’t — he advocates we become Bayesians.
Bayesian reasoning requires users to incorporate past beliefs into future predictions. Silver embraces this approach because it requires us to recognise our biases. Bayesian models begin with a statement of our assumptions: the first step toward accounting for them. As Bayesians, we can no longer fool ourselves about our objectivity or claim “the numbers speak for themselves”.
To some extent, then, The Signal and the Noise is the ultimate self-help book. Silver is suspicious of experts and wary of government regulation. Our fate, he tells us, lies not in our leaders or institutions but in ourselves. A more predictable, data-driven future is possible, but it first requires better self-knowledge. “Before we demand more of our data,” Silver writes, “we need to demand more of ourselves.” Bayesian reasoning may not have prevented Mitt Romney’s loss in November. But it would have saved Rove and Republicans the public meltdowns and mea culpas that followed Obama’s victory. What’s more, finding the signal through the noise would give us a clearer picture of things as they are, better equipping us to advance our vision of how things ought to be.