91av

Choosing your subject is key to accurate predictions, argues Nate Silver in The Signal and the Noise, but is playing it safe always a good thing?

THE pretensions of economics as a science have taken a mighty knock since the financial crisis of 2008. Most economists blithely ignore this great global experiment, and have so far resisted revising their underlying theory.

Yet the financial crisis has engrossed others, among them psychologists and even some philosophers of science. It is the last group, to some extent, that Nate Silver aspires to join in writing The Signal and the Noise.

He starts by discrediting the “efficient market hypothesis”, the notion that a market price (of a share, for example) is rational and “true” because it necessarily reflects the sum of available knowledge. It’s great for modelling equilibrium, but panic and herd instinct prevail when the balance tips, resulting in deep, long-lasting busts.

But if Silver’s book was provoked by the hubris of model-builders who predicted everlasting stability, it evolves into something deeper and wider: a meditation on uncertainty, probability and humility. When the data and theories don’t permit predictions, he argues, we should hold back, no matter how big the media clamour or corporate inducements.

“When the data and theories don’t permit predictions, we should hold back”

Silver uses examples taken from his day job crunching electoral data (see “The US presidential election is no contest“) and his sideline in quantitative analysis of baseball – two fields where modest predictability works OK. A former professional poker player, he races through a witty personal account of the odds of holding a winning hand.

The examples are vivid, though readers are sprayed with extraneous detail. To convince us that he talked to his interviewees, Silver tells us that one of them (a professional sports gambler) has five flat-screen Samsung TVs on his wall; another (Donald Rumsfeld, US defence secretary during the Iraq war) is recounted to be precisely five foot seven inches tall.

Underneath, Silver is struggling to say something profound, but he can’t quite decide what. Meteorologists can save lives by predicting the course of hurricanes, but geologists can’t do the same with earthquakes. Does he lament this as a deficiency, or should we celebrate the limits of predictive science?

A couple of chapters shiver on the edge of deep philosophical waters. Do we even need past observations of data to make probabilistic predictions, he wonders. He seems to favour inductive reasoning but does not say what the knowledge built on such foundations would look like. If his message is “bow in awe at prevalent uncertainty”, wouldn’t it discourage science, especially the classic Einstein or Higgs kind, where the gap between theory and empirical corroboration can be decades long?