Lady Astor, however, is not the only one who's found that predicting the future is difficult, especially when it's colored by wishful thinking. As I noted in a previous post ("How Good Are We at Predicting the Future?"), we love to predict the future but we're not very good at it. In fact, the psychologist Philip Tetlock ("Expert Political Judgment") conducted an experiment that asked nearly 280 political "experts" (economists, political scientists, intelligence analysts, journalists) to make a variety of forecasts about dozens of countries around the world. After tracking the accuracy of about 28,000 predictions over 20 years, Tetlock discovered that the experts weren't a whole lot better than the average person on the street. He also found that predictive ability did not vary by political ideology (i.e., liberals weren't any better than conservatives and vice versa), qualifications, access to classified information, or any other factor that might be thought to make a difference. This does not mean that non-experts are good predictors. Rather, it means that we're all pretty bad at it, expert and lay person alike.
However, he did find that one cluster of "predictors" did slightly better than the rest. These were those whom he called foxes as opposed to those whom he called hedgehogs. Drawing on an essay by the philosopher Isaiah Berlin ("The Hedgehog and the Fox"), foxes are those who know many things while hedgehogs are those who know one big thing. What Tetlock found was that the only experts with any real predictive insight were the foxes, those who drew on a wide variety of information and traditions to make their predictions.
Is there a moral to this story? I know I cringe when someone refers to me as an "expert" with regards to terrorist networks because I don't feel like an expert at all. The more I study, the more I realize how little I know. I just wished more academics (and the other talking heads who fill the Sunday morning talk shows) felt the same way. Indeed, I pretty much refrain from making predictions, with the one exception being Presidential elections. But even here, I draw heavily on the results of prediction markets, which reflect the opinions of thousands of people from a variety of perspectives (a "wisdom of crowds").
P.S. As another example of how expert opinion is often lacking: after Apple introduced its iPhone 5s and 5c, the company was panned for not knowing what consumers want and Apple stock dropped accordingly. As it turned out, however, Apple sold a record number of iPhones after the release of the 5c and 5s (9 million as compared to the market forecast of 6 million), leading to a huge jump in its stock price. As one market observer (Daniel Ernst) noted ("Apple Polishes Forecast After Selling 9 Million New iPhones"),
The critics have told you Apple lost its magic. Customers are telling you something very different. Clearly, people like the product. That sentiment is almost more important than the number.And the experts often don't know what they're talking about.