The limits of prediction, and what you can do about it

As policy makers, managers, and planners, we usually try to make decisions taking account of all the possible circumstances, as best as we can understand them. That is, we try to predict what is going to happen in the future. 

And that's not so easy. A recently released report, "Driving in the Dark: Ten Propositions About Prediction and National Security" by Richard Danzig, of the Center for a New American Security, explains why, and what we can do about it. We need to assume that long-term strategies will be wrong, and our predictive capacities limited, Danzig says. “Planners need to complement their efforts at foresight with thinking and actions that account for the high probability of predictive failure.”

Written with the military in mind, Danzig makes his case in 10 propositions, five descriptive, five prescriptive. Starting with the premise that experience and social science literature demonstrate that long-term predictions are consistently mistaken, Danzig points out that in the long range national security will continue to present unpredictable issues and confound prediction, and that planning across a range of scenarios will not prevent predictive failure. These insights are equally applicable to not-for-profits and governments as well. 

So what to do? Danzig says, “Policymakers are right to attempt to enhance their understanding of the future, but such efforts need to be complemented with a better recognition of likely failures of foresight. I recommend schizophrenia: People must simultaneously predict and plan for predictive failure.” He suggests that policymakers speed up decision making when they can, but insist on making some decisions as late as possible. Adaptability should be a priority. And nurture diversity and competition, because they produce a broad range of potential responses when the unpredicted events happen. Again, these insights are generalizable to public sector and not-for-profit managers (though much of Danzig's second half is more relevant to the US military). But this paper is clearly written, and anyone with an interest in defense, or budget issues, or politics, will find it worth reading.

I found this paper because I read James Fallows' terrific blog about politics at The Atlantic.