A reader writes:
As a former policy modeler myself (in the energy sector), I think that Ken Knight's point about the "complexity bias" is a big part of the reason we don't use models. But another problem with these complex models is the fact that because they are so complex, it can often be hard to tease out the various assumptions that went into generating the results. This is a problem because for many of these models, an irresponsible modeler could make the model say whatever they want, and figuring out whether or not this was done is like looking for a needle in a haystack.
In my experience, even when my old firm used Excel so that anyone could open, run, and tweak the model and avoided using any macro code to increase the simplicity and transparency, there is no one who understands the model like the modeler themselves. This means that it can be nearly impossible to distinguish a good model from a bad model: critics of a model's results can easily cast doubt on those results by criticizing the process and inputs.
Some areas may have more flexibility than others, but when a model has to go through a public process, the end product can often diverge wildly from the initial model though attempts to balance the interests of various groups. This balancing is influenced by the "expected outcomes" that various people have in mind, so that even when you start with a good, objective model, you might end up with a model that primarily serves to support preexisting notions of what the model should say.
Another writes:
Isaac Asimov beat everyone to the punch on this - it's called Psychohistory. If you're not familiar with the Foundation novels, a mathematician named Hari Seldon develops psychohistory as a means of predicting the future path of the Galactic Empire. Of course, it relied upon a massive sample size. On the idea of future-forecasting, Asimov himself had this to say in an interview with Terry Gross (quoted on the afore-linked Wikipedia page):
Gross: Do you think that would be good if there really was such a science?
Asimov: Well, I can't help but think it would be good, except that in my stories, I always have opposing views. In other words, people argue all possible… all possible… ways of looking at psychohistory and deciding whether it is good or bad. So you can't really tell. I happen to feel sort of on the optimistic side. I think if we can somehow get across some of the problems that face us now, humanity has a glorious future, and that if we could use the tenets of psychohistory to guide ourselves we might avoid a great many troubles. But on the other hand, it might create troubles. It's impossible to tell in advance.
The Dish 