Why Models Fail

Why are the models we use, not always fit for their purpose?

Economics, like all models, is an attempt to simplify something complex. Yet like the map, economics is not the territory. Human behaviour can’t easily be categorised into defined concepts like supply and demand. Every single one of us breaks the tenets of economics as we go about our daily lives. We’re not Homo economicus; consistently rational, narrowly self-interested people who pursue their subjectively defined ends optimally. We make choices based on emotion. As a result, we sometimes buy things we don’t need, often overpaying for them in the process.

Accountancy is another example of a model that often doesn’t reflect reality. By way of example, we can think about depreciation; the idea that we spread the cost of an asset over its useful economic life. In theory, it makes perfect sense. But in practice, when we buy a car, we’re likely to get far more enjoyment out of it when it’s shiny and new, than when we’ve had it for a while.

The real world does not neatly segment in the way that we’re taught it in school. It is convenient to base our learning on linear silos.

What is an example of a model that turned out to be flawed?

The Financial Services industry saw no need to regulate benchmark indices like LIBOR. Until it transpired that the people who were responsible for providing submissions, were behaving in an unethical way. The model that presumed that people would behave ethically, and didn’t need to be controlled, turned out to be fatally flawed.

What factor in model design makes them flawed?

Traditions, like models, are based on the human experience of the past. The data that we use to design, and in the case of algorithms, power, those models all come from one source: history. Yet we know that historical data is heavily biased.

The dangers of inferring the future by looking at the past, are well known. Financial services products are sold with a warning that past performance is not indicative of future results. Yet all too often we ignore this advice in other fields.