The Model Habituation Problem


Economists love to talk about government deficit spending as “borrowing money from the future.” It’s a neat mental model that helps explain fiscal policy: we take on debt today to fund infrastructure, social programs, or whatever, and future taxpayers will pay it back. The model implies a simple trade-off between present and future consumption, mediated by interest rates and growth projections.

There’s just one problem: you can’t actually borrow steel from the future.

Try this thought experiment. The government decides to build a massive infrastructure program - say, rebuilding every bridge in the country. They issue bonds, borrow trillions, and start buying steel. Lots of steel. More steel than has ever been bought at once.

What happens? Well, according to the deficit model, we’re just shifting future consumption to the present. Future taxpayers will pay back the bonds, and everyone’s happy. But in reality, once you try to buy more steel than exists, the model breaks down completely. Steel prices don’t just go up - they explode. The value of your borrowed money evaporates as fast as you can spend it. You end up with less infrastructure than if you’d just bought the steel at normal prices over time.

The borrowed money becomes worthless not because of inflation in the traditional sense, but because you’ve hit a hard physical constraint that the financial model completely ignores.

Models vs. Reality

This isn’t just an economics problem. It’s a fundamental issue with how we use models in general. Models are simplifications - they have to be, otherwise they’d be as complex as reality itself and therefore useless. But the danger comes when we forget they’re simplifications and start treating them as reality.

The deficit-spending model treats money as a pure abstraction that can be moved freely across time. It ignores physical constraints, supply chains, resource depletion, and the basic fact that you can’t manufacture materials retroactively. It assumes perfect substitutability between present and future consumption, which works fine for small perturbations but falls apart when you try to do anything at scale.

As the old saying goes “All models are wrong, but some are useful”; it’s old news really and everybody is aware of this, at least in theory. The problem is you have to use some model when you observe reality, whether you are aware of it or not. This happens so naturally and is a fundamental part of the training of every human being (conditioning, really) that we tend to forget we are using a model. As consequence, the basic assumptions we make about the world tend to be implicit, popping up in a nasty way only when they break.

Paradoxically, any model that is successful enough becomes prevalent to the point of “vanishing” into the background of implicit assumptions. For example, most people believe electrons and atoms are real things; they’ll tell you with confidence that atoms are made of protons, neutrons, and electrons, and that this is simply how reality works.

Except electrons aren’t particles. They’re not waves either. They’re not anything you can visualize or directly observe. The “planetary model” of the atom is a mathematical abstraction that works well enough for introductory teaching, though modern chemistry and physics rely on quantum mechanical models for actual problem-solving. Quantum wave functions aren’t “more real” either - they’re just better for other types of problems. The electron is whatever mathematical object makes our equations work. When we measure it as a particle, we get particle-like results. When we measure it as a wave, we get wave-like results. The electron itself? That’s a question that doesn’t even make sense.

Yet somehow, people think of electrons as “real things” rather than useful mathematical abstractions and have endless futile debates about the “true nature” of electrons. The model became a Dogma.

When I studied physics in university, we were taught to attach applicability conditions to every model or problem solution. Every equation came with footnotes: “valid for small angles,” “assumes no air resistance,” “ignores relativistic effects,” “applicable only at low temperatures.” This wasn’t pedantry - it was recognition that all models are at best approximations of actual reality, because generally speaking, reality is not computationally reducible. You can’t compress the universe into a simpler description without losing information.

It’s important to acknowledge not only that models only apply locally, but also where they apply so you don’t misuse them. A model that works perfectly for small perturbations might be catastrophically wrong for large ones. A scaling model that assumes infinite resources will fail the moment resources become scarce.

Experiential reality

Load balancers and autoscaling became a commodity and people spin them up without a second thought; They assume load will evenly distribute and compute resources will pop into existence when needed - and it works well enough most of time.

Except of course for when it doesn’t… in the real world, load is probabilistic, resulting in nuanced differences between load balancing algorithms and various queueing effects. Or when the network which is supposed to be “transparent” starts acting up. People keep being surprised that their load isn’t actually well balanced, the system doesn’t linearly scale or that autoscaling doesn’t work as expected; Seasoned engineers simply sigh and roll their eyes, because they’ve learned by brutal experience in the field that things don’t work as advertised.

Good engineering is grounded in experiential reality. Engineers believe their eyes and ears, and if reality disagrees with the model then the model is wrong, no matter how beautiful and elegant it is. Because reality isn’t elegant nor beautiful; It’s messy, ugly and full of uncertainties. Thus engineers tend to develop strong intuitions which informs them successfully, extracting formal models from them mainly for cooperation with others. The history of technology is full of examples where practical engineering preceded scientific models which back them up (e.g. Steam engine). Of course, intuition is in itself an elaborate model (albeit too elaborate for verbalization), but that should not discourage us from mindfully using it.

Breaking Free from Model Habituation

The solution isn’t to abandon models entirely - that’s impossible. Instead, we need to develop constant awareness that our mental frameworks are tools, not truths. This means:

Explicitly document assumptions and constraints. Every time you apply a model, write down what it assumes about the world. What conditions must be true for it to work? What happens if those conditions change? The deficit spending model assumes elastic supply chains and infinite substitutability. The planetary atomic model assumes you don’t need to predict chemical bonds or electron behavior.

Actively seek edge cases. Don’t just look for where your model works - look for where it breaks down. What happens if you scale it up 10x? 100x? What if key assumptions are violated? The steel example breaks the deficit model not because the model is stupid, but because it hits physical constraints the model doesn’t account for.

Build redundant models. Use multiple, contradictory frameworks to cross-validate your thinking. If your scaling model says you can handle infinite load, what does queuing theory tell you? If your financial model says infinite growth is possible, what do resource constraints suggest? When models disagree, pay attention.

Remember reality exists. No matter how elegant, mathematical, or widely accepted your model is, reality doesn’t care about your equations. The model isn’t reality, no matter how detailed or beautiful it might be and reality is always more complex than we imagine.

The next time you catch yourself thinking “that’s just how things work,” pause and ask: what model am I unconsciously applying here? What are its limits? What assumptions am I making about reality? What would happen if I pushed it to an extreme?

Because somewhere out there, reality is waiting to remind you that you can’t actually borrow steel from the future.

systems-thinking
comments powered by Disqus