the second hundred · metaphor 131
Two explanations fit what happened equally well: a plain one and an intricate one, full of special conditions and exactly-timed coincidences. Some instinct says trust the plain one. Where does that instinct come from — and when is it wrong?
The elaborate excuse paid the Bayesian Occam tax, charged automatically for every extra degree of freedom it claimed. This is the surprise: the preference for simpler explanations isn't a rule of thumb bolted on for taste or tidiness. It falls out of probability itself. An explanation that could have accounted for almost any outcome has spread its bets thin; when one particular thing happens, it gets little credit for it. The plain story, which predicted roughly this and little else, collects.
And the tax is automatic. Nobody imposes it; no one decides the flexible story is "cheating." It simply must spread its finite belief across all the outcomes its extra freedom allowed — so it has less to spend on the one that occurred. Simplicity wins by default, and keeps winning, right up until the data are strange enough that only complexity can explain them at all.
The idea
A model's evidence — its marginal likelihood — is the probability it assigned to the data you actually saw, averaged over every setting of its parameters its prior allowed. That averaging is the whole story. A simple model, with few knobs, makes sharp predictions: it bets heavily on a narrow range of possible datasets. A flexible model, with many knobs, can fit almost anything — so it must spread the same total probability across a vast range of possible datasets, leaving each one only a sliver.
Because probability must sum to one, breadth is expensive. The flexible model can always be tuned to fit the data better after the fact — its best-fit residual is smaller. But the evidence doesn't ask how well it fits once tuned; it asks how much probability it committed to this data in advance, across all its possible tunings. That is the Occam factor: an automatic penalty, in the mathematics, for every extra degree of freedom claimed.
The instrument computes both models' evidence exactly — no simulation, closed form — and shows the tug-of-war in the ledger below. One bar is the extra fit the flexible model buys; the other is the Occam tax it pays for its freedom. The longer bar wins. When the truth is plain, the tax dwarfs the fit; the simple story takes the evidence. The verdict bar splits belief between them accordingly.
What to notice
Watch the two ledger bars as you drag the truth's complexity. The Occam tax barely moves — it's a property of the flexible model's freedom, not of the data, and it's there even when the truth is a straight line. What changes is the extra fit: near zero when the simple story already explains everything, growing as the truth sprouts wiggles the simple model literally cannot bend to. The verdict flips at the exact moment the earned fit overtakes the standing tax. Simplicity isn't preferred a little — it's preferred until the data force the issue.
Now push the knobs slider up: give the flexible model more freedom and its tax climbs, so it needs even stronger evidence to justify itself. And drag N: with more data, real structure becomes unmistakable and the fit term grows faster than the tax, so genuine complexity eventually gets its due — but a flexible story still can't win on a handful of points. Press Draw new data and notice the verdict scarcely trembles: this isn't luck in one sample, it's the shape of the evidence itself.
The mapping
The elaborate excuse — the traffic, and then the phone died, and my sister called with an emergency, and the one road I know was closed — fits the evidence perfectly. So does I lost track of time. The instinct to trust the second isn't anti-intellectual suspicion; it's the Occam factor felt in the body. The intricate account had to reserve some plausibility for each of its many moving parts — each of which could have failed to happen — so it spent its credibility thin. The plain account predicted almost exactly this, and little else, and so it collects. The more special conditions a story needs, the more finely it has to slice its own believability.
But the tax is not a verdict against complexity — it's a burden of proof, and it can be met. When the data really are intricate — a life that genuinely turned on a chain of unlikely events, a theory that simpler ones provably fail to explain — the fit the flexible story buys grows until it overwhelms the tax, and the evidence rightly swings to it. That is the whole discipline: prefer the simple story by default, demand that any added complexity pay for itself in fit, and let it win the moment it does. Not skepticism, not credulity — an exchange rate between simplicity and surprise.
Read as life lessons
A story that could explain anything explains this thing only weakly. Every extra "it depends" and special case is a claim on believability you have to pay for up front.
The flexible model always fits the past better once tuned. Evidence ignores that and asks what you'd have bet beforehand — the honest test the elaborate excuse quietly fails.
When the data truly demand it, the added fit overwhelms the tax and complexity rightly wins. Simplicity is the default, not a dogma — the burden of proof, not a verdict.
In the wild
Bayesian evidence chooses how many clusters, factors, or spline terms a dataset supports — the Occam factor stopping the fit from sprouting parameters the data can't justify.
Comparing theories by evidence rather than best fit is how a simpler cosmological model can beat a more elaborate one that fits marginally better — the tax formalizes Occam's razor.
BIC and related criteria are cheap stand-ins for the Occam factor — a per-parameter charge subtracted from the fit, echoing the exact tax the marginal likelihood levies in full.
The mapping, exactly
| Mathematics | Life |
|---|---|
| the marginal likelihood | How much a story committed, in advance, to the thing that actually happened. |
| a flexible model's wide prior | An explanation with many "it depends" — able to fit almost any outcome after the fact. |
| the Occam factor | The credibility tax an elaborate account pays for reserving plausibility across all its parts. |
| best-fit residual | How well the story matches the past once you tune it — a test it can always game. |
| evidence = fit − Occam tax | Trust the plain story unless the added complexity pays for itself in genuine explanation. |
| the verdict flipping | The moment the data are strange enough that only the intricate account can explain them. |
The honest model
Both stories are linear models in a cosine basis on [0,1]. The simple one has two terms (a level and one gentle wave); the flexible one adds higher-frequency waves — the "knobs" slider sets how many. Coefficients carry a Gaussian prior β ~ N(0, α²I), the noise is Gaussian with known σ. For a linear-Gaussian model the marginal likelihood is exact: the data are distributed y ~ N(0, C) with C = σ²I + α²ΦΦᵀ, and the panel computes log p(y) = −½(yᵀC⁻¹y + log|C| + n·log 2π) by Cholesky — no Monte Carlo, no fudge.
The ledger decomposes that evidence honestly. It first computes each model's best-fit log-likelihood — the maximum-likelihood residual, the flexible model's home advantage. The Occam tax is exactly how much the evidence discounts that raw fit: tax = best-fit − log-evidence, always positive, larger for the freer model. The two ledger bars are the gaps between the models — the extra fit the flexible one buys (best-fitₓ − best-fitₛ) versus the extra tax it pays (taxₓ − taxₛ). Their difference is precisely the log Bayes factor driving the verdict bar, which shows the posterior model probabilities under equal prior odds.
Where the metaphor tears
The Occam factor is set by how wide the flexible model's prior is — how much freedom you granted it. Narrow that prior and its tax shrinks; widen it and the tax balloons. There is no assumption-free razor: "how complex is too complex" always smuggles in a prior about what you expected complexity to look like. The math is exact; its inputs are a judgment.
The evidence rewards the story that predicted the data best on average, which is often the plain one — but the world is under no obligation to be simple. Occam is a bet that pays off frequently, not a law. Sometimes the elaborate excuse is exactly what happened, and a mind too fond of the tax will keep disbelieving true, intricate accounts because they were a priori unlikely.
The clean tax requires a tidy space of models and a known noise law. A human explanation lives in an open-ended, ill-defined hypothesis space where you can't enumerate the alternatives or their priors. The instinct the metaphor names is real and often right — but the crisp number on the panel is a gift of the toy model, not something you can actually compute about your sister's excuse.