belief & evidence · metaphor 44 of 100
The monk's vow, the poet's meter, the scientist's Occam — all are the same move: a deliberate tax on complexity, paid up front, to generalize better later. Asceticism is not self-denial for its own sake; it is an epistemology.
People regularize because the elaborate answer is too easy. Give a theory enough clauses and it can explain anything — every awkward fact gets its own epicycle, every exception its own sub-rule. Give a life enough commitments and it can accommodate anything too: every invitation accepted, every identity kept on retainer. And accommodation is the enemy of truth, because a story that can bend around any fact was never really about the facts.
So you charge every explanatory clause rent. Not a ban — a price. Each piece of cleverness must now buy more accuracy than it costs in tax. What can't pay, drops away. What remains was earning its keep.
honest computation: 20 training points, 40 held-out points, noise σ=0.35, fixed seed · ridge solved in closed form, lasso by coordinate descent, live in your browser · features standardized on the training set; test error never touches the fit
The price of a clause
Fifteen clauses, twenty facts: the untaxed theory has almost enough freedom to thread every observation, and it uses that freedom without shame. Look at the fit at λ = 0 — it bends toward each accident of the sample as if the accident were a law. Its error on the past (train) is superb. Its error on the future (test) is several times worse, because most of what it so lovingly explained was never real. This is the quiet scandal of explanation: capacity flows to noise first, since noise is exactly the part of the data that only extra clauses can reach.
The penalty changes the economics, not the rules. Every clause may still enter the theory — but now each must reduce the misfit by more than its rent. A clause tracking a genuine pattern earns steady income from every data point and pays easily. A clause fitted to one lucky wobble earns almost nothing and defaults. Raise λ and watch the defaults: the theory becomes solvent.
What to try
Switch to L1 and ride λ up slowly from zero. The weights don't shrink in unison — clauses hit exactly zero one by one, and the first to go are the ones doing the least honest work. The order of renunciation is a ranking of usefulness.
The train curve only ever rises as λ grows — discipline always costs you the past. But the test curve dips: held-out error bottoms out at a moderate tax. Press the sweet spot and check the bars: under L1, exactly clauses 2, 5, 9 remain. The truth, recovered.
Now drag λ to the far right. Every clause is confiscated; the theory flattens to a single mumbled average. It offends no future because it predicts none. Too much asceticism is a theory that has renounced saying anything.
Toggle L2 ↔ L1 at a moderate tax. Ridge keeps all fifteen clauses, each smaller — the counter never drops below 15. Lasso keeps a few at nearly full strength and zeroes the rest. Same rent, two philosophies of paying it.
The two tax codes
L2 is temperance. It taxes the square of each weight, so the penalty bites hardest on the largest commitments; the cheapest way to pay is to shave a little off everything. Under ridge, no clause ever dies — the counter reads 15 of 15 at any finite tax. It is the moderate's discipline: keep every interest, every claim, every hedge, just smaller. Everything in moderation, including conviction.
L1 is renunciation. A flat rate on absolute size means a small clause pays the same marginal rent as a large one — so weak clauses can't hide by being modest. They go to exactly zero: vows of poverty for parameters. What survives is kept at nearly full strength. This is why traditions of excellence tend to look like lasso rather than ridge — few commitments, held hard — and why the interesting question about a serious person is what they have made themselves unable to be distracted by.
Style, vows, and Occam
Once you see the penalty term, you find it everywhere people produce work meant to outlive its occasion. Meter is a hard cap on the poet — no line may exceed the budget, whatever the sentence wants — and what it buys is the same thing the tax buys here: a poem that couldn't merely accommodate its first meaning tends to carry others. The sabbath is an L1 on the week, one coordinate forced to zero regardless of the week's pleading. And Occam's razor is the penalty stated as a philosophy: among theories that fit, prefer the one paying less rent.
The crucial word is in advance. The tax is set before the data arrive, precisely so that it cannot be negotiated down by whatever the data tempt you to say. A vow renewed daily by mood is not a vow. Regularization works because it is the one clause of the theory that the theory itself is not allowed to edit.
The mapping
| Mathematics | Life |
|---|---|
| weights wj | The clauses of your theory; the standing commitments of your life — each one a claim that this, too, matters. |
| the loss | Fitting your experience — the pull to accommodate everything that has actually happened to you. |
| λ | How heavily simplicity is paid for: the severity of the vow, the strictness of the meter, the sharpness of the razor. |
| L2 shrinkage | Temperance — everything kept, everything smaller; no passion abandoned, none at full strength. |
| L1 zeros | Renunciation — most things dropped entirely so that a few can be held without dilution. |
| the test-error dip | Why the disciplined theory meets the future better than the accommodating one — the whole reason the tax is worth paying. |
Where the metaphor tears
In the instrument, the sweet spot is found by held-out error — the discipline is calibrated against evidence of generalization, and a λ that tested worse would be abandoned without sentiment. Much human asceticism has no such feedback loop: the severity is chosen for how it looks or feels, then defended as virtue. Asceticism as cross-validation and asceticism as aesthetic are different practices that happen to share a posture, and only the first has any claim to be an epistemology.
Lasso finds the true three clauses here because the truth really is sparse — the world of this toy was built that way. If the truth were dense, with dozens of small real causes, the same tax would bury it, zeroing genuine signal and calling the burial rigor. Sparsity is a bet, not a law. A discipline only helps when the world resembles the assumption the discipline encodes; simplicity is not evidence.
In the model, the cost of a clause is a number — the same formula prices every weight. Human complexity has no such measure: whether a second language, a second career, or a second love counts as clutter or as depth is exactly what people disagree about. The math says tax complexity and let the weak clauses default; it is silent on which axis of your life is a coordinate. The metaphor tells you to renounce. It cannot tell you what.