Most coefficients are zero

A doctor faces a patient with a hundred symptoms. An analyst opens a spreadsheet a thousand columns wide. A person lies awake asking why their life feels wrong, and can name forty plausible reasons. All three are drowning in candidate causes — and all three are saved, when they are saved, by the same quiet empirical regularity: most of the coefficients are zero. The world is mostly sparse. A handful of factors carry a phenomenon and the rest are rounding error.

The art of insight is finding the few that carry the signal — and, harder, resisting the mind's hunger to keep every plausible cause alive, each defended by an anxious "but it might matter." The instrument below lets you build a phenomenon out of many candidate features, decide how many of them really matter, and then watch an honest method try to recover the few and zero the rest. It works beautifully — right up until the world stops being sparse, at which point the same method starts telling comforting lies. Both halves are shown, not hidden.

Fifty candidate features could explain the outcome. Set how many actually do — from 2 in 50 to 40 in 50 — and watch a real lasso try to recover them. Under true sparsity it catches the few and zeros the rest cleanly. As the truth turns dense, recovery frays and no small story suffices.

honest toy: 50 standardized features, n = 64 samples; a real lasso (coordinate descent with soft-thresholding) and forward selection, computed live from a seeded random phenomenon. Nothing here is faked.

1. Leave How many truly matter at 3. Watch the lasso paint three jade bars over the three ghosts and flatten everything else. This is clean recovery — essentialism vindicated: the phenomenon really did have a short cause.
2. Drag Lasso strictness λ up. Watch marginal causes wink out first, the strongest surviving longest — the razor deciding, in order, what it can bear to drop.
3. Find the elbow on the Occam curve, then set your k there. Note how little the ceiling rises past it. Adding causes has become vanity.
4. Now drag How many truly matter up toward 40. The curve straightens, the elbow marches right, and the lasso starts dropping coral bars — real causes discarded to keep the story short. Hold k at 3 and read the verdict: the clean little story is now a lie.

"What really matters here" has a bad reputation, and deservedly — it is the preferred sentence of the ideologue, the reductionist, the man with one idea. But strip the metaphysics and there is an empirical claim underneath, and it is often true: for a great many phenomena, a short list of causes carries nearly all the variance and the long tail carries nearly none. When that is so, the lasso recovers the short list without mysticism, and the elbow appears exactly where the causes run out. This is not a comforting story imposed on the data; it is a structure read off the data, testable and falsifiable.

So essentialism is rehabilitated — as a hypothesis. And it is bounded — because its validity is a bet about the phenomenon, not a law of thought. The sparse method does not assume the world is sparse; it discovers whether it is, and the residual keeps the score. The honest essentialist is the one who says "I think three things explain this" and then looks at what the three things leave out. The dishonest one skips the second clause.

Sparse thinking is a razor: prefer the explanation with fewer moving parts. But a razor with no stop cuts to the bone. The discipline has two clauses, and the second is the one people skip: prefer few causes, but check what a simple story leaves unexplained. If a three-cause story leaves half the variance in the residual, the world was dense and you are comforting yourself — the tidiness is yours, not the phenomenon's. The signal ceiling in the instrument is the honest referee: it tells you how much was ever explainable, so a large gap between your story and the ceiling is not humility, it is evasion.

The stakes are not only epistemic. "One root cause" is the grammar of every monocausal politics — the single enemy, the single reform, the single number that explains a society. Some problems really are sparse and yield to one lever. Many of the ones we argue about hardest — a personality, a civil war, why a generation is unhappy — are genuinely dense: dozens of small real causes, none dominant, none droppable without loss. Forcing a sparse story onto a dense world is not insight. It is the essentialist error wearing insight's clothes, and the residual is the tell.