the second hundred · metaphor 128
An experience arrives as a flood of unrepeatable detail. You cannot store all of it, and you don't try. So which parts are worth keeping — and which can you let fall away without losing anything that matters?
Her memory kept only sufficient statistics: the few numbers that held all an experience's signal, and let the rest fall away. It is a strange kind of grace — to forget almost everything about a stretch of your life and yet be able to answer, just as well as if you'd kept it all, the one question you actually carried forward. How many times did that happen. How did it usually go. How much did it vary. The raw days are gone; the verdict is intact.
This is not a trick of a good memory — it is a mathematical fact about certain questions. For a whole family of models, a handful of running totals carry every last bit the raw data holds about what you're trying to learn. Keep those, throw the rest away, and your best guess does not shift by a hair. The catch, of course, is which handful — and for which question.
The idea
Suppose you want to learn one thing — call it the signal μ — from a stream of noisy readings. The naive fear is that every reading you throw away is a little knowledge lost, so a careful mind must hoard them all. For many models that fear is simply wrong. There exists a short list of running totals — a sufficient statistic — such that, once you know those totals, the individual readings tell you nothing more about μ.
Here the readings are Gaussian, scattered around μ. Two quantities are enough: the count n and the sum Σx. Everything your best guess about μ could ever depend on flows through those two numbers. The instrument proves it the hard way: one engine recomputes its belief by grinding over the whole stored array of raw samples; the other keeps only n and Σx. The two belief curves are laid on top of each other, and the gap between them — measured live — is zero to the last decimal a computer can hold.
So press Forget the raw data. The stored dots are shredded; the raw engine goes dark, having nothing left to grind. The summary engine doesn't flinch — it keeps drawing the very curve the raw data would have drawn, and it keeps learning from new samples, because a new reading only ever nudges n and Σx. Two numbers, doing the work of a thousand.
What to notice
Keep drawing and watch the raw data kept meter climb — one float per sample, forever — while the summary kept meter sits flat at two. Both deliver identical beliefs; only one has a memory bill that grows without bound. That is the whole bargain of a sufficient statistic: constant memory, zero loss, for the question it was built to answer.
Now watch the third curve. A lossy memory keeps only its first twelve impressions and then stops looking. Early on it tracks the others; but as real evidence pours in, it freezes — its belief stays wide and a little off while the sufficient summary sharpens past it. This is the counter-lesson that makes "sufficient" mean something: not any summary will do. Keep the wrong numbers, or too few, and you stop learning. The art is keeping exactly the ones that carry the signal — no more, and crucially no less.
The mapping
We are told to cherish detail, and we can't. A childhood is millions of moments; what survives is a few load-bearing summaries — I was mostly safe; they mostly came when I cried; it varied more than I'd like. The dread is that forgetting the particulars means losing the truth of them. The sufficient-statistic view offers an unexpected mercy: for the questions you actually carry forward, the particulars were redundant. The verdict they support is preserved perfectly in the summary. You did not betray the experience by forgetting it; you distilled it.
The discipline the metaphor asks is honesty about the question. A sufficient statistic is sufficient for something — a specific thing you want to know, under a specific model of how the world works. Keep the totals that answer "how often, how much, how variable," and you can shed the rest with a clear conscience. Keep the wrong souvenir — a single vivid moment mistaken for the whole — and you have a lossy memory that stopped learning in month one. The skill of a life is not remembering everything. It is knowing which few numbers held the signal.
Read as life lessons
For the right question, discarding the raw days costs you nothing — the summary answers exactly as the full record would. Grief that a moment is gone assumes it carried signal it may not have.
A summary is enough only for a question. Change what you need to know — the spread, not just the average — and yesterday's "enough" is suddenly missing a number.
An insufficient memory — the wrong souvenirs, the first impression clung to — stops updating. It feels like wisdom and behaves like a bias that no new evidence can move.
In the wild
Databases and sensors that can't store every event keep running totals — count, sum, sum-of-squares — and reconstruct means and variances exactly, on unbounded data, in fixed memory.
Gaussians, Poisson, Bernoulli, and their kin all admit finite sufficient statistics. It is why so much of classical inference reduces to a handful of sums — the models were built that way.
Releasing only sufficient statistics can share all a dataset's signal about a parameter while withholding the raw records — the same principle, turned toward what is safe to forget on purpose.
The mapping, exactly
| Mathematics | Life |
|---|---|
| the raw sample x₁…xₙ | Every unrepeatable detail of an experience, exactly as it happened. |
| a sufficient statistic T(x) | The few numbers that carry all the signal the question you carry forward will ever need. |
| the factorization p(x|θ)=g(T,θ)·h(x) | The leftover detail h(x) is real, but tells you nothing further about the thing you're deciding. |
| posterior depends on data only through T | Two different pasts that summarize the same leave you believing exactly the same thing. |
| an insufficient summary | A memory that hoards the wrong souvenirs and quietly stops learning from what comes after. |
| sufficiency is relative to (model, question) | What's worth keeping depends entirely on what you'll later need to decide. |
The honest model
The samples are drawn from a normal distribution N(μ, σ²) with σ known; the goal is to learn μ under a conjugate normal prior N(μ₀, τ₀²). The posterior over μ is again normal, with precision 1/τ₀² + n/σ² and mean (μ₀/τ₀² + Σx/σ²) / precision. Read it off: the data enter only as n and Σx. That is the Fisher–Neyman factorization made concrete — the likelihood splits as g(n,Σx; μ)·h(x), and everything about μ lives in the first factor.
Nothing on this page is asserted; it is measured. The "raw" engine actually stores every sample and re-sums the array each frame; the "summary" engine holds only n and Σx. Both feed the same posterior formula, and the panel prints the largest gap between the two belief curves — it hovers at the machine epsilon, ~10⁻¹⁶, the rounding dust of floating point, not a real disagreement. Forget the raw data and the array is genuinely discarded; the two-number engine carries on because the formula never needed more. The lossy third curve freezes its totals at the twelfth sample, and you watch a belief that has stopped learning.
Where the metaphor tears
Keep only n and Σx and you can never recover how spread out the experience was — for that you'd have needed Σx² too. A life keeps revisiting old experiences with new questions, and the summary that was "enough" is suddenly missing the number the new question needs. What you're right to forget depends on a future you can't fully see.
The two-number summary is sufficient because the world here is exactly Gaussian. Let the readings have heavy tails, outliers, or structure the model never anticipated, and those discarded raw values were carrying real information after all. A summary is only as trustworthy as the story you assumed when you chose it.
Inference treats the particulars as interchangeable draws around a signal. But the texture of a specific afternoon — the thing memoir is made of — is not redundant to any parameter; it is the value. Distilling an experience to its sufficient statistics is exactly right for deciding, and can be a quiet act of loss for living.