the second hundred · metaphor 205

The Kalman filter.

How should you change your mind when new evidence arrives? Cling to your prediction and you ignore the world; lurch at every fresh rumor and you have no self. The wisdom is a proportion — to weigh your own read and the new report each by exactly how much it has earned, and to re-weigh that proportion every single moment.

You already run a rough version of this. A friend is usually punctual, so when they are ten minutes late you barely move — your prior is strong, one late night is weak. But if they are an hour late, and your picture of them can't absorb that, you finally update hard. The skill was never picking prediction or evidence. It is holding both, and letting the more trustworthy one pull harder.

Do it badly one way and you are rigid — defending the plans you made at nineteen against everything the years keep telling you. Badly the other way and you are a weathervane, spun by the last thing anyone said. Do it well and you look, from outside, like uncanny judgment: steady when the news is noise, quick when the news is real. It turns out there is a precise rule for exactly how far to move — and it re-derives itself at every step.

a moving target · tracked from noisy readings click the panel to throw in a wild reading →
true state (hidden) measurement z estimate x̂ ±√P uncertainty prediction x⁻
Kalman gain K
uncertainty √P
filter error (RMS)
raw reading error (RMS)
Every step, the estimate is a weighted blend steady-state gain K∞ =
◀ trust the prediction  (1−K) = K =   trust the reading ▶
= (1−K)·prediction + K·measurement   — the gain K slides the estimate between your model and the news.
Process noise · how fast the world moves0.08
still world · trust modelQ ↑fast world
Measurement noise · how unreliable the readings are0.40
crisp evidenceR ↑noisy · trust model
More process noise (Q) or less measurement noise (R) → larger gain K → the filter chases the data.
Presets
Balanced: the world drifts a little, the readings are a little noisy. The gain settles around 0.18 — the estimate leans on its own prediction and nudges toward each reading by a fifth.

Predict, then correct

Two moves, forever repeated.

The filter carries two things: its best guess of where the target is, and how sure it is — a spread P. Each tick it makes two moves. First it predicts: it rolls its guess forward by whatever it knows about how the world drifts, and — because the world also jitters unpredictably — it lets its uncertainty grow by the process noise Q. Then a reading z arrives, itself unreliable by the measurement noise R, and the filter corrects.

The whole art is in one number, the Kalman gain K = P⁻/(P⁻+R), a fraction between 0 and 1. The corrected estimate is x̂ = x⁻ + K·(z − x⁻) — which is exactly the weighted average x̂ = (1−K)·x⁻ + K·z. When your own uncertainty P⁻ dwarfs the reading's noise R, K→1: seize the data. When the reading is far noisier than your own doubt, K→0: hold your prediction. Nothing is chosen by mood; the proportion falls out of whose error bar is smaller.

And it re-weighs itself. A good reading shrinks P, so the next reading counts for less; a stretch of silence, or a fast-moving world, lets P swell, so the next reading counts for more. Feed it a steady world and the gain glides to a fixed steady-state value set entirely by the ratio Q:R — a permanent, optimal exchange rate between prior and evidence.

What to try

Move the two knobs. Watch the gain decide.

Nothing on the panel is scripted: a real hidden target wanders, real noisy readings scatter around it, and a real recursion tracks it — every readout is measured off that running filter. Drag process noise Q up and the target genuinely thrashes harder; the filter can no longer trust its stale prediction, the gain K climbs, and the amber estimate turns twitchy and responsive. Drag measurement noise R up and the blue readings spray wider; now the filter distrusts them, K falls, and the estimate goes smooth and calm — lagging, but unbothered by the spray.

Two things are worth catching. First, the error chips: the filter's RMS error stays well under the raw reading's RMS error — it beats every single measurement it's given, because it never bets everything on one. Second, click the panel to throw in one wild reading. When the filter is confident (small √P), watch it barely flinch — it files the shock under "noise" and holds course. That serene refusal to overreact is the gift and, as you'll see, the trap. The presets stake out the corners: trust the model, trust the data, and balanced.

The mapping

Belief, in the right proportion.

Swap "target" for "the truth you are tracking" — a friend's reliability, a market, your own health, whether a plan is working — and the recursion becomes a theory of changing your mind well. Your prediction is your standing model of how things go. The measurement is whatever just happened. The gain is how far this one event should move you, and the filter's answer is neither "believe your gut" nor "believe the evidence" but a blend weighted by relative trust: how noisy is this source, and how stale is my model?

That reframes both failure modes as bad gains. The rigid person runs K stuck near zero — supremely confident, discounting every correction. The anxious person runs K near one — no prior survives contact with the latest headline. Wisdom is the middle recursion that also updates its own confidence: certain enough to ignore a rumor, humble enough that a run of surprises finally moves it. You cannot see someone's gain directly. You can only watch how much one piece of news moves them — which is the tell.

Read as life lessons

Three things the gain teaches.

01

Trust is a ratio, not a side

You don't pick prediction or evidence. You weight each by its error bar. The confident source wins not because it's louder but because its bar is smaller — and yours can be the small one.

02

Update your confidence too

The gain only stays wise because P breathes — shrinking on good data, swelling when the world moves or the news goes quiet. A belief that never revisits its own certainty will misweight everything after.

03

Steadiness ≠ stubbornness

Ignoring a wild reading can be optimal, not closed-minded — if your model is sound. The same shrug is wisdom or delusion depending on one thing: whether the model is actually right.

In the wild

Where the blend runs the machine.

SPACE & FLIGHT

Kalman's filter flew on Apollo's navigation computer, fusing a physics model of the trajectory with sparse star and radar sightings. Guidance, missiles, and drones still run its descendants.

EVERY GPS

Your phone blends a motion model with noisy satellite and sensor readings dozens of times a second. The dot glides smoothly because a gain is quietly deciding how much to trust each fix.

FORECASTS & FINANCE

Weather assimilation, economic nowcasting, and quant trading all fold fresh data into a running model in tuned proportion — the same predict-then-correct heartbeat, scaled up.

The mapping, exactly

Mathematics ↔ life.

MathematicsLife
prediction x⁻What your model of the world expects next — your standing prior, rolled forward.
measurement zThe fresh evidence that just arrived — the thing that happened, report and all.
innovation z−x⁻The surprise: the gap between what you expected and what you actually saw.
measurement noise RHow unreliable this source is — rumor and rounding, a metric you half-trust.
process noise QHow fast the world really changes — how quickly your model goes stale between looks.
Kalman gain KHow much weight you give this news versus your prior — the exact size of the update.
uncertainty PYour live confidence — shrinking with good data, growing when the world moves or the news stops.
update (1−K)x⁻+KzChanging your mind in exact proportion to relative trust — no more, no less.

The honest model

What's really under the hood.

The hidden target obeys a random walk on a known slow drift: xₜ = xₜ₋₁ + uₜ + wₜ, with wₜ ~ N(0, Q). Each reading is zₜ = xₜ + vₜ, with vₜ ~ N(0, R). The filter runs exactly the textbook scalar recursion — predict x⁻=x̂+u, P⁻=P+Q; update K=P⁻/(P⁻+R), x̂=x⁻+K(z−x⁻), P=(1−K)P⁻. The band you see is ±√P; the gain, uncertainty, and both RMS errors are accumulated live from the true-versus-estimate stream. The panel is genuinely optimal here — because it is fed the very Q and R that generate the world.

One clean fact makes the steady state visible. Because P obeys a recursion that ignores the data, it converges to a fixed point P⁻ = Q + R·P⁻/(P⁻+R), the scalar Riccati equation, solved by P⁻ = ½(Q + √(Q²+4QR)) and hence K∞ = P⁻/(P⁻+R). That's the number printed as K∞; the live gain crawls to it within a few ticks. Move a slider and watch it re-solve — the "right proportion" is a computed quantity, not a temperament.

Where the metaphor tears

Optimal — only if you're right about how wrong you are.

It is optimal only if the model and the noises are right.

The filter is provably best on the condition that the dynamics are linear, the noises are Gaussian, and their variances Q and R are known. People almost never know their own Q and R — how fast their world truly changes, how unreliable a source truly is. Guess those wrong and the "right proportion" is precisely, confidently the wrong proportion. The metaphor's whole authority rests on numbers we usually don't have.

A wrong model is serenely, optimally wrong.

If your picture of how the world moves is mis-specified, the filter can drive P small — grow certain — while tracking a fiction. And a small P means a small gain, so it discounts the very evidence that could correct it. That is the smug expert, or the settled grudge: the tighter the false confidence, the more each disconfirming fact is filed away as noise. Real systems bolt on defenses the pure filter lacks — inflating Q, watching the innovations, resetting when surprise runs too high.

Surprises get filed as noise.

The blend assumes small Gaussian wobble. A genuine outlier, a regime change, a rare event outside the model — the filter down-weights it by design, exactly when it most needs to lurch. Sometimes the wild reading isn't noise; it's the world changing, and the optimal move is to distrust your model, not the data. Naming the tear sharpens the lesson: update in proportion to trust, yes — but keep auditing what you trust, because a filter that never questions its Q will glide, perfectly calibrated, off a cliff.