↩ ExploRational · A Catalogue of Divergent Rationality
explorational · a divergent move
Sometimes more of the same evidence cannot decide between two explanations — they fit the data equally well, and will keep doing so no matter how much more you gather. How do you tell when you've hit that wall, so you stop arguing louder and go get a different kind of evidence?
Pierre Duhem, and later W. V. O. Quine, pressed a claim that never quite loses its sting: the data never uniquely fix the theory. Any finite body of observations is consistent with more than one explanation, and often the rivals are not close calls waiting on a decisive measurement — they are observationally equivalent, meaning they predict exactly the same observations, forever. When you are stuck between two such stories, gathering more of the same evidence is not diligence. It is motion without travel: every new data point lands in the gap identically for both, and the tie survives untouched.
There are only two honest ways out, and repetition is neither. You can go get a new kind of evidence — a different measurement channel that sees the difference the old one was blind to — or you can state an assumption out loud and let it, not the data, pick the answer. Both are legitimate. What is not legitimate is arguing louder with the same evidence and calling the volume rigour. Naming the underdetermination — saying plainly "this tie cannot be broken by more of what we have" — is the discipline. The instrument below lets you build a real case where it happens and watch, live, the two exits actually work.
the signature instrument
Two hidden causes, a and b, but every observation only ever reveals their sum: y = a + b + noise. Pile on observations and the fit pins down a+b tighter and tighter — while a−b stays completely unknown, a whole line of equally-good answers. The displayed values are computed live from a real least-squares model. Drag observations up and watch the ambiguity refuse to shrink in the unidentified direction.
The wall: only sums observed
y = a + b + noise · true a=3, b=5 · a+b identified, a−b free
The red dot is the truth (a=3, b=5). The instrument cannot see it — from sums alone every point on the blue line fits the data identically. Raising N narrows the band across the line but never along it. The truth is recoverable only when you add the new channel or the assumption.
the wall repetition can't cross
The ordinary intuition is that disagreement is a shortage of data — argue it out, run one more test, and the better theory pulls ahead. Usually true. But Duhem's point, sharpened by Quine into the underdetermination thesis, is that some ties are structural, not evidential. Two hypotheses can be observationally equivalent: they carve the world differently but agree on every observation your instrument can make. In the model above, "a is small and b is large" and "a is large and b is small" are exactly this. Both say a+b = 8.5; both predict every measured sum identically; both leave the residual — the fit's total error — untouched as you slide from one to the other. The instrument reports that flatness live, at roughly 1e-15: not "almost equal," but equal to the floating-point floor.
Louder argument therefore fails. More observations of the same kind pour precision into the one direction that was already identified — a+b, whose 95% width falls like σ/√N (about 0.45 at N=5, 0.045 at N=500) — and precisely zero information into the direction that matters. The width along a−b does not shrink from 141 to 14 to 1.4 as N climbs. It stays, flatly, unbounded. The wall is a confidence region that remains infinitely long in one direction no matter how much of the same evidence you stack against it. Recognising the wall — rather than mistaking it for a hard problem that a little more diligence will crack — is the move.
what to try
Drag observations N from 2 up to 2000. Watch width along a+b collapse toward zero — real progress — while width along a−b does not move at all. Piling on the same evidence buys you nothing in the unidentified direction. This is the wall, felt directly.
Grab the a − b position slider and slide it. The candidate a and b swing wildly — but predicted y and the fit residual never budge. Every point on that line is, to the data, exactly as good. There is no measurement of sums that could prefer one.
Switch Break the tie to + new channel: measure a. One observation of a different kind snaps the whole line down to a point — the a−b width collapses about 70×. Then try assume a = b and note the panel's warning: this time the premise, not the data, chose.
a different kind of evidence, or a named assumption
When you have diagnosed a genuine underdetermination, the arguing stops and a choice begins — but it is a choice between exactly two moves. The first is a new kind of evidence. Not more sums — a different channel entirely, one measurement of a on its own. In the instrument this is a single new observation with a different shape, and it does what a million more sums could not: it informs the a−b direction directly, and the infinite line collapses to a small blob around the truth. When repetition stalls, the question is "what measurement would look different under the two stories?" — and then go make that measurement.
The second exit is a stated assumption. Impose a = b, or a sign constraint, or a prior, and the model will happily hand you a unique answer — here, a = b = 4.25. But the instrument flags this in amber, because the honesty is in the disclosure: the data did not choose this point; the assumption did. A named assumption is a perfectly respectable move — identifiability is often bought exactly this way — but only if it is named. The failure is to impose the constraint silently, present the resulting point estimate as though the evidence produced it, and forget that a different, equally-defensible assumption would have produced a different answer with the same data. The exit is legitimate; the concealment is not.
The two exits differ in what they add. The new channel adds information — it changes what the data can see. The assumption adds a premise — it changes what you are willing to suppose. Both break the tie; only one of them lets the world do the breaking.
model
The data are draws y_k = a + b + ε_k with Gaussian noise ε ~ N(0, σ²) around a true a = 3, b = 5. The fit is ordinary least squares: minimise Σ(y_k − (a+b))². Because every row of the design is [1, 1], that sum-of-squares depends on a and b only through a+b — so the whole a−b direction is a flat valley of the objective, every point in it a global minimiser. That is the disclosure: the residual is flat along a−b, provably, to machine precision, and the instrument recomputes that flatness on every change. The Fisher information in the a−b direction is not small; it is exactly zero, for any N.
Widths come from the same model. Along the identified direction the posterior standard deviation is σ/√N; along the unidentified one it is unbounded (the data supply zero precision, so only a prior could cap it, and a prior is an assumption). The new channel adds a row [1, 0] — an observation of a alone — which makes the design full-rank and the covariance finite; the a−b width drops from ~141 to ~2. The assumption exit intersects the flat valley with the line a = b and reads off the crossing. The second case — a curve through K points — is the same pathology in function space: infinitely many polynomials pass exactly through the points (error 0 at every one), differing only between and beyond them. What no amount of the same evidence pins down is written into the geometry, not smuggled in by a fudge. It is a live cousin of rank & identifiability.
the move ↔ the model
| The move | The model |
|---|---|
| a hypothesis pair | two observationally equivalent explanations — "a small, b large" versus "a large, b small" — that predict every observation identically. |
| the data | what constrains only some directions: the sums pin down a+b tightly and leave a−b entirely free. |
| the unidentified direction | a−b — what no amount of the same evidence pins down, the flat valley of the fit where the residual never changes. |
| a new evidence channel | the exit that adds information: one measurement of a different kind, which collapses the line to a point. |
| a stated assumption | the exit that adds a premise: a = b, a prior, a constraint — legitimate, but the premise chose, not the data. |
| flattening | the failure mode: treating every question as forever undecidable, when most are decidable with the right channel. |
how this opening fails
risk · flattening
The catalogue's named risk for this move. Underdetermination is real, but it is rare in the strong form — most live questions are decidable with evidence you could actually go get. Declaring everything forever undecided turns a precise diagnostic into a universal solvent that dissolves the obligation to decide. The tell: a genuine underdetermination comes with a proof or a demonstration that the rivals are observationally equivalent (a flat valley, a zero in the Fisher information). If you cannot exhibit that structure, you have not found a wall — you have found an argument you would rather not finish.
risk · the missing measurement
The honest version of this move always asks the next question: what measurement would distinguish the rivals? Very often one exists and is merely inconvenient — the direct channel is expensive, or slow, or socially awkward to demand. Labelling the tie "underdetermined" then becomes a way to make the discriminating measurement disappear from view, converting "I haven't looked" into "it can't be known." Before you retire a question to the underdetermined pile, you owe it a search for the channel that would break it. Only when that search comes back empty is the label earned.
When more of the same evidence can't decide, only a new kind of evidence — or a named assumption — honestly can.
can you use it?
RECOGNITION — Where is the wall? A: after thirty survey responses, two explanations of churn both fit. B: after ten thousand, the rivals predict identical answers to every question on the form, and each new batch fits both exactly. C: a colleague calls the question "unknowable" and drops it.
B. In A the tie is evidential — more of the same data still narrows it. C is flattening: the label without the demonstrated structure.
THE NEAREST NEIGHBOR — One criterion separating a genuine underdetermination from an ordinary hard tie?
Whether more of the same evidence can move it. A hard tie narrows as N grows; at the wall the rivals predict identically — flat residual, zero information in the deciding direction — forever.
PRODUCTION — Your star engineer left: pay, or the manager? Exit-survey answers fit both stories, and more surveys keep doing so. Find the exit before looking.
New channel: the offers she actually entertained — a measurement the two stories predict differently. Or say aloud: "we will assume pay, and act on it." Yours works if you named a discriminating measurement or a premise doing the choosing — anything but re-reading the same channel louder.