the second hundred · metaphor 157
Old friends need fewer words. A shrug does the work of a paragraph; a single name summons an entire afternoon neither of you has to describe again. The longer two people share a history, the less they have to spell out — because everything already said is standing by, ready to be pointed at.
Strangers must say everything in full. There is no shorthand yet, no "you know how you get," no reference that lands — so the first conversations are long and careful, each thing explained from the beginning. It is exhausting, and it is not intimacy; it is the cost of having no shared past to lean on.
Then the history accumulates. A phrase from a bad night becomes a word. A word becomes a look. Whole stretches of experience get filed away under a private handle, and from then on you don't re-tell them — you just refer. The talk gets shorter and shorter while carrying more and more, until an outsider can't follow a word of it and the two of you have never felt more understood.
The idea
Most codes are fixed: they decide in advance how many bits every symbol costs and never change their mind. Lempel–Ziv refuses to. It reads a stream once, and as it goes it builds a dictionary of every substring it has already seen. When a run of text matches something in the dictionary, it doesn't re-spell it — it sends a short pointer to the earlier occurrence, plus the one new character that extended it. Then it files that longer phrase away too, ready to be pointed at next time.
The trick is that the dictionary is shared: the decoder builds the exact same one as it reads, so a pointer is all it needs. No table has to be transmitted; the history is the codebook, and both sides grow it in lockstep from nothing. Early on this is a bad deal — you're paying to establish references that haven't paid off yet, and the cost per character sits above a plain fixed code. But every phrase you bank makes the next repetition cheaper.
So the cost per symbol falls as repetition accumulates. Where a fixed code would spell the boat from scratch every time — the same bits, forever — Lempel–Ziv spells it once, then recalls it in a pointer or two. The more a stream repeats itself, the further the line drops: given enough shared history, the bits per symbol approach the true rate at which the source actually produces new information, and everything predictable becomes almost free to say.
What to notice
The instrument runs a real Lempel–Ziv (LZ78) pass over whatever text is in the box — nothing is faked; every bit is counted from the actual parse. Press read and watch the dictionary fill, one phrase at a time, each freshly-banked reference flashing teal. Follow the purple bits-per-symbol line: it starts high — even above the flat amber fixed-code line, the overhead of a memory not yet earning its keep — and then, as the boat comes back and comes back, it dives beneath it.
Now swap presets. The chorus repeats whole lines and the curve collapses fast — a song is almost all recall. The strangers text barely repeats, so the dictionary fills with junk that's never reused and the line hugs the baseline, learning nothing. Best of all: type your own. Paste a paragraph and watch where it turns compressible — the moment a phrase returns is the moment the line bends. That bend is the birth of a shorthand.
The mapping
Two people who have known each other for years are running Lempel–Ziv on each other. Every shared experience is a phrase entered into a private dictionary; every later mention is a pointer, not a retelling. "That thing at the wedding." "You know how you get." A name, a date, a single word that unpacks into a whole story only the two of you can decode. The talk gets terse not because there's less to say but because so much of it is already stored and need only be referenced.
This is why jargon, family in-jokes, and old-married shorthand all feel the same from outside — impenetrable — and the same from inside — effortless. It is also why the early days are so much work: no dictionary yet, everything spelled out in full, the cost per exchange high. Intimacy isn't measured by how much you say; it's measured by how little you need to, which is exactly the compression ratio of a history you both carry.
Read as life lessons
Before there's a reference to point at, everything must be said in full. New relationships, new teams, new fields all feel slow for the same reason: the dictionary is empty and every idea is spelled from scratch.
When old friends say almost nothing and understand everything, that's not telepathy — it's recall. The shorthand is the receipt for years of things once said in full and never needing saying again.
An in-joke is a pointer into a dictionary you don't have. It isn't that the words are hard; it's that the history they reference is missing. Belonging is holding the same codebook.
The honest model
The pass is textbook LZ78. It walks the text keeping a growing phrase in hand; as long as that phrase-plus-next-character is already in the dictionary, it extends. The moment it isn't, it emits a token — the dictionary index of the part it knew, plus the one new character — and files the whole new phrase under the next index. Each token costs ⌈log₂(entries so far)⌉ bits for the pointer and ⌈log₂(alphabet)⌉ bits for the new character. The panel sums these exactly; nothing is estimated.
The purple line is that running total divided by characters read, bits ÷ symbols; the amber line is the same alphabet encoded with a fixed ⌈log₂(alphabet)⌉ bits per character, a code that never learns. Because pointers grow only like log₂ of the dictionary while the phrases they name grow linearly, a repetitive stream drives the ratio down; a random one can't, and LZ's small per-token overhead even leaves it slightly above the baseline. This is a real, if deliberately small, model — a few hundred characters, one alphabet, one pass — not the industrial LZ inside your zip files, but the same idea, honestly counted.
The mapping, exactly
| Mathematics | Life |
|---|---|
| the growing dictionary | Shared history — the accumulating store of things two people have already been through together. |
| a dictionary entry | An in-joke, a private handle, a reference — one experience filed under a name you both now hold. |
| a pointer token | A shrug, a glance, a single word that recalls the whole stored thing instead of re-telling it. |
| bits per symbol falling | Needing fewer and fewer words to convey more and more — the felt shorthand of long closeness. |
| early overhead above baseline | The exhausting fullness of first conversations, before any reference has been established to lean on. |
| the decoder builds the same dictionary | Understanding: the shorthand only works because both of you grew the identical codebook, in step. |
Where the metaphor tears
Lempel–Ziv only works because encoder and decoder build the identical dictionary, deterministically, forever in sync. People don't. Two friends can file the same event under different handles, or remember a reference to mean different things, and then a pointer that felt cheap decodes to the wrong story. Much of the pain of long relationships is exactly this — a shorthand that both sides think is shared but silently isn't. The math has no equivalent of a codebook quietly diverging; ours do it constantly.
A stream can be highly compressible and mean nothing — "na na na" forever squeezes to almost zero bits. Low bits-per-symbol measures predictability, not richness, so a couple who have simply stopped saying anything new will compress beautifully too. Shorthand born of intimacy and shorthand born of a rut look identical to the counter. The metaphor rewards brevity, but silence and understanding are not the same thing, and the ratio can't tell them apart.