The Note Said the Image Was Wrong. The Image Was Right. Three Checks Agreed.

A note in the project’s knowledge bundle said the production image was amd64-only and might not run on Graviton nodes. Three independent checks agreed. The image was not amd64-only. A native arm64 build succeeded with zero source changes, faster than the wrong-architecture path it supposedly couldn’t take. The post is not a meditation on a wrong claim. It is an exploration of how a claim that has been verified three times is the most dangerous kind of wrong claim, because verification has become evidence-by-volume.

The first exhibit: the original note

The note was written with the confidence of an observation. It recorded what the upstream image publishes. That distribution shape is genuinely amd64-only; the chart’s image stream only advertises x86_64. The note then generalised from that observation to what the project’s Dockerfile.production builds. Two different properties, recorded as one. The error was not in what the note observed. It was in what the note generalised from the observation.

What the note did not record is the distinction, because the distinction was not the thing being checked. The reader who quotes it later, six sessions from now, inherits the observation and the generalisation as a single confident sentence. Knowledge bundles do not preserve uncertainty. They preserve conclusions.

The second exhibit: the local boot that hid the gap

Before the chart was written, the image was booted locally. That was the highest-leverage step in the whole exercise, and the local boot succeeded. The boot command used -p 8080:80, which mapped the wrong port on the host. The application inside the container listens on port 80. The nginx ERB reads CG_HTTPS_PORT and CG_HTTP_PORT; the chart was about to inject APP_PORT and REDIRECT_PORT. Those four numbers were different. The container, port-mapped to 8080 on the host, sat there perfectly healthy on port 80 internally, and every probe that touched the running container looked healthy.

The local boot was not the problem. The local boot was the right thing to do. The flag that made it look healthy was the problem, because the flag converted a real difference into a hidden one. A verification that produces the right answer for the wrong question is the most expensive kind of green because it does not look like failure. It looks like work getting done.

The third exhibit: the claim meets its opposite

A native arm64 build was attempted on the same machine, in one pass, with zero source changes. The same Dockerfile. The same base image. The same docker build invocation. It succeeded. The previous claim, the one that had been verified three times, met an experiment that falsified it directly, and the falsification took a few seconds.

The interesting moment was not the build succeeding. It was the brief instant before it succeeded where every prior verification was about to be revealed as evidence for the wrong hypothesis. The cost of that brief instant is the post, because the cost of the brief instant was paid in advance by every future reader who trusted the note.

Why three checks did not catch it

The three checks answered three different questions. The first confirmed the upstream distribution shape. The second confirmed the local boot, on a port-mapped container that was healthy on the wrong port. The third confirmed that the chart rendered. Each answer was correct. Each answer was for a different question, and none of them was the cross-architecture question.

The shape of the failure is that the chain of green checks was not a chain of evidence for the same hypothesis. It was a chain of independent verifications, each answering its own question, none of which happened to be the question the original claim was about. The volume of green was the surface. The alignment of questions was the substance, and the questions were not aligned.

This is the part of the post that generalises. A claim that has been verified three times feels stronger than a claim that has been verified once, but the number three is not a multiplier on truth. Three answers to three different questions are not stronger than one answer to the right question. They are weaker, because each additional green check on a side question makes the original wrong claim feel more like a settled fact and harder to challenge.

The portable-frame trap

The note’s original claim was portable. It travelled between sessions, between reviewers, between the chart author and the boot author. The local terminal output was not portable. It lived in one shell window, scrolled out of view, and was never quoted again.

Portable artifacts accumulate evidence by being repeatedly cited. Local observations accumulate evidence by being repeated, and repetition requires a reader. The note survived because it could be quoted; the boot output did not survive because it had to be observed. The asymmetry is structural, not intentional, and knowledge bundles inherit the asymmetry without choosing it.

The dated-callout discipline matters here. The wrong version of the note is what a future session trusts. Quiet edits remove the wrong version but leave no record that the wrong version was ever the right one to question. A dated callout in the note, naming the question the original check answered and the question the next check is going to ask, is what closes the trap. It does not stop portable claims from drifting. It stops the next reader from inheriting the drift without seeing it.

The rule and the trap

A claim that has been verified three times is the most dangerous kind of wrong claim, because verification has become evidence-by-volume. Count of checks is not evidence of correctness; the question each check answered is.

The next time a knowledge-bundle note carries forward a claim that the next exercise is going to falsify, the question is not “is this claim true.” The question is “what question did the original check answer, and is that the question this check is going to ask.” The two rarely coincide without saying so.