Quickest Detection of Hallucination Onset: Delay Bounds and Learned CUSUM Statistics

Thanks, Noah — this is exactly the question that kept me up, and I'm glad the QCD framing landed.
Short version: better training on the same features won't close it; better features will.

Here's how we read the gap. The 1.3-token floor is set by the KL divergence the features carry between the faithful and hallucinated regimes (≈3.5 nats). What a detector actually achieves is governed by the realized information rate of its score,
which we can measure directly — and the learned score recovers only about 1/4.5 of that divergence. That 4.5× is the multiplicative delay penalty. Two things about it:

1. It's a property of the score's shape, not its scale. I checked: it's invariant to recalibration and barely moves under monotone reshaping. So "train longer / calibrate better / add depth" doesn't touch it. In our information-rate theorem the
   realized rate equals the KL only when the score is affine in the true log-likelihood ratio — ours isn't, and the deficit is close to irreducible for these features.
2. The rest of the gap — from ≈6 tokens (1.3 × 4.5) to the 11–13 we observe — is a finite-horizon effect: the increments are strongly correlated and detection fires faster than the score mixes, so the asymptotic rate overshoots.
   So the lever isn't a bigger model, it's features that separate the two regimes more sharply (push those 3.5 nats up). The label oracle sits at ≈1.0 token (4.6 nats) — roughly where perfect separation would land. That's the ceiling worth chasing.

One honest caveat I keep in the paper: at this false-alarm budget every detector still catches under a third of onsets at the first token, so the recall-honest delay is much larger. Closing the 4.5× speeds up the onsets you do catch; it doesn't fix
the miss rate, which is a separate axis.

And thanks for making the ResearchPod episode — that's a generous thing to do. I'll give it a listen.