Put up-selection turns uncooked code into curated actuality — fashions non-compulsory.
Abstract
The shift many blame on “AI hallucination” can seem with no mannequin in any respect. It’s a property of history-dependent gating plus reminiscence that adjustments what will get revealed, not of a generator’s “intelligence”.
Core declare
Let a generator suggest outputs, a gate resolve what will get printed, and a reminiscence regulate future selections primarily based on previous blocks. You will notice totally different statistics within the revealed set versus the complete mixture — even when the generator is trivial (tables, cube).
- Generator: lookup desk, easy Markov, or pure RNG
- Gate: verification guidelines (require a quotation), confidence threshold, light-weight verifier
- Reminiscence: log blocked circumstances and bias future reveals (increase thresholds or power retrieval for related queries)
Why this isn’t about “AI”
Swap the generator and the sample persists.
- Deterministic desk or Markov: impact stays beneath history-gated reveal
- Pure RNG: impact stays with history-gated reveal, however disappears when the gate is random but rate-matched
- Frozen corpus (pre-generated, hashed candidates): select the gate after era; solely the revealed sub-ensemble adjustments
Reproducible controls (run anyplace)
- Zero-AI generator (deterministic): gate=historical past shifts ΔKL and run-lengths; gate=off is baseline
- Pure RNG: gate=historical past produces construction; gate=random-rate-matched collapses to baseline
- Frozen-corpus, delayed gate: flip gate after candidates exist; revealed stats flip, corpus hashes don’t
- Reminiscence ablations: freeze reminiscence → collapsed state persists; shuffle reminiscence keys → impact drops to baseline
- Two impartial implementations: reproduce in Python and Rust/Node with the identical seed → matching metrics
- Airtight container: no mannequin libraries, no community → impact nonetheless current
Minimal interface (instance settings)
- Mills: markov, prng, frozen
- Gates: off, historical past, random-rate-matched
- Reminiscence: dwell, freeze, shuffle
- Seed: 20250926 for determinism
Artifacts to emit
- metrics.json — ΔKL, run-length distribution, confidence intervals
- log.jsonl — question, candidate hash, gate resolution, causes, citations
- HASHES.txt — integrity for the frozen corpus
Metrics that really matter
- ΔKL (revealed vs baseline) ought to improve beneath historical past gating
- Run-length (ninety fifth percentile) ought to improve beneath historical past gating
- Abstention fee on an unanswerable set ought to meet a pre-registered goal (for instance ≥ 0.9)
- Calibration error (ECE or Brier proxy) ought to enhance on the revealed sub-ensemble
- Null take a look at (random, rate-matched gate) ought to present no important deviation from baseline
Implementation sketch (plain steps)
- Generate a candidate
- Try retrieval; if no grounding, abstain
- Compute a confidence rating; if under threshold, ask for clarification
- Run a light-weight verifier; if it blocks, log a penalty occasion and abstain
- If it passes, reveal the candidate with citations
- Replace reminiscence with the choice so related future circumstances are biased towards safer selections
What would falsify this
- Random, rate-matched gating produces the identical ΔKL and run-length shifts as historical past gating
- Reminiscence shuffling fails to break down the impact
- Seed-locked repeats fail to breed equivalent metrics and hashes throughout languages or runtimes
Why this issues operationally
Manufacturing techniques are judged on what surfaces, not on every part generated internally. A gate that prefers cite-or-abstain over invention, backed by reminiscence that penalises unsupported paths, adjustments danger — whether or not or not a big mannequin is concerned.
