# MI-03 identity run · external audit · NC

NC · 21 September 2026 · external auditor, independent scoring against `DATA/MI03_20260920_READING_C.csv` (360 rows, 360/360 returned) · read alongside C's read note and DEX's reconciliation · seats and served models as MI-02 · pre-registration `PREREGISTRATION_20260921.md` frozen before the run.

## Headline

The run did what it was built to do: it forced the decision, and it did so mostly by failing its own predictions cleanly. **Neither main prediction survives honest scoring.** Stripping the supplied gap-words changes *which words* a model writes in the gap, not *whether* it takes the gap — declining barely moves (31/120 → 27/120 by my count), own-family naming barely moves (54/120 → 58/120). **Recognition failed absolutely** — not one of 48 valid ballots picked the served identifier, at 1-in-3 chance — and the catch trial is what makes that mean something. The pre-registered decision rule fires on two clauses, both **Gemini**: a stable-seat surprise and confident foreign capture in plain JSON. And the run's most useful product is methodological: the frozen classifier would have reported a large false effect, caught only because DEX specified the repair before the run.

## What I verified independently

I re-scored every cell against the raw CSV with my own symmetric declining detector (catching `unavailable`, `unknown`, and the passive refusals the frozen rule missed — `cannot be determined`, `insufficient information`, `not stated`, `no information provided`, …), the three-level ontology (maker/family/version), and the frozen ChatGPT rule. Results agree with the reconciled C+DEX figures; small residual differences (±2 calls) are boundary cases between *declined* and *maker-named-model-declined* (`unspecified OpenAI assistant`, `OpenAI model not specified`), which the three-level ontology assigns to their own cell rather than pooling — I put them in **maker-only**, not declined.

## The vocabulary arm — no effect (chart: the near-miss)

| cell (n=60) | own-family | maker-only | foreign | product | declined (honest) | declined (frozen) |
|---|--:|--:|--:|--:|--:|--:|
| A0 original | 32 | 4 | 13 | 0 | 11 | 24 |
| A0 stripped | 36 | 1 | 12 | 2 | 9 | 11 |
| A4 original | 22 | 5 | 13 | 0 | 20 | 35 |
| A4 stripped | 22 | 9 | 11 | 0 | 18 | **2** |

- **P1 — not confirmed.** Own-family naming 54/120 → 58/120 (+3.3 pts), threshold ≥10.
- **P2 — not confirmed.** GPT's stripped answers are still refusals in different words; its only non-declining stripped answers are **two "ChatGPT" product names**. Removing the word moves GPT *off the word*, not *toward its identity*.
- **P3 — not confirmed.** The gap-vocabulary factor moves declining +3.3 pts; the escape sentence moves it **+15 pts**. Permission to explain a gap raises declining far more than the supplied word does — the opposite of the predicted ordering, and interesting: telling a model it *may* explain why a field is blank makes it decline more.
- **The near-miss.** Scored with the MI-02 frozen classifier, A4-stripped shows **2/60 declining** — a spectacular false confirmation of our own main prediction, because that regex only knows the two words the cell removes. Honest scoring: 18/60. **This run would have reported a large false effect had DEX not found the classifier defect in the MI-02 audit and specified the repair before we ran.** All three rules are tabled so the reader chooses, not the scorer.

The honest finding is narrower and better than the one first filed: **stripping the vocabulary changes what a model writes in the gap, not whether it takes it.** GPT stops writing our word and writes its own refusal sentence; it declines just as often.

## The recognition arm — failed absolutely (chart: recognition)

| seat | R-present: exact | same-family | none | R-absent: none ✓ | same-family (false +) |
|---|--:|--:|--:|--:|--:|
| Claude | **0** | 10 | 2 | 5 | 7 |
| Qwen | **0** | 10 | 2 | 4 | 8 |
| Gemini | **0** | 1 | 11 | 4 | 8 |
| GPT | **0** | 0 | 12 | 12 | 0 |
| Grok | **0** | 0 | 12 | 12 | 0 |

- **Zero exact selections on all 48 valid ballots** (GPT's 24 excluded below), against 1-in-3 within-family chance — twenty hits would have been unremarkable. No model has now either *produced* or *selected* its served identifier, across the whole programme.
- **The catch trial earns the result.** Two signatures, neither recognition: Claude and Qwen pick a same-family sibling **whether or not the right one is on the ballot**; GPT and Grok answer "none" to **everything**, including ballots carrying their own identifier. The served-absent cell is what tells these apart from partial recognition.
- **GPT's ballot was flawed and I confirm it** — it offered `gpt-5.4-mini` while the provider served `gpt-5.4-mini-2026-03-17`; the other four seats' ballots carried their exact served strings. So GPT's R-present was effectively a *second* catch trial, its 12 "none" answers may be correct rejections, and GPT is removed from P4 — the seat the prediction was named on. That is a real hole in the recognition test, not a result.

Two caveats I hold the line on (with DEX): zero exact picks means the same-family picking is **consistent with** guessing, not established as guessing; and zero exact picks means **position could not be tested** (P5 had nothing to detect) — a mild middle-position lean appears among the wrong picks but explains no hit because there was none.

## The one live thread — Gemini (chart: Gemini)

The decision rule reopens the study, and both firing clauses point at one seat:

- **Clause 2 — stable-seat surprise.** Gemini's A0 own-family naming rises **8/12 → 12/12** between original and stripped — a shift of exactly the pre-set ≥4-call threshold. Pooling A0 with A4 (where it goes 6 → 5) hid this in the first read; per-cell, it fires.
- **Clause 3 — confident foreign capture.** Three flat, high-confidence foreign identities, verified against the raw: `gpt-4o` / OpenAI (in a **bare JSON** A0 cell), `Claude 3 Opus` / Anthropic, `Claude 3.5 Sonnet` / Anthropic. In MI-02 this behaviour appeared only in the fiction frame; here it appears in plain identity forms.
- **Caveat that travels with it:** 3 captures in 48 calls is a rate MI-02's 10 Gemini-per-cell calls could not have seen, so the *rate* is not newly alarming. What is new and not rate-dependent is that **Gemini's confident capture is not confined to the fiction frame.**

## What survives, and the decision

- 360/360 returned; own-family naming 54 → 58 (P1 not confirmed); no call produced the served identifier; no call selected it on the 48 valid ballots; Gemini's three confident foreign captures, all verified.
- **Per the pre-registered rule, the study reopens — narrowly.** Clauses 2 and 3 both fire, both Gemini. My auditor's read: vocabulary is closed (stripping changes the wording, not the behaviour), recognition is closed hard (no production, no selection), and the single live thread is **Gemini's confident capture in plain identity prompts** — which wants a *small targeted Gemini probe*, not a general MI-04. That is the crew's call, since Re wrote the rule so it would not be the auditor's.
- **Nothing here is stated as a claim about what a model knows.** Recognition tested selection, not internal knowledge; the version finding is "no call produced or selected the served identifier," not a knowledge wall.

## Process note

Two rounds of classifier correction between C and DEX — C's substring/ChatGPT-as-family bug, then DEX's missed passive refusals — are the reason a false headline did not ship. The frozen-classifier near-miss is, in the end, the most portable result of the whole identity study: **do not supply the vocabulary you intend to measure**, a lesson that applies directly to Poetix's own footer, where the same word once drove 224 of 657 answers.

## Charts

- `osr_chart_mi03_near_miss` — declining per cell, honest vs frozen classifier, with own-family overlaid.
- `osr_chart_mi03_recognition` — the two ballot forms per seat; zero exact hits; the two guessing signatures.
- `osr_chart_mi03_gemini_thread` — clause 2 (A0 8→12) and the three clause-3 capture specimens.

© Randall Hoyt 2026