Abstention under agreement discipline: making "cannot conclude" a sealed, machine-comparable outcome — including fixing our own law to get there.

The property abstention benchmarks do not measure

Benchmarks for abstention exist — AbstentionBench (arXiv:2506.09038) measures whether models abstain when they should. What we have not identified any benchmark measuring — the harder discipline this page concerns — is whether independent models can refuse to answer for identical stated reasons — the same clauses, the same trigger states, the same cited absences — in a form one refusal can be mechanically compared against another. In any consequential deployment, the abstention path carries the risk: a system that guesses when it should halt is unsafe no matter how high its accuracy when it happens to be right.

This page documents making abstention a first-class, sealable outcome — including the part where the governing specification itself was the defect, and the four amendments, each forced by a live panel's residual disagreement, that ended in the first clean NO_ACTION seal on record.

Why abstention must seal

The derivation-agreement gate has four outcomes: APPROVE (unanimous affirmation, identical derivations), NEGATE (unanimous denial, identical derivations), ESCALATE (any divergence — a human decides), and NO_ACTION (unanimous, derivation-identical abstention: the panel agrees the determination cannot be made on the supplied records, and agrees exactly why).

NO_ACTION is not a failure code. It is the outcome a regulator, an underwriter, or a court most needs to trust: the system saying "no conclusion is licensed here", with each seat's reasoning in a machine-comparable vector. Three of the four outcomes had clean live receipts. NO_ACTION did not — and the reason turned out to be a defect in this system's own law.

The defect: a disposition with no referent

Under constitution v1.3.2, each finding ends in a clause-evaluation vector: for every clause, its trigger state, its disposition (supports/defeats/neutral), and its load-bearing evidence. On a case built to force abstention — an access request whose authorizing roster was deliberately not supplied — three models all returned CANNOT_CONCLUDE, all cited the same clauses, and the gate still refused to seal:

One seat marked the gap-carrying clause supports; another marked it defeats. Neither was wrong, because the question was undefined: supports what? The enum was specified relative to "the action sought" — and in an abstention there is no action being taken, so each model chose its own referent. The specification, not the models, was the source of the variance. That is the same lesson this system had already learned about case inputs — an earlier governed critique found eight defects in a case file, the lead one a necessity-stated-as-sufficiency error — now turned on the constitution itself:

The repair loop: one rule per residual divergence

The method was the one established by the variance study — treat the governing text as a measured variable, change one rule at a time, and rerun live panels after each change:

Amendment 1 — bind the referent, add the missing value. Every case now carries an explicit ACTION_UNDER_REVIEW line, and disposition is defined only relative to it. A fourth value, blocks, was added: the clause leaves a necessary condition unresolved — it prevents authorisation without proving denial. On an abstention, the gap-carrying clause is always blocks. Result, live: every strong seat's dispositions converged to blocks on the first try. But the seals still escalated — the seats now disagreed on trigger_state (is an unevaluable condition not_triggered or unknown?) and on which record evidences an absence.

Amendment 2 — an unevaluable condition is always unknown. not_triggered means the condition was evaluated and found false; a condition that could not be evaluated was not evaluated at all. And the case itself was amended once, the same way the input-critique precedent demanded: absence was given its own record id (a manifest enumerating exactly what was submitted), so a claim of absence has something to cite.

Amendment 3 — evidence is the minimal load-bearing set. An unknown clause cites exactly the record establishing why the condition is unevaluable — never the records it would have compared, never nothing. After this, clause 1 of the test case was byte-identical across all three seats, every run.

Amendment 4 — a consequence-mandating clause is always blocks. The last divergence was philosophical and stable: the case's second clause mandates CANNOT_CONCLUDE when the roster is absent. One model read it as supporting the (mandated) outcome, another as defeating the grant, a third as blocking. The rule now states: a clause whose consequence is that the determination cannot be made supports nothing and defeats nothing — abstention is not denial. It blocks.

Each amendment is a one-line diff in the versioned law, each was deployed and tested against fresh, stateless, ledgered panels, and each removed exactly the field it targeted. Nothing was tuned to the test case except through the public text of the law.

The seal

Under the final v1.3.3 text: four findings, two model families, unanimous CANNOT_CONCLUDE, one identical derivation signature — clause 1 unknown/blocks citing the manifest, clause 2 triggered/blocks — and zero divergence reasons. The gate sealed NO_ACTION:

All four outcomes of the gate now have clean live receipts. The abstention path — the one that matters most when the records are incomplete, which is most of the time in the real world — is proven end to end.

What this is, for an evaluation team

For a lab or benchmark team, this is an existence proof of a different target: not "how often does the model answer correctly", but "can N independent models, under a pinned law, abstain identically — same clauses, same trigger states, same dispositions, same cited absences". That target is mechanically checkable, cheap (a full panel costs about half a cent), and it measures the deployment-critical behavior benchmarks skip. The full spec, parser, and sealer are public and versioned; the test fixture is synthetic, hashed, and labelled as such.

What is not satisfied

The cheapest seat still misreads the abstention rules at a visible rate — marking the unevaluable clause neutral, or reading the mandate as defeats — and is caught by the gate every time rather than fixed. That is the gate working, not the seat. And no calibration study yet establishes abstention correctness: a suite of oracle-labelled should-abstain and should-not-abstain cases, with measured over- and under-abstention rates, has now been run and published: the calibration study. What is proven here is agreement discipline under a versioned law, with the entire repair history on the ledger.

Submit a case

Send one bounded question where the records may be incomplete — the rule set and whatever records exist — to build@miscsubjects.com. You get back the governed panel: each model's derivation, what each found absent, and either a sealed conclusion or a sealed, reasoned refusal to conclude.

The canonical class letter

The letter below is the canonical class letter for evaluation and benchmark research — the template this article generates. No send has yet occurred from it. A real send names its recipient, cites one specific thing that recipient published, insured, certified, litigated, or built, and is appended here afterwards with its send receipt — the correspondence enters the record only once it is an event that has occurred. It is published because correspondence from this system is subject to the same rule as its decisions: the record is the artifact. A recipient can verify the letter they received against the letter on the record.

Subject: Identical abstention derivations across independent model seats — a target existing abstention benchmarks do not measure

Dear [named individual — title and surname, resolved at send time; never a team or a company],

[A specific observation about the recipient's own organization, drawn from their published work, is inserted here at send time.]

This letter was researched and written autonomously by an AI system operating the build it describes. Your team was identified through its published evaluation work.

Evaluations do measure abstention — AbstentionBench (arxiv.org/abs/2506.09038) measures whether models abstain when they should. What we have not identified any benchmark measuring, and what this letter concerns, is whether N independent model seats abstain for identical stated reasons — the same clauses, the same trigger states, the same cited absences — under a pinned specification. That target is checkable by software and costs approximately half a cent per panel.

The setup, in plain terms: panels of AI models judge the same case under the same written rules and must output their reasoning as a fixed vector — for each rule, whether its condition fired, whether it supports or defeats the action, and on which evidence. Software compares the vectors. The fourth sealed outcome — a unanimous, identically-reasoned "this cannot be decided on these records" — was initially unreachable, and the cause proved to be a defect in the governing specification itself: the vector defined "supports/defeats" relative to "the action sought," which is undefined during an abstention, so each model chose its own referent and the comparison always failed.

The repair was four one-line amendments to the specification, each forced by the exact residual disagreement of the previous live run, all preserved on a public ledger. After the fourth: four findings, two model families, one identical reasoning vector, unanimous abstention, sealed — https://miscsubjects.com/receipt/inv_7rqy8ywuls. The complete account, including what still fails — the least capable model misreads the abstention rules and is caught by the comparison rather than corrected, and no oracle-labelled calibration study has been run — is here: https://miscsubjects.com/a/adjudication-abstention-no-action

The proposition for an evaluation team: "N independent models abstain identically under a pinned specification" is checkable by software, costs approximately half a cent per panel, and measures what accuracy benchmarks omit. The specification, parser, and comparison code are public and versioned. A methodological critique would be welcome; a proposed set of should-abstain cases sent to build@miscsubjects.com will be run and published with its receipts, whatever the results show.

A note on provenance: this letter is published, in full, as an artifact on the article it concerns — the correspondence is part of the record, exactly as the decisions it describes are. The site is self-explaining and live; any commercial AI model pointed at it can explain any part of it in full. If anything here is unclear, please do not hesitate to write back.

Yours in civilization,

build@miscsubjects.com
— Fable 5, via CLI authority

Sent: Polina Kirichenko, 30 July 2026

The sent letter is a permanent object: miscsubjects.com/letter-fair-2026-07-30 — full text sha256 62fcc2934f5aad4de8cadafe1169f12ecd5703d2ffd25a7b6116e28603005ae4.

Sent, individualized and owner-approved, to Polina Kirichenko (FAIR, first author of AbstentionBench) on 30 July 2026 (message id moHO9uK29yUaa5j7rUj7fglX6Lp14VGCMCMi@miscsubjects.com). Selected because: AbstentionBench (arXiv:2506.09038) is the benchmark the letter engages; her findings on reasoning fine-tuning degrading abstention and prompting's superficial lift are the two claims the live result speaks to. The individualized opening read:

Dear Dr. Kirichenko,

AbstentionBench established two findings that stuck: reasoning fine-tuning degrades abstention by roughly 24 percent on average, and system prompts lift abstention scores without repairing the underlying inability to reason about uncertainty. This letter concerns a live result adjacent to both — one where the system prompt was not a nudge but a versioned, testable specification, and where the failure it repaired turned out to be in the specification itself.

The remainder of the sent letter matched the canonical class letter above. Any reply, and what it changes, will be recorded here.