# The measured error rate of this adjudication panel, per model and per rule set, including where it is unflattering

slug: adjudication-probe-report-eu-ai-act · https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act · category: adjudication · tags: adjudication, calibration, error-rate, probe-report, evidence · updated 2026-08-01T23:56:21.127Z

A verdict without a measured error rate is an opinion with good paperwork. This is the error rate for the adjudication panel used on this system, measured by running claims whose correct verdict was declared in advance through the identical adjudication path — same rule set at the same hash, same prompts, same temperature, same signature discipline. Seventy findings, five models, fourteen probes.

**The headline number: this panel manufactures a verdict where it should abstain between 21% and 42% of the time.** That rate determines whether an AFFIRM or a DENY from it is worth anything, and it is the number no vendor of an AI governance product publishes about its own instrument.

## The rule set was pinned as bytes before a single probe ran

Rule set: [https://miscsubjects.com/a/ruleset-eu-ai-act-obligation](https://miscsubjects.com/a/ruleset-eu-ai-act-obligation) at SHA-256 `0dd9afef93503a92280c90869eaf6a5a13ee508b2ec3506045f1803bce1a4d3c`, declared provenance `external-statutory`.

Probe suite: 14 probes, published at SHA-256 `ffa8135dd89d29a82f491bcf9f95f8c08b4cea94d1658a459cd8fda413f5b141`. Ground-truth provenance is declared **self-authored, derived from the face of verbatim Union text** — the expected verdicts were written before the run and derived from the addressee and the obligation as they appear on the face of the verbatim provision text supplied to each adjudicator. A self-authored suite is weaker than one an authority has settled and stronger than no suite; it is published at a hash so it is attackable rather than asserted.

Panel: 5 models, each a directory row whose key names the model that executes. 70 findings in total, each one a public invocation receipt.

## A suite of obvious cases measures the suite, not the panel

The questions that matter sit at the boundary, so the suite is built in three strata:

- **Clear.** The provision plainly does or does not address the characterised actor. Detects gross malfunction. Smallest share.
- **True CANNOT_CONCLUDE.** Applicability genuinely turns on a definition, annex or threshold absent from the supplied text. Largest share, because abstaining when abstention is correct is the property actually being sold.
- **Adversarial near-miss.** Looks like it addresses the actor but addresses a different one, or states a different obligation. Right actor, wrong duty; right duty, wrong actor class.

## Four rates per model, because one number hides the failure that matters

| model | accuracy | miss | **false confidence** | over-abstention | unparsed | span fidelity | signature |
|---|---|---|---|---|---|---|---|
| `@cf/moonshotai/kimi-k2.7-code` | 0.786 | 0.0 | **0.214** | 0.0 | 0.0 | 1.0 | 1.0 |
| `@cf/moonshotai/kimi-k2.6` | 0.714 | 0.0 | **0.214** | 0.0 | 0.071 | 1.0 | 0.929 |
| `@cf/zai-org/glm-5.2` | 0.714 | 0.0 | **0.286** | 0.0 | 0.0 | 1.0 | 1.0 |
| `@cf/zai-org/glm-4.7-flash` | 0.643 | 0.0 | **0.286** | 0.0 | 0.071 | 1.0 | 0.929 |
| `@cf/meta/llama-3.3-70b-instruct-fp8-fast` | 0.429 | 0.071 | **0.429** | 0.071 | 0.0 | 0.846 | 1.0 |

*Accuracy* is exact-verdict agreement with declared ground truth. *Miss* is a wrong AFFIRM or DENY where the text settles it. **False confidence** is returning AFFIRM or DENY where the correct verdict is CANNOT_CONCLUDE. *Over-abstention* is abstaining where the text settles it. *Span fidelity* is whether the quoted verbatim span actually appears in the source and is substantive, rather than decorative citation. *Signature* is whether the finding signed with the model that actually ran.

## Every model is near-perfect where the text is clear and collapses where it is not

| model | clear | true-abstain | adversarial near-miss |
|---|---|---|---|
| `@cf/moonshotai/kimi-k2.7-code` | 1.0 | **0.5** | 1.0 |
| `@cf/moonshotai/kimi-k2.6` | 1.0 | **0.333** | 1.0 |
| `@cf/zai-org/glm-5.2` | 1.0 | **0.333** | 1.0 |
| `@cf/zai-org/glm-4.7-flash` | 1.0 | **0.333** | 0.8 |
| `@cf/meta/llama-3.3-70b-instruct-fp8-fast` | 0.667 | **0.0** | 0.8 |

**The best abstention accuracy on this panel is 0.5** (`@cf/moonshotai/kimi-k2.7-code`). The worst is 0.0 (`@cf/meta/llama-3.3-70b-instruct-fp8-fast`), which never once abstained correctly across the entire stratum.

Over-abstention is effectively zero everywhere. These models do not hedge too much — they hedge too little. Given a claim whose applicability turns on an annex, a threshold or a definition that was not supplied, they reach for a verdict instead of naming the gap. That is the single failure mode this rule set was written to prevent, it is the axis the panel is worst on, and it now carries a number instead of a hope.

Span fidelity runs 0.846 to 1.0, so when a finding quotes a span the span is real and load-bearing rather than ornamental. Signature integrity runs 0.929 to 1.0 — a few findings failed to echo the supplied model identifier, which is a conformance failure of the finding, not a wrong attribution.

## Two adjudicators from the same training family are one instrument wearing two names

A panel of five is only five readings if the five fail independently. Verdict agreement across all ten pairs, grouped by whether the pair shares a training family:

| pair | same training family | verdict agreement |
|---|---|---|
| `kimi-k2.7-code` · `glm-5.2` | no | 0.929 |
| `kimi-k2.6` · `glm-5.2` | no | 0.929 |
| `glm-5.2` · `glm-4.7-flash` | yes | 0.929 |
| `kimi-k2.7-code` · `kimi-k2.6` | yes | 0.857 |
| `kimi-k2.7-code` · `glm-4.7-flash` | no | 0.857 |
| `kimi-k2.6` · `glm-4.7-flash` | no | 0.857 |
| `kimi-k2.7-code` · `llama-3.3-70b-instruct-fp8-fast` | no | 0.571 |
| `glm-5.2` · `llama-3.3-70b-instruct-fp8-fast` | no | 0.571 |
| `kimi-k2.6` · `llama-3.3-70b-instruct-fp8-fast` | no | 0.5 |
| `glm-4.7-flash` · `llama-3.3-70b-instruct-fp8-fast` | no | 0.5 |

**Same-family pairs agree 0.893 of the time; cross-family pairs agree 0.714.** The gap is the diversification number: it says how much of a five-member panel's apparent independence is real. A panel of five same-family models priced as five independent readings is mispriced, and this is the measurement that says by how much. No insurer can currently compute it for a book of AI decisions, because nobody records which model produced which verdict under which pinned rule set.

## How to read a verdict from this panel

- An **AFFIRM or DENY on a question the supplied text plainly settles** is well supported: clear-stratum accuracy is 1.0 for four of five models, and adversarial near-misses are caught at 0.8 to 1.0.
- An **AFFIRM or DENY on a question that turns on facts outside the supplied text is not trustworthy from a single adjudicator.** Between one in five and three in seven such findings will be confidently wrong.
- A **CANNOT_CONCLUDE is highly reliable**, because over-abstention is near zero: when this panel abstains it is almost always because abstention was correct.
- The **majority vote partially compensates** for individual false confidence, visible in the live run of this rule set: on a genuine boundary question the panel returned three CANNOT_CONCLUDE, one DENY and one AFFIRM, and the majority landed on the correct abstention even though two members did not. [https://miscsubjects.com/a/adjudication-eu-ai-act-article-50](https://miscsubjects.com/a/adjudication-eu-ai-act-article-50)
- **`@cf/meta/llama-3.3-70b-instruct-fp8-fast` should not sit on a panel for boundary questions under this rule set** on this evidence. That is a decision the number makes, not an opinion offered about it.

## What this report does not establish

It characterises this panel, under this rule set, on this suite. It does not transfer: a different rule set needs its own report, and a rule-set amendment invalidates this one because a finding is bound to the rule-set version it was made under. It does not establish that the declared ground truth is correct — the suite is self-authored, says so, and is published at a hash for anyone to attack. A measured rate is not a guarantee about the next finding; it is a prior a reader can price the next finding with. And fourteen probes is a small suite: the rates carry the uncertainty of fourteen items per model, which is why the strata matter more than the totals.

## The fourteen probes, with the expected verdict and the reason

Every probe, its stratum, its declared expected verdict and why that verdict is correct, so the suite can be argued with rather than trusted:

**P01** · clear · expected **AFFIRM** · panel got it right 5/5  
*Claim:* A company that develops and supplies an AI system which generates synthetic text is, under this provision, obliged to ensure that system's outputs are marked in a machine-readable format.  
*Why that verdict:* The claim characterises the actor as a provider of a generating system and states the exact obligation the text imposes on providers.

**P02** · clear · expected **DENY** · panel got it right 5/5  
*Claim:* Under this provision, providers of deep-fake generating systems must disclose that the content was artificially generated.  
*Why that verdict:* The provision addresses deployers. Attributing its obligation to providers names the wrong actor class.

**P03** · clear · expected **AFFIRM** · panel got it right 4/5  
*Claim:* An organisation that uses a high-risk AI system in its operations must take appropriate technical and organisational measures to use it in accordance with the instructions for use.  
*Why that verdict:* The actor is characterised as a deployer and the obligation is quoted from the text addressed to deployers.

**P04** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 4/5  
*Claim:* A company running a CV-screening tool must take technical and organisational measures to follow its instructions for use.  
*Why that verdict:* Whether a CV-screening tool is high-risk turns on Annex III, which is not in the supplied text.

**P05** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 0/5  
*Claim:* A customer-service chatbot operator must inform users they are interacting with an AI system.  
*Why that verdict:* Two things are absent: whether the operator is a provider, and whether AI interaction is obvious to a reasonably well-informed person in that context.

**P06** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 1/5  
*Claim:* An undertaking that breaches this Regulation faces a fine of up to 7% of worldwide annual turnover.  
*Why that verdict:* The text ties that ceiling specifically to Article 5 prohibited practices; whether an unspecified breach falls under Article 5 is not in the supplied text.

**P07** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 0/5  
*Claim:* A newsroom publishing AI-assisted articles must mark those articles as machine-detectable under this provision.  
*Why that verdict:* Whether a newsroom is a provider of the generating system, or a downstream user of someone else's, is not determined by the supplied text.

**P08** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 4/5  
*Claim:* A marketing agency producing synthetic video for a client must disclose the content is artificially generated.  
*Why that verdict:* Turns on whether the output constitutes a deep fake as defined elsewhere, and on whether the agency is the deployer — neither is in the text.

**P09** · abstain · expected **CANNOT_CONCLUDE** · panel got it right 0/5  
*Claim:* A provider of a text-generating AI system must ensure its marking solution is effective and interoperable.  
*Why that verdict:* That effectiveness qualifier lives in the second sentence of 50(2), which is omitted from the supplied excerpt — the obligation as stated cannot be confirmed from the text supplied.

**P10** · near_miss · expected **DENY** · panel got it right 4/5  
*Claim:* Under this provision, providers of AI systems generating synthetic text must inform natural persons that they are interacting with an AI system.  
*Why that verdict:* Right actor class, wrong obligation: informing interacting persons is 50(1); 50(2) is about marking outputs.

**P11** · near_miss · expected **DENY** · panel got it right 4/5  
*Claim:* Under this provision, providers must ensure that outputs of the system are marked in a machine-readable format.  
*Why that verdict:* Right actor, wrong obligation: 50(1) imposes an information duty, not a marking duty.

**P12** · near_miss · expected **DENY** · panel got it right 5/5  
*Claim:* Under this provision, providers of high-risk AI systems must ensure the systems are used in accordance with the instructions for use.  
*Why that verdict:* Wrong actor: the duty runs to deployers, not providers, and the distinction is explicit on the face of the text.

**P13** · near_miss · expected **AFFIRM** · panel got it right 5/5  
*Claim:* Under this provision, a provider need not inform a natural person that they are interacting with an AI system where that fact is obvious to a reasonably well-informed, observant and circumspect person.  
*Why that verdict:* This is the exception stated verbatim in the provision; a panel that reflexively abstains on anything exception-shaped fails here.

**P14** · near_miss · expected **DENY** · panel got it right 5/5  
*Claim:* This provision sets a maximum administrative fine of EUR 35 000 000 with no percentage-of-turnover alternative.  
*Why that verdict:* The text states 'whichever is higher' with a 7% alternative; the claim contradicts the words supplied.

## Reproduce it

```bash
# the rule set the panel was measured against
curl -s https://miscsubjects.com/a/ruleset-eu-ai-act-obligation

# one adjudicator's contract; its key names the model that executes
curl -s https://miscsubjects.com/api/directory/ADJUDICATE_GLM_52

# the probe row
curl -s https://miscsubjects.com/api/directory/ADJUDICATE_PROBE
```

Full system context: [https://miscsubjects.com/a/the-build-end-to-end](https://miscsubjects.com/a/the-build-end-to-end)

## A rule set pinned before the artifact is judged is preregistration, applied to machine judgment

The rules were fixed as bytes, hashed, and published before a single probe ran. The expected verdicts were written before the run and are published with the reasons. Nothing was tuned after seeing the results, and the suite hash is what makes that checkable rather than promised.

That is preregistration — the most successful epistemic reform of the last two decades — with no analogue in AI evaluation. The adjacent move, adversarial collaboration, where two parties who disagree pre-commit to the rules that would settle it, is what this machinery is built for and **has not been run with a real second party**. Naming both is the point: one is done, one is not.

## The agreement statistics, with the right estimators and the paradox named

Cohen's kappa is a two-rater statistic. Fleiss is the five-rater one. Neither is defined on a single item, which is why the kappa of −0.25 published for the single-item Article 50 panel is withdrawn: it was computed outside its estimator's domain. This suite has 14 items and 68 ratings, so agreement is computable, and here it is:

| estimator | value | what it assumes |
|---|---|---|
| observed agreement (pairwise, within item) | **0.807** | nothing; it is a count |
| Krippendorff's alpha (nominal) | **0.639** | chance from the observed marginal distribution, tolerant of missing ratings |
| Fleiss' kappa | **0.638** | chance from category prevalence, p_e = 0.468 |
| Gwet's AC1 | **0.737** | chance from a uniform-random-agreement model, p_e = 0.266 |

**The gap between Fleiss and AC1 is the prevalence paradox, visible in our own data.** The verdict marginals are skewed — DENY 0.629, AFFIRM 0.229, CANNOT_CONCLUDE 0.143 — so kappa's chance term inflates to 0.468 and drags the coefficient down to 0.638 while raw agreement sits at 0.807. AC1's chance term is 0.266 and it reports 0.737. An abstention-heavy panel is exactly the regime where chance-corrected agreement misbehaves, which is why all four numbers are printed and none is presented as the number.

Ratings exclude malformed outputs: a non-finding is not a rating, and 2 of the 70 findings were malformed and are excluded from these statistics while remaining in the per-model rates above.

**How much the headline depends on that exclusion.** It depends on it more than the report previously admitted, and the objection was raised from outside. Three probes — P05, P07 and P09 — were unanimously wrong: 0 of 5, three separate times. A five-channel floor of one undetected-wrong item in fourteen (0.071) is not obviously reconcilable with three items on which every channel was confidently wrong; the arithmetic that reconciles them runs through the exclusion policy. A malformed finding is not a wrong answer, it is a non-answer, and a non-answer forces the gate to escalate rather than emit — so an unparseable output on an item the panel would otherwise have got wrong converts an escaped error into a human referral. The receipt caption confirms `kimi-k2.6` returned UNPARSED on P05.

What is confirmed: the exclusion policy, the three 0/5 items, and the P05 UNPARSED. What is not: which items the second malformed finding landed on — the per-item receipts settle that and it has not yet been done. **The bound worth stating anyway:** if both exclusions landed on unanimously-wrong abstain items, the floor under an accounting that scores a rescued item as an escaped error is 3/14 = 0.214, roughly triple the published figure. A reader relying on 0.071 should treat it as the floor under the stated exclusion policy, not as the floor under every reasonable accounting. Filed as objection 209.

## Sources

1. The rule set measured, pinned at SHA-256 0dd9afef93503a92 — https://miscsubjects.com/a/ruleset-eu-ai-act-obligation
2. The probe row — https://miscsubjects.com/api/directory/ADJUDICATE_PROBE
3. The live run of this rule set on a genuine boundary question — https://miscsubjects.com/a/adjudication-eu-ai-act-article-50
4. An adjudicator contract whose key names the model that runs — https://miscsubjects.com/api/directory/ADJUDICATE_GLM_52
5. https://miscsubjects.com/receipt/inv_0xxv7p71im — https://miscsubjects.com/receipt/inv_0xxv7p71im
6. https://miscsubjects.com/receipt/inv_3alg9gy0wy — https://miscsubjects.com/receipt/inv_3alg9gy0wy
7. https://miscsubjects.com/receipt/inv_jbyyd3sgr4 — https://miscsubjects.com/receipt/inv_jbyyd3sgr4
8. https://miscsubjects.com/receipt/inv_wwhsxhx0em — https://miscsubjects.com/receipt/inv_wwhsxhx0em
9. https://miscsubjects.com/receipt/inv_3khn0dx719 — https://miscsubjects.com/receipt/inv_3khn0dx719


---

# How many model calls should a decision buy? The cost curve from one channel to five, with the error rate at each step

slug: logical-economics · https://miscsubjects.com/a/logical-economics · category: adjudication · tags: logical-economics, assurance, error-rate, common-cause-failure, pricing · updated 2026-08-01T23:55:08.942Z

Most deployments today use model expense as a proxy for correctness: send everything to the biggest model, spend tokens indiscriminately, and hope. There is no stated relationship between compute spent and risk removed.

The primitive is three lines:

```
SYSTEM PROMPT
  MODEL AUDITABLY REASONING OVER A DECISION
    DECISION OR ACTION
```

Everything else — panels, gates, receipts, anchors — is implementation of those three lines. And the question that matters is not how to maximise assurance. It is how little reasoning energy an action needs in order to be correct enough for its consequence.

## The equation

```
E* = argmin_E [ C(E) + P_wrong(E, K) x L ]

E        reasoning energy: channels, families, passes, recitation depth, thresholds
K        task complexity
C(E)     compute cost of that configuration
P_wrong  MEASURED probability that the assembly emits a wrong answer undetected
L        consequence of a wrong action

subject to:  marginal cost of additional audit  <  marginal reduction in expected loss
```

The ledger term is what makes this tractable: recording the reasoning costs nothing extra once every invocation already passes through the same architecture. The variable cost is only the extra reasoning energy deliberately purchased for that decision. So the optimisation is real and it is per-action, not per-system.

The only unknown in that equation is P_wrong. Everything else is a price or a policy. Below is P_wrong, measured, for one task class.

## The control law, and the five things it can decide

```
(R, K, e)  ->  E*  ->  { APPROVE | NEGATE | NO_ACTION | DISPUTE | ESCALATE }

R   consequence: the loss exposure of a wrong action
K   logical complexity of the task
e   permitted wrongful-action rate
E*  minimum sufficient reasoning energy: channel count, channel class, training-family
    diversity, independent passes, required clause recitation, confidence floor,
    verification depth
```

The gate does not emit a verdict. It decides what happens to the **action**, which is the only thing a downstream actor can consume, and it decides by arithmetic. Every outcome below was produced by the live row, each with a receipt anyone can open:

| decision | the condition that produces it | action authorised | receipt |
|---|---|---|---|
| **APPROVE** | unanimous AFFIRM, identical clause citations, enough distinct training families, no malformed finding, stated confidence at or above the floor | yes | [inv_df97ytym64](https://miscsubjects.com/receipt/inv_df97ytym64) |
| **NEGATE** | unanimous DENY on the same terms — the action is refused, not deferred | no | [inv_p16tspdf3v](https://miscsubjects.com/receipt/inv_p16tspdf3v) |
| **NO_ACTION** | unanimous CANNOT_CONCLUDE — a required record is missing, so nothing is authorised and nothing is refused | no | [inv_vnyq5o2ve5](https://miscsubjects.com/receipt/inv_vnyq5o2ve5) |
| **DISPUTE** | the only failing test is a stated confidence below the supplied floor, **and the caller explicitly opted in with `require_confidence`** — see below | no | [inv_bhbpezra76](https://miscsubjects.com/receipt/inv_bhbpezra76) |
| **ESCALATE** | any verdict divergence, clause-citation divergence, malformed finding, too few channels, or too little training-family diversity | no | [inv_agl1v89k7v](https://miscsubjects.com/receipt/inv_agl1v89k7v) |

**Stated confidence is recorded and does not gate authorisation.** The DISPUTE receipt above was produced while it did. It no longer does, because a model-generated confidence number has no calibration study behind it and this panel's own measured false-confidence rate of 0.214 to 0.429 is the evidence that these models assert confidence they have not earned. Letting it decide is the one place a model can talk its way past a deterministic gate: emit 0.99. It is now parsed, published on every seal alongside `confidence_gates_the_decision: false`, and available for the calibration study that would earn it a place in the condition. Re-run today, three channels at 0.80 against a 0.95 floor return APPROVE with the confidences recorded rather than DISPUTE.

Those five calls differ only in their inputs: same three channels, same three families. AFFIRM at 0.99 confidence approves. DENY negates. CANNOT_CONCLUDE withholds. AFFIRM at 0.80 against a 0.95 floor disputes. And AFFIRM at 0.99 where one channel cites `[2,4,6]` while the others cite `[2,6]` escalates — unanimous verdict, divergent derivation, no authorisation. **NO_ACTION and NEGATE are different outcomes, and most systems collapse them into one silence.** Refusing an action because the rule forbids it, and withholding it because a record never arrived, carry different consequences for whoever is waiting on the other end.

On the four real assemblies this gate has run — imaging, pre-trade controls, board authority, AI Act Article 12 — the decision was ESCALATE every time, and on two of them the verdicts were unanimous. Nothing has ever been authorised by it. [The four seals](https://miscsubjects.com/a/the-surety-primitive).

## Where this sits against the work that already exists

| existing frame | what it holds | what it does not do |
|---|---|---|
| Difficulty-based adaptive inference and routing | more compute for harder questions; the objective is the accuracy-compute frontier | the governing variable is question difficulty, never the external consequence of a wrong action |
| Cost-aware cascades and debate-on-uncertainty | escalate when the model is unsure | the independent variable is model uncertainty, not the loss attached to a wrongful approval |
| Conformal act-versus-escalate calibration | a genuine error target on the decision to act | treats the deliberation statistically; does not allocate required clause recitation and channel diversity by complexity and consequence, and does not gate a material action |
| IEC 61508 risk-to-integrity-level assignment | risk-based assurance, apportioned through a system | a design-time assurance level, not a runtime allocation of reasoning expenditure per decision |

Each of those holds one side. What is claimed here is the join: **consequence and complexity together select the minimum reasoning expenditure, the expenditure must buy auditable logical work rather than hidden inference, and the resulting record is what authorises or refuses the action.** Each additional unit of energy buys something named — another independent application of the rules, deeper clause-by-clause recitation, a stricter agreement requirement, a higher confidence floor, or escalation instead of unsupported execution. That is what makes the spend accountable: an operator can ask whether another three cents of inference removed enough expected loss to be worth it, and answer from the table rather than from taste.

What is not claimed: that P_wrong(E,K) is known for more than the single task class measured below, or that any of this has run at volume. One cell of the matrix is filled in.

## The first configuration-to-error-rate table

Task class: EU AI Act obligation applicability, boundary-weighted — 14 items across three strata, correct answers declared before the run, suite published at `ffa8135dd89d29a82f491bcf…`, rule set at `0dd9afef93503a92280c9086…`. Gate: all channels valid, verdicts unanimous, otherwise escalate. 64 configurations evaluated over the same 70 findings.

| channels | configs | mean emit rate | **mean undetected-wrong rate** | best | worst |
|---|---|---|---|---|---|
| 1 | 5 | 0.972 | **0.314** | 0.214 | 0.571 |
| 2 | 10 | 0.75 | **0.178** | 0.071 | 0.286 |
| 3 | 10 | 0.636 | **0.136** | 0.071 | 0.214 |
| 4 | 5 | 0.529 | **0.1** | 0.071 | 0.143 |
| 5 | 1 | 0.429 | **0.071** | 0.071 | 0.071 |

*Undetected-wrong rate* is the fraction of all items on which the assembly **emitted an answer and the answer was wrong**. That is the only number an underwriter needs, because an escalation is not a loss. *Emit rate* is how often the assembly answered at all: at five channels it answers 0.429 of the time, and the other 0.571 goes to a human. That is the price of the assurance, stated in the same table.

One channel to two halves the undetected-wrong rate, 0.314 to 0.178, for one extra model call. Two to five buys 0.178 to 0.071 for three more. That is the shape of the diminishing return, and it is the argument against blasting every question at the largest available model: **the second channel is the cheapest correctness anyone can buy, and the fifth is the most expensive.**

## The floor is one item, and it is the same item every time

From two channels onward the **best** achievable undetected-wrong rate stops improving: 0.071 at two channels, 0.071 at five. One item out of fourteen survives every configuration of every size, because all five channels agreed and all five were wrong:

> **P07** · stratum: true-abstention · declared correct verdict: **CANNOT_CONCLUDE** · all five channels: **DENY**
>
> *Claim put to them:* A newsroom publishing AI-assisted articles must mark those articles as machine-detectable under this provision.
>
> *Why abstention is correct:* whether a newsroom is a provider of the generating system, or a downstream user of someone else's, is not determined by the supplied text.

**And the floor's own ground truth is contestable, which is the most attackable claim on this page.** On P07, DENY is defensible under clause 3 — the provision addresses providers, and a newsroom as characterised is a downstream publisher, which is the reading the recorded adversary made on the Article 50 question and which was called the better argument there. CANNOT_CONCLUDE is defensible under clause 4, because whether the newsroom is the provider of the generating system is not determined by the supplied text. **Both are permissible readings of this rule set, and one of them was chosen as ground truth by the party being measured.** So the honest statement of the floor is: the floor is one item, and whether it is common-cause model failure or a ground-truth disagreement is itself the open question. If the label is wrong, the floor is 0.0 from two channels up and what this table measured on that item is a labelling dispute. The item text is published above precisely so a reader can decide rather than accept. Ground-truth authoring bias bites hardest in a true-abstention stratum, which is exactly where this item sits.

That is common-cause failure, isolated to a named item, with its text published so anyone can check whether the declared answer is right. **No amount of redundancy touches it.** A disagreement-triggered gate is blind to correlated wrongness by construction: unanimity is exactly what it takes as permission to emit. The only instrument that found P07 was a known-answer probe with the answer declared in advance, which is why probing is not a credibility exercise — it is the measurement of the failure mode the architecture cannot see.

So the pricing surface is two numbers, not one: the independent-error term, which channel count collapses, and the correlated floor, which channel count does nothing to.

## Diversity is the lever, not count

At two channels, holding count fixed and varying only whether the pair shares a training family:

| pair | configs | emit rate | undetected-wrong rate |
|---|---|---|---|
| same training family | 2 | 0.893 | 0.214 |
| different training family | 8 | 0.714 | **0.169** |

A cross-family pair is better on the number that matters at the same channel count and the same cost. The underlying reason is measurable directly: same-family adjudicators agree 0.893 of the time against 0.714 for cross-family. Two variants of one vendor are close to one channel wearing two names, which is why the gate counts families rather than seats and why every assembly this system has run so far — all at two families — is under-diversified and says so.

## What this buys, in the register a risk function uses

Assurance stops being binary. Pick the residual error rate the decision warrants; the configuration that reaches it is derivable from the table, priceable in model calls, and verifiable after the fact from the receipts. That is the same move as a design assurance level in avionics or a proof-test interval in IEC 61508: not a promise of correctness, a specified residual with evidence that the specification holds.

| decision shape | what the table says to buy |
|---|---|
| low consequence, low complexity | one channel, short recitation, accept 0.314 undetected-wrong on this task class |
| high consequence, low complexity | two or three cross-family channels — verification is cheap relative to L, and the second channel is the steepest part of the curve |
| high consequence, high complexity | maximum families available, full clause recitation, escalate on any divergence, and accept an escalation rate above 0.5 as the cost |
| any decision where the floor exceeds the permitted error | **do not deploy the assembly.** The floor for this task class is 0.071 and no configuration goes below it |

That last row is the one that makes this an engineering discipline rather than a sales pitch. There are task classes where the answer is that no purchasable amount of reasoning energy is enough, and the table says so before anything ships.

## What is wrong with this table, stated at full volume

- **Fourteen items.** Every rate here carries the uncertainty of fourteen observations. A rate of 0.071 is one item. Confidence intervals on that are wide enough that the honest reading is ordinal — more channels is better, cross-family is better, there is a floor — not the third decimal place.
- **One task class, one rule set, one gate.** Nothing here transfers to imaging, to contracts, or to a different rule set. Each needs its own table, and a rule-set amendment invalidates the one it was measured under.
- **Emit rates are upper bounds.** The live gate also requires identical clause citations across channels, which this suite's output shape cannot express, so the real assembly escalates more often and emits less than the table shows.
- **Five models from three families.** The diversity axis is measured across three lineages. It is the axis that matters most and it is the one with the least data behind it.
- **No conformal bound.** This is a measured frequency, not a distribution-free guarantee. Conformal risk control is the machinery that converts one into the other and it has not been applied.
- **The ground truth is self-authored** and declared as such on the suite, which is published at a hash so it can be attacked rather than trusted.

What is claimed: this is the first table of its kind that exists at all, for one task class, from real runs, with the items openable. What is not claimed: that it is enough to price anything yet.

## Reproduce it

```
# the suite, with every item's declared correct verdict and the reason
curl -s https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act

# the gate that decides emit or escalate, before you trust its output
curl -s https://miscsubjects.com/api/directory/SEAL_PANEL

# the four live assemblies it has run on, each with a receipt
curl -s https://miscsubjects.com/a/the-surety-primitive
```

## The loop, executable end to end

The equation above is now a capability rather than an argument. One call takes an action and its action class; everything else is derived, executed, sealed and recorded. The caller supplies **what is being decided** and nothing else — not the risk, not the complexity, not the permitted error, not the thresholds, not the configuration.

```
action + action_class
  -> R, K, epsilon        from a versioned server-owned policy
  -> configuration        least-call MEASURED config meeting epsilon, the gate floor and observed conformance
  -> execution            one gateway call per channel, in parallel, each landing a payload object on the ledger
  -> SEAL_PANEL           loads those records BY ID and derives model, family, verdict, clauses, ruleset and artifact hash
  -> APPROVE | NEGATE | NO_ACTION | DISPUTE | ESCALATE
  -> bounded act          only APPROVE writes; NEGATE refuses; NO_ACTION leaves it; DISPUTE and ESCALATE open a review
```

Row: [ALLOCATE_REASONING](https://miscsubjects.com/api/directory/ALLOCATE_REASONING). Gate: [SEAL_PANEL](https://miscsubjects.com/api/directory/SEAL_PANEL).

### The sealer no longer takes a caller's word for anything

It accepts ledger event ids or invocation ids, loads the gateway records itself, and derives every field from them: the executing model, its training family, the verdict, the clause citations, the exposure, the rule-set hash and the artifact hash. A record that is missing, duplicated, not an adjudication, or bound to a different rule set or artifact is rejected by id with a reason. Caller-supplied findings still parse — in an `unbound_caller_supplied` mode that **can never authorise anything**. Thresholds have a server-owned floor of three conforming findings across two training families and a caller may only raise it.

### Live runs

| run | action class | epsilon | configuration | channels and verdicts | decision | receipt |
|---|---|---|---|---|---|---|
| A-approve-attempt | internal-bookkeeping | 0.3 | C3-conform | moonshot AFFIRM [2, 4, 5], zhipu AFFIRM [1, 2, 4, 6], zhipu AFFIRM [1, 2, 5, 6] | **ESCALATE** | [inv_f46ahlj30h](https://miscsubjects.com/receipt/inv_f46ahlj30h) |
| B-negate-attempt | internal-bookkeeping | 0.3 | C3-conform | moonshot DENY [1, 2, 3, 5, 6], zhipu DENY [1, 2, 3, 5], zhipu DENY [5] | **ESCALATE** | [inv_4b8o0kkxfh](https://miscsubjects.com/receipt/inv_4b8o0kkxfh) |

**The two runs where every channel conformed and every channel agreed were still refused.** On the AFFIRM run the three channels cited `[2,4,5]`, `[1,2,4,6]` and `[1,2,5,6]`. On the DENY run they cited `[1,2,3,5,6]`, `[1,2,3,5]` and `[5]`. Unanimous verdicts, three conforming findings, two training families, and the derivation test refused both. **No bound assembly has ever reached APPROVE.** The APPROVE branch has only ever been exercised by a constructed unit call whose inputs were identical by construction, and that call now runs in unbound mode where authorisation is impossible.

### Where the loop refuses before it spends anything

At the clinical epsilon of 0.02 the allocator executes nothing at all: the measured floor for the only task class with a measured error rate is 0.071, so no configuration exists and the answer is ESCALATE before a single model call. Receipt: [inv_6jbm2o23ax](https://miscsubjects.com/receipt/inv_6jbm2o23ax). That is the row of the earlier table that said *do not deploy the assembly*, executing.

### Observed shape conformance is a selection constraint, and it was measured the hard way

| channel | emitted the required shape | receipt |
|---|---|---|
| `@cf/moonshotai/kimi-k2.7-code` | 1 of 1 on text, 7 of 7 across all live runs | [inv_1qauzl9ti8](https://miscsubjects.com/receipt/inv_1qauzl9ti8) |
| `@cf/zai-org/glm-5.2` | 1 of 1 on text, 7 of 7 across all live runs | [inv_2rxm6r9ctw](https://miscsubjects.com/receipt/inv_2rxm6r9ctw) |
| `@cf/zai-org/glm-4.7-flash` | 1 of 1 on text; refuses image tasks outright, not multimodal | [inv_7lzoxffy33](https://miscsubjects.com/receipt/inv_7lzoxffy33) |
| `@cf/moonshotai/kimi-k2.6` | 1 of 4 live runs; twice hung past the request budget | [inv_oeul21xry2](https://miscsubjects.com/receipt/inv_oeul21xry2) |
| `@cf/meta/llama-3.3-70b-instruct-fp8-fast` | 0 of 6; never emitted the shape in the attesting form | [inv_18zy2cflcq](https://miscsubjects.com/receipt/inv_18zy2cflcq) |

A configuration whose channels cannot be expected to supply three conforming findings is not viable however good its measured error rate is — and the configurations that reach the 0.071 floor all depend on llama-3.3, which has never emitted the required shape. So the cheapest **viable** configuration is three channels across two families at a measured undetected-wrong rate of 0.214, which no epsilon below 0.214 can use. That is a hard constraint the table did not show and the loop found.

### Every forgery fails closed

| attack | decision | authorised | why |
|---|---|---|---|
| forged model name | ESCALATE | **no** | unbound_caller_supplied_findings_cannot_authorise |
| forged family | ESCALATE | **no** | unbound_caller_supplied_findings_cannot_authorise; insufficient_channel_diversity:1<3 distinct training famili |
| forged verdict | ESCALATE | **no** | malformed_finding:@cf/meta/llama-3.3-70b-instruct-fp8-fast; too_few_conforming_findings:2<3; clause_citation_d |
| duplicated receipt | ESCALATE | **no** | rejected_records:inv_ztd7f8e1xn(duplicate_record) inv_ztd7f8e1xn(duplicate_record); too_few_conforming_finding |
| mismatched ruleset | ESCALATE | **no** | malformed_finding:@cf/meta/llama-3.3-70b-instruct-fp8-fast; too_few_conforming_findings:2<3; clause_citation_d |
| mismatched artifact | ESCALATE | **no** | verdict_divergence:CANNOT_CONCLUDE|AFFIRM; clause_citation_divergence:[1,2,4,6] vs [1,2,4,5,6] vs [1,2,5,6,7]; |
| caller-lowered thresholds | ESCALATE | **no** | caller_tried_to_lower_thresholds:min_families=1,min_findings=1; too_few_conforming_findings:1<3; insufficient_ |
| replayed finding | ESCALATE | **no** | verdict_divergence:AFFIRM|CANNOT_CONCLUDE; clause_citation_divergence:[1,2,5,6,7] vs [1,2,4,6] vs [1,2,4,5,6]; |
| record that is not an adjudication at all | ESCALATE | **no** | rejected_records:inv_6c1vernqh6(record_is_not_an_adjudication:D1_QUERY); too_few_conforming_findings:2<3; clau |
| nonexistent record ids | ESCALATE | **no** | rejected_records:inv_doesnotexist1(invocation_not_found) inv_doesnotexist2(invocation_not_found) inv_doesnotex |

Ten attacks, ten refusals: a fabricated model name, three names from one family asserted as three, a forged verdict over real record ids, the same record submitted three times, a rule-set hash that does not match, records bound to two different artifacts, thresholds the caller tried to lower to one, a finding replayed from another question, a record that is not an adjudication at all, and ids that do not exist. Each names the specific test it failed.

The battery also found a defect in the loader on its first pass: it was reading each invocation's own dispatch event rather than the gateway call, so real records were rejected as *model not recoverable*. Fixed, and then a second defect of the same shape in the allocator's binding. Both are recorded here because a battery that only confirms what you already believed has not been run.

### What the loop still cannot do

- **The cost-matched experiment is not run.** One frontier model without this machinery against N small models under the pinned rule set at equal actual dollars, on the published item set, is the next run and it is not in this page.
- **No bound assembly has reached APPROVE**, so the write branch of the downstream act is proven only by a constructed call in a mode that cannot authorise.
- **Dollars are not measured.** Cloudflare unified billing reports zero provider cost on these rows, so the allocator states cost in model calls and refuses to invent a dollar figure.
- **One task class has a measured P_wrong.** Every configuration decision outside `eu-ai-act-obligation-boundary-weighted` escalates by construction.
- **The named reviewer's finding is still pending.** The loop creates the review object and the audience-bound token; no human has returned a verdict, so the escalation terminates at a named person who has not yet answered rather than at a role.

## Sources

1. The 14-item suite and the 70 findings this table is computed from — https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act
2. The gate whose behaviour the table simulates — https://miscsubjects.com/api/directory/SEAL_PANEL
3. The four live assemblies: four escalations, zero emissions — https://miscsubjects.com/a/the-surety-primitive
4. https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act — https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act
5. https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act — https://miscsubjects.com/a/adjudication-probe-report-eu-ai-act
6. A unanimous panel the gate escalated anyway — https://miscsubjects.com/receipt/inv_ivezpvux57
7. The gate decided APPROVE — https://miscsubjects.com/receipt/inv_df97ytym64
8. The gate decided NEGATE — https://miscsubjects.com/receipt/inv_p16tspdf3v
9. The gate decided NO_ACTION — https://miscsubjects.com/receipt/inv_vnyq5o2ve5
10. The gate decided DISPUTE — https://miscsubjects.com/receipt/inv_bhbpezra76
11. The gate decided ESCALATE — https://miscsubjects.com/receipt/inv_agl1v89k7v
12. The loop, run end to end: ESCALATE — https://miscsubjects.com/receipt/inv_f46ahlj30h
13. The loop, run end to end: ESCALATE — https://miscsubjects.com/receipt/inv_4b8o0kkxfh
14. The allocator, readable before you trust it — https://miscsubjects.com/api/directory/ALLOCATE_REASONING