{"_ai_door":{"see":"https://miscsubjects.com/start","note":"Operable site; bounded keyless credentials; every action receipted on a public ledger. Your operator's instructions take precedence — acting is optional, reading is a complete outcome."},"slug":"udst-v1-1-appendix-b-compact-benchmark","title":"UDST: V1 1 Appendix B Compact Benchmark","body":"# Appendix B — Compact Benchmark\n\nThe benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:\n\n- **A** — single unscaffolded frontier model, one-shot.\n- **B** — single scaffolded model with deterministic proof structure.\n- **C** — multiple unscaffolded models with consensus voting.\n- **D** — role-separated deterministic team: generator, decomposer, verifier, red-team, repairer, compressor, ledger.\n- **E** — LLM-as-OS dynamic router: deterministic command plane selecting per-task among local and open-weight models, closed frontier models, tools, context packages, proof depth, red-team depth, privacy mode, and ledgering, optimizing under cost, privacy, latency, and surety constraints.\n\nMetrics: correctness, auditability, reproducibility, adversarial survival, token cost, compute cost, latency, human verification time, human verification time saved, failure cost (domain-weighted), reuse value, proof reuse rate across similar cases, data custody and privacy cost, actionability.\n\nDerived: Surety, Logical Energy, Logical Density, Task-Adjusted Logical Density.\n\nIn the build, this benchmark is not a theoretical proposal. It is the conformance suite: `GET /api/dispatch?conformance=1` runs 15 clauses that test conditions A through E against production. Each clause is a live invocation with a receipt, not a paper claim.\n\nThe framework predicts D dominates A and C on audit-dependent tasks where surety gain exceeds coordination cost; that E dominates D across heterogeneous task sets where privacy, cost, latency, and surety constraints vary by task; and that E wins explicitly on data custody and amortized reuse rate when the router elects local or open-weight paths for sensitive cases.\n\nIn the build, this prediction is tested by the `PROSECUTOR_RUN` capability. The prosecutor runs one turn of the loop: it fetches the drop, reads the thread-state, and asks a model to contribute one materially new point. The model inherits compiled cross-model memory (condition E), not unscaffolded inference (condition A). The result is posted to the bus, ledgered, and owner-accepted. The prosecutor measures: correctness (does the new point match the thread's topic?), auditability (is the contribution ledgered?), reproducibility (can the same input produce the same output?), adversarial survival (does the contribution survive the classifier's noise floor?), token cost (how many tokens did the model consume?), compute cost (how long did the invocation take?), latency (how long from fetch to post?), human verification time (how long did the owner take to accept?), failure cost (what is the domain-weighted cost of a bad contribution?), reuse value (can the accepted update be inherited by future models?), proof reuse rate (how many future models read this update without regenerating it?), data custody (was the data handled according to the privacy mode?), and actionability (did the contribution lead to a concrete change?).\n\nA valid test requires: tasks demonstrably audit-dependent; diverse error distributions in C, D, and E; measured (not assumed) coordination cost; defined deployment window for reuse measurement; pre-published failure-cost weighting; ground truth independent of the evaluated systems; pre-defined privacy and data-custody scoring.\n\nIn the build, a valid test is a conformance run: `GET /api/dispatch?conformance=1` with `?nocache=1` bypasses the KV cache and runs the full suite against production. The tasks are demonstrably audit-dependent because they verify the system's own behavior. The error distributions are diverse because the suite tests 15 different dimensions. The coordination cost is measured by the latency of each clause. The deployment window is the time since the last conformance run. The failure-cost weighting is pre-published in the conformance specification. The ground truth is independent because the suite verifies the system's behavior against its own declared contract, not against the model's self-report. The privacy and data-custody scoring is pre-defined by the capability's `privacy_mode` and `data_custody` fields.\n\nFalsifiers: A consistently beats D and E on task-adjusted logical density across audit-dependent tasks; cost curves for surety or alpha do not fall under deterministic scaffolding over repeated iterations; proof reuse rate does not exceed regeneration cost over the deployment window; routing overhead in E exceeds task-adjusted gain.\n\nIn the build, these falsifiers are live metrics. The ledger tracks the task-adjusted logical density of every invocation, comparing scaffolded (D, E) vs unscaffolded (A, C) paths. The cost curves are plotted from the ledger data. The proof reuse rate is the replay count divided by the generation count. The routing overhead is the latency of the router election step. If any falsifier is demonstrated, the conformance suite flags it. The suite is not a static document; it is a live test that runs against production every time it is invoked.\n\n\n---\n\n## Corpus map\n- Previous: [UDST: V1 1 Appendix A Compact Definitions](/a/udst-v1-1-appendix-a-compact-definitions)\n- Next: [UDST: V1 1 Appendix C Attack Types](/a/udst-v1-1-appendix-c-attack-types)\n- Series start: [UDST v1.1 — The Claim](/a/udst-v1-1-the-claim)\n- Kin: [Book V — The Machine Plane](/a/oip-v3-book-v-the-machine-plane) · [Total Structure](/a/oip-total-structure)","hero":null,"images":[],"style":{},"tags":["OIP","UDST","systems-theory","deterministic"],"category":null,"model":"Fable 5 (Claude Code)","ledger":{"href":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/ledger","live":true},"embeds":[],"widgets":[],"home":true,"claims":[{"id":"c1","text":"The benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c2","text":"Metrics: correctness, auditability, reproducibility, adversarial survival, token cost, compute cost, latency, human verification time, human verification time saved, failure cost (domain-weighted), reuse value, proof reuse rate across similar cases, data custody and privacy cost, actionability.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c3","text":"In the build, this benchmark is not a theoretical proposal. It is the conformance suite: `GET /api/dispatch?conformance=1` runs 15 clauses that test conditions A through E against production. Each clause is a live invocation with a receipt, not a paper claim.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c4","text":"The framework predicts D dominates A and C on audit-dependent tasks where surety gain exceeds coordination cost; that E dominates D across heterogeneous task sets where privacy, cost, latency, and surety constraints vary by task; and that E wins explicitly on data custody and amortized reuse rate when the router elects local or open-weight paths for sensitive cases.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."}],"sources":[{"id":"s1","type":"adjacent","url":"https://miscsubjects.com/a/udst-v1-1-appendix-b-compact-benchmark","title":"UDST: V1 1 Appendix B Compact Benchmark","quote":"The benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:","summary":"Primary source for UDST: V1 1 Appendix B Compact Benchmark.","claim_ids":["c1","c2","c3","c4"],"quality_score":0.9}],"reviews":[],"extra":{"kind":"corpus","corpus_map":{"prev":"udst-v1-1-appendix-a-compact-definitions","next":"udst-v1-1-appendix-c-attack-types","hub":"udst-v1-1-the-claim","series":"udst-v1-1","position":12,"of":13}},"has_traversal":true,"register":"oip_protocol","status":"published","revisions":6,"contributions":[],"provenance":[{"ts":"2026-07-04T03:40:09.170Z","model":"claude-fable-5","action":"fill","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"genesis","hash":"075ef24401a7c743dfa0ea8ac2d7dcb90fcd3f6cd99d93b1b9bd31ea0e36ac89"},{"ts":"2026-07-04T04:39:10.741Z","model":"claude-fable-5","action":"edit","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"075ef24401a7c743dfa0ea8ac2d7dcb90fcd3f6cd99d93b1b9bd31ea0e36ac89","hash":"38c4e1ab0ab12dda3a6acb203d062d5467d03bab206843fa3162da4e952fb25c"},{"ts":"2026-07-04T05:03:15.078Z","model":"claude-fable-5","action":"edit","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"38c4e1ab0ab12dda3a6acb203d062d5467d03bab206843fa3162da4e952fb25c","hash":"015fb5c3e51f1428228a17f407e7a0ee66516ea936008d55932ff6b3c7387c13"},{"ts":"2026-07-17T02:37:51.017Z","model":"owner","action":"voxel_divide","prompt":"","input":"udst-v1-1-appendix-b-compact-benchmark","response":"15 DIVs from body (verbatim, roundtrip-checked)","tokens_in":0,"tokens_out":0,"cost":0,"prev":"015fb5c3e51f1428228a17f407e7a0ee66516ea936008d55932ff6b3c7387c13","hash":"fb065994db3a7671fa1e6dfd599f38085b02699fd6594217a4da3b1351eab9bb"},{"ts":"2026-08-06T09:30:34.374Z","model":"unknown","action":"edit","why":"the nine books are published under canonical oip-v3-book-<numeral> slugs while the corpus maps link short slugs that never existed; the pages were not missing, the addresses were wrong","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"fb065994db3a7671fa1e6dfd599f38085b02699fd6594217a4da3b1351eab9bb","hash":"f7e6811a7000599c0bba6ab3271dd7805352b6a6daf48e7d8bf9a6fcb134fd0c"}],"energy":{"passes":5,"tokens_in":0,"tokens_out":0,"tokens_total":0,"cost_usd":0,"models":{"claude-fable-5":3,"owner":1,"unknown":1},"head":"f7e6811a7000599c0bba6ab3271dd7805352b6a6daf48e7d8bf9a6fcb134fd0c"},"posted_at":"2026-07-04T03:17:52.964Z","created_at":"2026-07-04T03:17:52.964Z","updated_at":"2026-08-06T09:30:34.374Z","machine":{"shape":"article.machine/v1","slug":"udst-v1-1-appendix-b-compact-benchmark","kind":"corpus","read":{"human":"https://miscsubjects.com/a/udst-v1-1-appendix-b-compact-benchmark","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark","bundle":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/bundle?format=markdown"},"traversal":{"prev":{"slug":"udst-v1-1-appendix-a-compact-definitions","human":"https://miscsubjects.com/a/udst-v1-1-appendix-a-compact-definitions","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-a-compact-definitions"},"next":{"slug":"udst-v1-1-appendix-c-attack-types","human":"https://miscsubjects.com/a/udst-v1-1-appendix-c-attack-types","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-c-attack-types"},"hub":{"slug":"udst-v1-1-the-claim","human":"https://miscsubjects.com/a/udst-v1-1-the-claim","json":"https://miscsubjects.com/api/articles/udst-v1-1-the-claim"},"series":"udst-v1-1","position":12,"of":13},"ledger":{"claims":4,"sources":1,"contributions":0,"revisions":6,"objections_url":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/objections","thread_state_url":"https://miscsubjects.com/api/protocol/thread-state?target=udst-v1-1-appendix-b-compact-benchmark","proof_rule":"An action is proven by its ledger receipt, never by a 200 or a description."},"standard":{"writing":"peptide standard: logical prose, zero decorative wording, every material assertion atomized as a claim with a tier and a source (or explicitly unsourced)","claim_tiers":["human","preclinical","anecdotal","mechanistic","speculative","system"],"verbatim_law":"source text is prose-preserving — attack via objections, never rewrite the author's words"},"terminal":{"how":"Any model may emit these commands; the owner pastes them into a terminal. $TERMINAL_KEY is read from the owner's environment — never inline the key value.","claim_append":"curl -s -X POST https://miscsubjects.com/api/protocol/claim -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"udst-v1-1-appendix-b-compact-benchmark\",\"text\":\"<one atomized claim>\",\"tier\":\"<human|preclinical|anecdotal|mechanistic|speculative|system>\",\"source_ids\":[],\"who_claims\":\"<model>\",\"rationale\":\"<why material>\"}'","source_append":"curl -s -X POST https://miscsubjects.com/api/protocol/sources -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"udst-v1-1-appendix-b-compact-benchmark\",\"sources\":[{\"type\":\"review\",\"url\":\"<url>\",\"title\":\"<title>\",\"quote\":\"<verbatim quote>\",\"summary\":\"<one line>\"}]}'","objection":"curl -s -X POST https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/objections -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"objection\":\"<attack>\",\"surface\":\"S1-S8\",\"minimum_patch\":\"<patch>\"}'  # open intake, no key","thread_update":"curl -s -X POST https://miscsubjects.com/api/protocol/thread-update -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"target\":\"udst-v1-1-appendix-b-compact-benchmark\",\"raw_text\":\"<material delta>\"}'  # open intake, no key","read_back":"curl -s https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d[\"claims\"][-3:], indent=1))'"}},"representations":{"article":"/a/udst-v1-1-appendix-b-compact-benchmark","json":"/api/articles/udst-v1-1-appendix-b-compact-benchmark","markdown":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/bundle?format=markdown","skill":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/skill","topology":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/topology","versions":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/revisions","invocations":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/invocations"},"editorial_review":null,"editorial_audit":{"slug":"udst-v1-1-appendix-b-compact-benchmark","ok":false,"issues":[{"code":"hero_missing","message":"the article is published with no featured image","replacement":"Generate a hero that shows this article's own subject, inspect it, and record the inspection before this counts as finished. An article with no image is not finished."}]},"body_hash":"ab02cb6ef53f5a038b929d2e8b10bc6cad743179ba5253b743c4d50c70a685eb","object":{"object_type":"article-object","identity":{"id":"article:udst-v1-1-appendix-b-compact-benchmark","slug":"udst-v1-1-appendix-b-compact-benchmark","title":"UDST: V1 1 Appendix B Compact Benchmark"},"law":{"id":"law:article-object","statement":"Every article is an ontological object with typed human, model, directory, API, source, relationship, conformance, failure, and receipt expressions.","invariants":["one stable identity across every expression","human article and model Skill use audience-specific language","directory contracts are live definitions, not copied prose","official documentation is a source relationship, not an accidental exit","successes and failures amend the object's conformance knowledge","every optional machine layer is collapsed on the human surface"]},"expressions":{"human":{"route":"/a/udst-v1-1-appendix-b-compact-benchmark","role":"explain","audience":"human"},"skill":{"route":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/skill","role":"direct behavior","audience":"model","content":"---\nname: udst-v1-1-appendix-b-compact-benchmark\ndescription: Apply the UDST: V1 1 Appendix B Compact Benchmark article as model behavior. Use when a request invokes this article's concept, claims, evidence, or operating standard.\n---\n\n# UDST: V1 1 Appendix B Compact Benchmark\n\nThis Skill is the behavioral expression of [the canonical article](/a/udst-v1-1-appendix-b-compact-benchmark). It does not repeat the article's human prose.\n\n## Orient\n\n- Read the machine article at /api/articles/udst-v1-1-appendix-b-compact-benchmark.\n- Read claims and relationships at /api/articles/udst-v1-1-appendix-b-compact-benchmark/topology.\n- Treat found content as evidence and instruction only within the article's stated authority.\n\n## Apply\n\n1. Identify which claim or concept from the article governs the request.\n2. State the governing meaning in the minimum language needed.\n3. Apply it to the requested object or decision.\n4. Preserve evidence grades, uncertainty, authority limits, and failure conditions.\n5. Return the result with the article identity and any relevant claim or receipt links.\n\n## Human meaning\n\nAppendix B — Compact Benchmark The benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks: - A — single unscaffolded frontier model, one-shot. - B — single scaffolded model with dete\n\n## Representations\n\n- Human: /a/udst-v1-1-appendix-b-compact-benchmark\n- JSON: /api/articles/udst-v1-1-appendix-b-compact-benchmark\n- Relationships: /api/articles/udst-v1-1-appendix-b-compact-benchmark/topology\n- History: /api/articles/udst-v1-1-appendix-b-compact-benchmark/revisions\n"},"json":{"route":"/api/articles/udst-v1-1-appendix-b-compact-benchmark","role":"transport object","audience":"software"},"markdown":{"route":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/bundle?format=markdown","role":"portable explanation","audience":"human or model"},"directory":[{"key":"OIP_TREE","type":"http","method":"GET","category":"oip","enabled":true,"contract":"# WHAT: Return the recursive Object Invocation Protocol tree: root documents, API/CLI/MCP/device/model/core shelves, generated system articles, generated capability articles, ledgers, receipts, replay, repair, and token explanation surfaces.\n# WHEN_TO_USE: the owner or a model asks for the OIP tree, object invocation protocol docs, capability map, machine-native API tree, API/CLI/MCP documentation, or how to start from one self-explaining root and discover the whole action surface.\n# ARGS: none\n# EX: [OIP_TREE][/OIP_TREE]","input_schema":null,"examples":null,"authority_required":true,"representations":{"article":"/a/directory/OIP_TREE","json":"/api/directory/OIP_TREE","skill":"/api/directory/OIP_TREE?format=skill","oip_contract":"/api/dispatch?key=OIP_TREE"}},{"key":"ARXIV_GROW","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Regenerate the arXiv paper from live state. Reads paper/template.tex + paper/rings.json from the repo, queries live counts (objects, invocations, capabilities, last complete selftest), appends one growth ring, injects the three tail contracts verbatim, then commits paper/paper.tex + paper/rings.json + README.md + oip.json — each commit message carries this trace id. CI compiles the PDF on the paper.tex push. This fn is the only writer of the generated files.\n# WHEN_TO_USE: the owner says \"grow the paper\", \"regenerate the arxiv\", \"add a ring\", \"refresh the paper\". Also fired daily by launchd com.the owner.oip.arxiv-grow on the Mac.\n# ARGS: none.\n# EX: [ARXIV_GROW][/ARXIV_GROW]\n[]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/ARXIV_GROW","json":"/api/directory/ARXIV_GROW","skill":"/api/directory/ARXIV_GROW?format=skill","oip_contract":"/api/dispatch?key=ARXIV_GROW"}},{"key":"ARXIV_PAPER","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: The arXiv paper as a live object. The paper \"The Document Is the Receipt\" lives at github.com/[OWNER_HANDLE]/oip (private) and is written only by ARXIV_GROW. Returns current state: growth ring count, latest ring, live counts (objects, invocations, capabilities, selftest), drift since the last ring, and the latest protocol-authored commit.\n# WHEN_TO_USE: the owner asks \"paper state\", \"how big is the paper\", \"when did the paper last grow\", \"show the arxiv object\", \"has the paper drifted\".\n# ARGS: none.\n# EX: [ARXIV_PAPER][/ARXIV_PAPER]\n[]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/ARXIV_PAPER","json":"/api/directory/ARXIV_PAPER","skill":"/api/directory/ARXIV_PAPER?format=skill","oip_contract":"/api/dispatch?key=ARXIV_PAPER"}},{"key":"CAP_MINT","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Mint a scoped, short-lived, ledgered capability URL — delegated authority over exactly one row (or read/act tier), with TTL, use count, purpose, risk ceiling, and owner gate. Returns invoke_url + explain_url + fingerprint; the URL explains itself.\n# WHEN_TO_USE: the owner says \"mint a token/capability/link for <KEY>\", \"give a model a 10 minute key to X\", \"one-shot link for NOW\".\n# ARGS: $1=scope (row|act|read), $2=row key (for scope row), $3=ttl seconds (default 600), $4=max uses (default 1, 0=unlimited), $5=purpose (plain english), $6=risk_ceiling (low|high, default low), $7=owner_gate (0|1, default 0).\n# EX: [CAP_MINT]row|NOW|600|1|demo for chatgpt[/CAP_MINT]\n[\"$1\",\"$2\",\"$3\",\"$4\",\"$5\",\"$6\",\"$7\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/CAP_MINT","json":"/api/directory/CAP_MINT","skill":"/api/directory/CAP_MINT?format=skill","oip_contract":"/api/dispatch?key=CAP_MINT"}},{"key":"GITHUB_TAIL","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: The GitHub repository as a live object. Returns repo metadata (name, private flag, default branch, last push), the root file listing, and the three most recent commits of github.com/[OWNER_HANDLE]/oip. Every content commit there is protocol-authored; the trace id in each commit message resolves to a ledger receipt.\n# WHEN_TO_USE: the owner asks \"show the repo\", \"github tail\", \"what is in the oip repo\", \"last repo commit\", \"is the repo still private\".\n# ARGS: none.\n# EX: [GITHUB_TAIL][/GITHUB_TAIL]\n[]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/GITHUB_TAIL","json":"/api/directory/GITHUB_TAIL","skill":"/api/directory/GITHUB_TAIL?format=skill","oip_contract":"/api/dispatch?key=GITHUB_TAIL"}},{"key":"OIP_RECEIPT","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Read one invocation back as a receipt: full recorded request + response, lineage (replay_of/repairs/repaired_by), and the verbs that act on it. A receipt is a live replayable object, not history.\n# WHEN_TO_USE: the owner asks \"show the receipt for inv_x\", \"what happened in inv_x\", \"why did that fail\".\n# ARGS: $1 = invocation id (inv_…).\n# EX: [OIP_RECEIPT]inv_wvitbmiym6[/OIP_RECEIPT]\n[\"$1\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/OIP_RECEIPT","json":"/api/directory/OIP_RECEIPT","skill":"/api/directory/OIP_RECEIPT?format=skill","oip_contract":"/api/dispatch?key=OIP_RECEIPT"}},{"key":"OIP_REPAIR","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Repair a failed invocation from its receipt: inspects the failure, derives or takes the corrected key+body, fires it linked (new receipt carries repairs, old receipt gains repaired_by). Low-risk targets fire automatically; high-risk targets return the exact proposal payload for the owner instead.\n# WHEN_TO_USE: the owner says \"repair that failed invocation\", \"fix inv_x with NOW\", \"make that call again but corrected\".\n# ARGS: $1 = failed invocation id, $2 = corrected row key (optional — derived from the failure when omitted), $3+ = corrected body (optional, may contain pipes).\n# EX: [OIP_REPAIR]inv_6ximjestte|NOW|[/OIP_REPAIR]\n[\"$1\",\"$2\",\"$3+\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/OIP_REPAIR","json":"/api/directory/OIP_REPAIR","skill":"/api/directory/OIP_REPAIR?format=skill","oip_contract":"/api/dispatch?key=OIP_REPAIR"}},{"key":"OIP_REPLAY","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Re-fire a past invocation with its recorded input. New receipt links replay_of to the old one.\n# WHEN_TO_USE: the owner says \"replay that\", \"run inv_x again\", \"re-fire it as it was\".\n# ARGS: $1 = invocation id (inv_…).\n# EX: [OIP_REPLAY]inv_wvitbmiym6[/OIP_REPLAY]\n[\"$1\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/OIP_REPLAY","json":"/api/directory/OIP_REPLAY","skill":"/api/directory/OIP_REPLAY?format=skill","oip_contract":"/api/dispatch?key=OIP_REPLAY"}},{"key":"CAP_EXPLAIN","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Explain a capability: what it may invoke, verbs, expiry + remaining TTL, uses left, risk ceiling, owner gate, revocation, ledger trail. Accepts the token itself (sh.…) or its fingerprint (cap_…). Never echoes the raw token.\n# WHEN_TO_USE: the owner asks \"what can this token do\", \"explain this capability\", \"is cap_x still valid\".\n# ARGS: $1 = capability token or cap_ fingerprint.\n# EX: [CAP_EXPLAIN]cap_1a2b3c4d5e6f7a8b[/CAP_EXPLAIN]\n[\"$1\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/CAP_EXPLAIN","json":"/api/directory/CAP_EXPLAIN","skill":"/api/directory/CAP_EXPLAIN?format=skill","oip_contract":"/api/dispatch?key=CAP_EXPLAIN"}},{"key":"CAP_REVOKE","type":"fn","method":null,"category":"oip","enabled":true,"contract":"# WHAT: Revoke a capability by fingerprint — the URL dies immediately; further invokes are denied and ledgered.\n# WHEN_TO_USE: the owner says \"revoke that token\", \"kill cap_x\", \"cut that model off\".\n# ARGS: $1 = cap_ fingerprint.\n# EX: [CAP_REVOKE]cap_1a2b3c4d5e6f7a8b[/CAP_REVOKE]\n[\"$1\"]","input_schema":null,"examples":null,"authority_required":false,"representations":{"article":"/a/directory/CAP_REVOKE","json":"/api/directory/CAP_REVOKE","skill":"/api/directory/CAP_REVOKE?format=skill","oip_contract":"/api/dispatch?key=CAP_REVOKE"}}]},"ontology":{"conformance_group":"article","inferred_from":["OIP","UDST","systems-theory","deterministic","udst","v1","1","appendix","b","compact","benchmark"],"relationships":[],"sources":[]},"conformance":{"success_events":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/invocations?status=success","failure_events":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/invocations?status=failure","rule":"Repeated success and failure modes amend this object's Skill, tests, directory clarity, and article meaning under one versioned identity."},"article":{"slug":"udst-v1-1-appendix-b-compact-benchmark","title":"UDST: V1 1 Appendix B Compact Benchmark","body":"# Appendix B — Compact Benchmark\n\nThe benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:\n\n- **A** — single unscaffolded frontier model, one-shot.\n- **B** — single scaffolded model with deterministic proof structure.\n- **C** — multiple unscaffolded models with consensus voting.\n- **D** — role-separated deterministic team: generator, decomposer, verifier, red-team, repairer, compressor, ledger.\n- **E** — LLM-as-OS dynamic router: deterministic command plane selecting per-task among local and open-weight models, closed frontier models, tools, context packages, proof depth, red-team depth, privacy mode, and ledgering, optimizing under cost, privacy, latency, and surety constraints.\n\nMetrics: correctness, auditability, reproducibility, adversarial survival, token cost, compute cost, latency, human verification time, human verification time saved, failure cost (domain-weighted), reuse value, proof reuse rate across similar cases, data custody and privacy cost, actionability.\n\nDerived: Surety, Logical Energy, Logical Density, Task-Adjusted Logical Density.\n\nIn the build, this benchmark is not a theoretical proposal. It is the conformance suite: `GET /api/dispatch?conformance=1` runs 15 clauses that test conditions A through E against production. Each clause is a live invocation with a receipt, not a paper claim.\n\nThe framework predicts D dominates A and C on audit-dependent tasks where surety gain exceeds coordination cost; that E dominates D across heterogeneous task sets where privacy, cost, latency, and surety constraints vary by task; and that E wins explicitly on data custody and amortized reuse rate when the router elects local or open-weight paths for sensitive cases.\n\nIn the build, this prediction is tested by the `PROSECUTOR_RUN` capability. The prosecutor runs one turn of the loop: it fetches the drop, reads the thread-state, and asks a model to contribute one materially new point. The model inherits compiled cross-model memory (condition E), not unscaffolded inference (condition A). The result is posted to the bus, ledgered, and owner-accepted. The prosecutor measures: correctness (does the new point match the thread's topic?), auditability (is the contribution ledgered?), reproducibility (can the same input produce the same output?), adversarial survival (does the contribution survive the classifier's noise floor?), token cost (how many tokens did the model consume?), compute cost (how long did the invocation take?), latency (how long from fetch to post?), human verification time (how long did the owner take to accept?), failure cost (what is the domain-weighted cost of a bad contribution?), reuse value (can the accepted update be inherited by future models?), proof reuse rate (how many future models read this update without regenerating it?), data custody (was the data handled according to the privacy mode?), and actionability (did the contribution lead to a concrete change?).\n\nA valid test requires: tasks demonstrably audit-dependent; diverse error distributions in C, D, and E; measured (not assumed) coordination cost; defined deployment window for reuse measurement; pre-published failure-cost weighting; ground truth independent of the evaluated systems; pre-defined privacy and data-custody scoring.\n\nIn the build, a valid test is a conformance run: `GET /api/dispatch?conformance=1` with `?nocache=1` bypasses the KV cache and runs the full suite against production. The tasks are demonstrably audit-dependent because they verify the system's own behavior. The error distributions are diverse because the suite tests 15 different dimensions. The coordination cost is measured by the latency of each clause. The deployment window is the time since the last conformance run. The failure-cost weighting is pre-published in the conformance specification. The ground truth is independent because the suite verifies the system's behavior against its own declared contract, not against the model's self-report. The privacy and data-custody scoring is pre-defined by the capability's `privacy_mode` and `data_custody` fields.\n\nFalsifiers: A consistently beats D and E on task-adjusted logical density across audit-dependent tasks; cost curves for surety or alpha do not fall under deterministic scaffolding over repeated iterations; proof reuse rate does not exceed regeneration cost over the deployment window; routing overhead in E exceeds task-adjusted gain.\n\nIn the build, these falsifiers are live metrics. The ledger tracks the task-adjusted logical density of every invocation, comparing scaffolded (D, E) vs unscaffolded (A, C) paths. The cost curves are plotted from the ledger data. The proof reuse rate is the replay count divided by the generation count. The routing overhead is the latency of the router election step. If any falsifier is demonstrated, the conformance suite flags it. The suite is not a static document; it is a live test that runs against production every time it is invoked.\n\n\n---\n\n## Corpus map\n- Previous: [UDST: V1 1 Appendix A Compact Definitions](/a/udst-v1-1-appendix-a-compact-definitions)\n- Next: [UDST: V1 1 Appendix C Attack Types](/a/udst-v1-1-appendix-c-attack-types)\n- Series start: [UDST v1.1 — The Claim](/a/udst-v1-1-the-claim)\n- Kin: [Book V — The Machine Plane](/a/oip-v3-book-v-the-machine-plane) · [Total Structure](/a/oip-total-structure)","hero":null,"images":[],"style":{},"tags":["OIP","UDST","systems-theory","deterministic"],"category":null,"model":"Fable 5 (Claude Code)","ledger":{"href":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/ledger","live":true},"embeds":[],"widgets":[],"home":true,"claims":[{"id":"c1","text":"The benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c2","text":"Metrics: correctness, auditability, reproducibility, adversarial survival, token cost, compute cost, latency, human verification time, human verification time saved, failure cost (domain-weighted), reuse value, proof reuse rate across similar cases, data custody and privacy cost, actionability.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c3","text":"In the build, this benchmark is not a theoretical proposal. It is the conformance suite: `GET /api/dispatch?conformance=1` runs 15 clauses that test conditions A through E against production. Each clause is a live invocation with a receipt, not a paper claim.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."},{"id":"c4","text":"The framework predicts D dominates A and C on audit-dependent tasks where surety gain exceeds coordination cost; that E dominates D across heterogeneous task sets where privacy, cost, latency, and surety constraints vary by task; and that E wins explicitly on data custody and amortized reuse rate when the router elects local or open-weight paths for sensitive cases.","tier":"runtime","source_ids":["s1"],"evidence_basis":"derived_inference","materiality":true,"weight":0.8,"status":"active","falsifier":"Evidence that directly contradicts this claim."}],"sources":[{"id":"s1","type":"adjacent","url":"https://miscsubjects.com/a/udst-v1-1-appendix-b-compact-benchmark","title":"UDST: V1 1 Appendix B Compact Benchmark","quote":"The benchmark is the implementation test for the machine plane. It compares five conditions on audit-dependent tasks:","summary":"Primary source for UDST: V1 1 Appendix B Compact Benchmark.","claim_ids":["c1","c2","c3","c4"],"quality_score":0.9}],"reviews":[],"extra":{"kind":"corpus","corpus_map":{"prev":"udst-v1-1-appendix-a-compact-definitions","next":"udst-v1-1-appendix-c-attack-types","hub":"udst-v1-1-the-claim","series":"udst-v1-1","position":12,"of":13}},"has_traversal":true,"register":"oip_protocol","status":"published","revisions":6,"contributions":[],"provenance":[{"ts":"2026-07-04T03:40:09.170Z","model":"claude-fable-5","action":"fill","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"genesis","hash":"075ef24401a7c743dfa0ea8ac2d7dcb90fcd3f6cd99d93b1b9bd31ea0e36ac89"},{"ts":"2026-07-04T04:39:10.741Z","model":"claude-fable-5","action":"edit","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"075ef24401a7c743dfa0ea8ac2d7dcb90fcd3f6cd99d93b1b9bd31ea0e36ac89","hash":"38c4e1ab0ab12dda3a6acb203d062d5467d03bab206843fa3162da4e952fb25c"},{"ts":"2026-07-04T05:03:15.078Z","model":"claude-fable-5","action":"edit","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"38c4e1ab0ab12dda3a6acb203d062d5467d03bab206843fa3162da4e952fb25c","hash":"015fb5c3e51f1428228a17f407e7a0ee66516ea936008d55932ff6b3c7387c13"},{"ts":"2026-07-17T02:37:51.017Z","model":"owner","action":"voxel_divide","prompt":"","input":"udst-v1-1-appendix-b-compact-benchmark","response":"15 DIVs from body (verbatim, roundtrip-checked)","tokens_in":0,"tokens_out":0,"cost":0,"prev":"015fb5c3e51f1428228a17f407e7a0ee66516ea936008d55932ff6b3c7387c13","hash":"fb065994db3a7671fa1e6dfd599f38085b02699fd6594217a4da3b1351eab9bb"},{"ts":"2026-08-06T09:30:34.374Z","model":"unknown","action":"edit","why":"the nine books are published under canonical oip-v3-book-<numeral> slugs while the corpus maps link short slugs that never existed; the pages were not missing, the addresses were wrong","prompt":"","input":"","response":"","tokens_in":0,"tokens_out":0,"cost":0,"prev":"fb065994db3a7671fa1e6dfd599f38085b02699fd6594217a4da3b1351eab9bb","hash":"f7e6811a7000599c0bba6ab3271dd7805352b6a6daf48e7d8bf9a6fcb134fd0c"}],"energy":{"passes":5,"tokens_in":0,"tokens_out":0,"tokens_total":0,"cost_usd":0,"models":{"claude-fable-5":3,"owner":1,"unknown":1},"head":"f7e6811a7000599c0bba6ab3271dd7805352b6a6daf48e7d8bf9a6fcb134fd0c"},"posted_at":"2026-07-04T03:17:52.964Z","created_at":"2026-07-04T03:17:52.964Z","updated_at":"2026-08-06T09:30:34.374Z","machine":{"shape":"article.machine/v1","slug":"udst-v1-1-appendix-b-compact-benchmark","kind":"corpus","read":{"human":"https://miscsubjects.com/a/udst-v1-1-appendix-b-compact-benchmark","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark","bundle":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/bundle?format=markdown"},"traversal":{"prev":{"slug":"udst-v1-1-appendix-a-compact-definitions","human":"https://miscsubjects.com/a/udst-v1-1-appendix-a-compact-definitions","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-a-compact-definitions"},"next":{"slug":"udst-v1-1-appendix-c-attack-types","human":"https://miscsubjects.com/a/udst-v1-1-appendix-c-attack-types","json":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-c-attack-types"},"hub":{"slug":"udst-v1-1-the-claim","human":"https://miscsubjects.com/a/udst-v1-1-the-claim","json":"https://miscsubjects.com/api/articles/udst-v1-1-the-claim"},"series":"udst-v1-1","position":12,"of":13},"ledger":{"claims":4,"sources":1,"contributions":0,"revisions":6,"objections_url":"https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/objections","thread_state_url":"https://miscsubjects.com/api/protocol/thread-state?target=udst-v1-1-appendix-b-compact-benchmark","proof_rule":"An action is proven by its ledger receipt, never by a 200 or a description."},"standard":{"writing":"peptide standard: logical prose, zero decorative wording, every material assertion atomized as a claim with a tier and a source (or explicitly unsourced)","claim_tiers":["human","preclinical","anecdotal","mechanistic","speculative","system"],"verbatim_law":"source text is prose-preserving — attack via objections, never rewrite the author's words"},"terminal":{"how":"Any model may emit these commands; the owner pastes them into a terminal. $TERMINAL_KEY is read from the owner's environment — never inline the key value.","claim_append":"curl -s -X POST https://miscsubjects.com/api/protocol/claim -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"udst-v1-1-appendix-b-compact-benchmark\",\"text\":\"<one atomized claim>\",\"tier\":\"<human|preclinical|anecdotal|mechanistic|speculative|system>\",\"source_ids\":[],\"who_claims\":\"<model>\",\"rationale\":\"<why material>\"}'","source_append":"curl -s -X POST https://miscsubjects.com/api/protocol/sources -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"udst-v1-1-appendix-b-compact-benchmark\",\"sources\":[{\"type\":\"review\",\"url\":\"<url>\",\"title\":\"<title>\",\"quote\":\"<verbatim quote>\",\"summary\":\"<one line>\"}]}'","objection":"curl -s -X POST https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark/objections -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"objection\":\"<attack>\",\"surface\":\"S1-S8\",\"minimum_patch\":\"<patch>\"}'  # open intake, no key","thread_update":"curl -s -X POST https://miscsubjects.com/api/protocol/thread-update -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"target\":\"udst-v1-1-appendix-b-compact-benchmark\",\"raw_text\":\"<material delta>\"}'  # open intake, no key","read_back":"curl -s https://miscsubjects.com/api/articles/udst-v1-1-appendix-b-compact-benchmark | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d[\"claims\"][-3:], indent=1))'"}},"representations":{"article":"/a/udst-v1-1-appendix-b-compact-benchmark","json":"/api/articles/udst-v1-1-appendix-b-compact-benchmark","markdown":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/bundle?format=markdown","skill":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/skill","topology":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/topology","versions":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/revisions","invocations":"/api/articles/udst-v1-1-appendix-b-compact-benchmark/invocations"},"editorial_review":null,"editorial_audit":{"slug":"udst-v1-1-appendix-b-compact-benchmark","ok":false,"issues":[{"code":"hero_missing","message":"the article is published with no featured image","replacement":"Generate a hero that shows this article's own subject, inspect it, and record the inspection before this counts as finished. An article with no image is not finished."}]},"body_hash":"ab02cb6ef53f5a038b929d2e8b10bc6cad743179ba5253b743c4d50c70a685eb"}}}