{"slug":"tooling-as-data","verification":{"valid":false,"broken_at":0,"reason":"prev mismatch"},"count":33,"sources":[{"id":"s1","type":"publisher_documentation","url":"https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool","title":"Tool search tool","publisher":"Anthropic","author":"Anthropic","date":"2026-07-26","quote":"defer_loading controls what enters the context window, not what you send in the request: You still send every tool's full definition in the tools array on every request, including the deferred ones. The API needs them server-side to run the search and expand tool_reference blocks.","summary":"The vendor's own statement that deferral removes definitions from context but not from the request. Positive for the mechanism, and the exact fact that makes catalogue size a structural property of the design rather than a billing artefact. Also states the ~55k-token five-server figure and the 30-50 tool accuracy ceiling.","claim_ids":["c3","c4","c5"]},{"id":"s2","type":"publisher_documentation","url":"https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview","title":"Tool use with Claude — Pricing","publisher":"Anthropic","author":"Anthropic","date":"2026-07-26","quote":"The total number of input tokens sent to the model (including in the tools parameter)","summary":"The billing rule: tool names, descriptions and schemas are input tokens on every request, plus a per-model tool-use system prompt (286 tokens on Claude Opus 5 with tool_choice auto). Neutral vendor documentation; it is what makes definitions-in-context a per-turn tax.","claim_ids":["c2"]},{"id":"s3","type":"publisher_documentation","url":"https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool","title":"Tool search tool — context bloat and selection accuracy","publisher":"Anthropic","author":"Anthropic","date":"2026-07-26","quote":"A typical multiserver setup (GitHub, Slack, Sentry, Grafana, and Splunk) can consume ~55k tokens in definitions before Claude does any work. Tool search typically reduces this by over 85 percent, loading only the 3-5 tools Claude needs for a given request.","summary":"Vendor-published magnitude for the problem and the claimed reduction from deferral. Positive on tool search; the 85% figure is the vendor's, not independently reproduced here.","claim_ids":["c2","c4"]},{"id":"s4","type":"specification","url":"https://modelcontextprotocol.io/specification/2025-11-25/server/tools","title":"Model Context Protocol specification 2025-11-25 — Server Features: Tools","publisher":"Model Context Protocol","author":"MCP maintainers","date":"2025-11-25","quote":"When the list of available tools changes, servers that declared the listChanged capability SHOULD send a notification: { \"jsonrpc\": \"2.0\", \"method\": \"notifications/tools/list_changed\" }","summary":"The specification's answer to adding a capability at runtime, and it is a SHOULD on the server with no obligation on the client. Also defines tools/list and the tool object (name, title, description, inputSchema, outputSchema, annotations). Neutral; read alongside the client bug reports where the notification is ignored.","claim_ids":["c12","c13"]},{"id":"s5","type":"specification","url":"https://modelcontextprotocol.io/specification/2025-11-25/server/tools","title":"Model Context Protocol specification — User Interaction Model","publisher":"Model Context Protocol","author":"MCP maintainers","date":"2025-11-25","quote":"For trust & safety and security, there SHOULD always be a human in the loop with the ability to deny tool invocations.","summary":"The consent surface MCP clients are expected to provide — tool lists, per-call confirmation prompts, visual indicators. Cited here against this design, which has no client-side discovery or consent UI and must build one.","claim_ids":["c15"]},{"id":"s6","type":"repository","url":"https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/docs/specification/2025-11-25/server/tools.mdx","title":"modelcontextprotocol/modelcontextprotocol — docs/specification/2025-11-25/server/tools.mdx","publisher":"GitHub","author":"modelcontextprotocol","date":"2026-07-23","quote":"listChanged indicates whether the server will emit notifications when the list of available tools changes.","summary":"The specification text in its source repository, so the wording above can be diffed against revisions. Neutral; establishes that the dynamic-catalogue story is normative text, not a client feature.","claim_ids":["c12","c13"]},{"id":"s7","type":"repository","url":"https://github.com/scalekit-inc/mcp-vs-cli-benchmark","title":"scalekit-inc/mcp-vs-cli-benchmark — runnable harness","publisher":"GitHub","author":"scalekit-inc","date":"2026-07-26","quote":"Token overhead: MCP uses 1.3x to 80x more tokens than CLI, primarily due to tool schema overhead included in every request.","summary":"A rerunnable benchmark: `uv run bench run --runs 30`, LiteLLM model strings, 5 read-only GitHub tasks, results committed. Negative on definitions-in-context, but the same README reports both modalities completing 100% of tasks, which is a check on the harsher reliability claims made elsewhere.","claim_ids":["c16","c2"]},{"id":"s8","type":"independent_measurement","url":"https://www.scalekit.com/blog/mcp-vs-cli-use","title":"MCP vs CLI: Benchmarking AI Agent Cost & Reliability","publisher":"Scalekit","author":"Scalekit","date":"2026-07-26","quote":"The difference is almost entirely schema: 43 tool definitions injected into every conversation, of which the agent uses one or two.","summary":"Harness named: Claude Sonnet 4, GitHub's official Copilot MCP server at api.githubcopilot.com/mcp/, 5 deterministic read-only tasks against anthropics/anthropic-sdk-python, median tokens per run, differences significant at p < 0.05. 1,365 tokens via shell against 44,026 via MCP on the simplest task. Negative on definitions-in-context; the same post argues the data misleads outside personal developer tooling.","claim_ids":["c16","c2"]},{"id":"s9","type":"github","url":"https://github.com/anomalyco/opencode/issues/35376","author":"jijoyo","date":"2026-07-05","quote":"With 9 MCP servers connected (project-tools, supabase, agent-browser, chrome-devtools, playwright, memory, context7, sequential-thinking, fetch), this results in ~40,000-70,000 tokens of tool definitions loaded upfront","summary":"A user counting 250+ definitions across 9 servers, paid on every message, asking for lazy loading. Negative on the all-definitions-in-context model as a per-message tax.","title":"[Feature] Lazy-load MCP tool definitions to reduce token overhead","publisher":"GitHub","claim_ids":["c2"]},{"id":"s10","type":"github","url":"https://github.com/G-Core/gcore-mcp-server/issues/14","author":"lCrazyblindl","date":"2026-07-11","quote":"with `GCORE_TOOLS=*` it advertises **741 tools / ~488,013 tokens** (659/tool) — that exceeds a 200K context window on its own, so the full config can't actually be used with most models.","summary":"A tiktoken measurement against a live MCP server listing: 741 tools, ~488k tokens, one tool alone ~7,046 tokens of schema, with a proposed $defs hoisting saving of ~120,808 tokens. Strongly negative on definitions-in-context at catalogue scale.","title":"`GCORE_TOOLS=*` advertises ~488k tokens of tool definitions","publisher":"GitHub","claim_ids":["c1","c2"]},{"id":"s11","type":"github","url":"https://github.com/nimbalyst/nimbalyst/issues/914","author":"Eventlessdrop","date":"2026-07-18","quote":"A one-word prompt (\"Reply with exactly one word: pong\") produced a request with `prompt_tokens: 154,367`","summary":"150 MCP tool definitions plus a skills catalogue made a local 35B model unusable: rejected at 131k context, and 21m22s of prefill at 262k for a one-word reply. Negative, with a measured comparison to deferred schemas.","title":"Local models via opencode: slim or defer MCP tool and skills injection","publisher":"GitHub","claim_ids":["c2"]},{"id":"s12","type":"github","url":"https://github.com/ruvnet/ruflo/issues/2726","author":"shaal","date":"2026-07-19","quote":"`/compact` succeeded in producing a summary, but the **very next request still failed** — the non-compactable overhead (system prompt + ruflo tool schemas + plugin agent/skill listings) alone exceeded the limit.","summary":"35 plugins exposing ~200 tools produced a fixed per-request overhead larger than a 32k window, bricking the session until /clear. Negative, and the clearest case that definition overhead is not recoverable by compaction.","title":"Full plugin suite's MCP tool-schema overhead makes small-context backends unusable","publisher":"GitHub","claim_ids":["c2"]},{"id":"s13","type":"hn","url":"https://news.ycombinator.com/item?id=47400262","author":"gertjandewilde","date":"2026-03-16","quote":"We built a unified API with a large surface area and ran into a problem when building our MCP server: tool definitions alone burned 50,000+ tokens before the agent touched a single user message.","summary":"A vendor engineer who replaced a fat MCP server with a CLI contract of ~80 tokens plus --help discovery, and is candid that CLIs lose on streaming, delegated auth and distribution. Negative on schema size, honest about the trade.","title":"Comment on: Apideck CLI — lower context consumption than MCP","publisher":"Hacker News","claim_ids":["c2"]},{"id":"s14","type":"hn","url":"https://news.ycombinator.com/item?id=48330912","author":"0xbadcafebee","date":"2026-05-29","quote":"Say each tool is 150 tokens, that's 150 * 50, or 7500 tokens, dumped into the beginning of every session.","summary":"The most balanced account in the thread: gives the per-definition arithmetic, then argues shell one-liners are far more non-deterministic than schema'd calls so re-runs pollute context too, and concludes you do not have to pick a side. Mixed.","title":"Comment on: MCP is dead?","publisher":"Hacker News","claim_ids":["c1","c17"]},{"id":"s15","type":"hn","url":"https://news.ycombinator.com/item?id=48336021","author":"brookst","date":"2026-05-30","quote":"It's like saying APIs are dead because you can just use HTTP. They're not the same thing, though of course you can hand-roll the higher layer in the lower one. It's just more work, less standard, less valuable.","summary":"The sharpest rebuttal of the just-HTTP framing and the strongest objection to this page's design: a decorated index of features beats a bare endpoint list for a model. Positive on MCP; quoted and answered rather than dismissed.","title":"Comment on: MCP is dead?","publisher":"Hacker News","claim_ids":["c10","c9"]},{"id":"s16","type":"hn","url":"https://news.ycombinator.com/item?id=48331540","author":"827a","date":"2026-05-30","quote":"The idea that MCP tool definitions take up a certain number of tokens is laughable. That's an implementation detail of the agent harness.","summary":"The dissent against every measurement on this page: MCP is a specification, not a context-loading strategy, and harnesses have options. Contrarian and correct as far as it goes — which is why this design refuses to depend on a harness property.","title":"Comment on: MCP is dead?","publisher":"Hacker News","claim_ids":["c10"]},{"id":"s17","type":"hn","url":"https://news.ycombinator.com/item?id=48594160","author":"connor4312","date":"2026-06-19","quote":"Most mature harnesses do some kind of tool search and/or progressive disclosure.","summary":"Treats deferral as table stakes and locates token cost in the harness, not the protocol. Positive on the current ecosystem; the counterweight to the cost table above.","title":"Comment on: Zero-Touch OAuth for MCP","publisher":"Hacker News","claim_ids":["c10"]},{"id":"s18","type":"hn","url":"https://news.ycombinator.com/item?id=46039648","author":"morelandjs","date":"2025-11-24","quote":"Seems like we traded scalability for accuracy, then accuracy for scalability… but I guess maybe we've come out on top because whatever they are using for tool search is better than RAG?","summary":"The accuracy objection to any queryable catalogue, including this one: retrieval was abandoned for full definitions because full definitions were more accurate. Sceptical, and the reason the resolver's weakness is published here rather than hidden.","title":"Comment on: Claude Advanced Tool Use","publisher":"Hacker News","claim_ids":["c8"]},{"id":"s19","type":"github","url":"https://github.com/anthropics/claude-code/issues/57033","author":"brasscats","date":"2026-05-07","quote":"the `ToolSearch` deferred-tool discovery mechanism does NOT include them in its index. Any `ToolSearch` query that should match a claude.ai MCP tool returns zero results","summary":"On Claude Code 2.1.114, servers added at claude.ai/settings/connectors show Connected in /mcp but are invisible to deferred search, isolating the gap to the claude.ai-hosted transport. Negative on deferral as a dependable host feature.","title":"ToolSearch does not index tools from claude.ai-hosted MCP servers","publisher":"GitHub","claim_ids":["c11"]},{"id":"s20","type":"github","url":"https://github.com/NousResearch/hermes-agent/issues/57520","author":"JT-III","date":"2026-07-03","quote":"For any session running a model *other than* the configured default (e.g. `--model qwen3.6-27b --provider llamacpp`, or a model switched via `/model` in the TUI), the gate is scaled to the wrong window.","summary":"Traces the deferral decision to a function reading config rather than the session model, so a 98,304-token local model gets a threshold computed from a 256K cloud model. Negative, with the offending code quoted.","title":"tool_search auto-gate computes its threshold from model.default","publisher":"GitHub","claim_ids":["c11"]},{"id":"s21","type":"github","url":"https://github.com/anthropics/claude-code/issues/76372","author":"NAJEMWEHBE","date":"2026-07-10","quote":"With tool search active (`ENABLE_TOOL_SEARCH` unset), third-party MCP tools correctly defer to names-only. But three Desktop built-in servers load complete schemas upfront every session","summary":"Session-start context audited at ~51.6-52.3k tokens across 6 sessions using session JSONL usage records, isolating ~3.9k tokens of built-in schemas that escape deferral with no opt-out. Negative on first-party exemptions; positive that deferral works for third-party servers.","title":"Desktop: ~3.9k tokens of built-in MCP tool schemas load non-deferred","publisher":"GitHub","claim_ids":["c11"]},{"id":"s22","type":"github","url":"https://github.com/openai/codex/issues/24536","author":"yanxiyue","date":"2026-05-26","quote":"`codex exec` can silently finish with no assistant message when an explicitly configured MCP tool is deferred behind `tool_search`.","summary":"On codex-cli 0.133.0 with a 90-tool GitHub connector, a healthy registered server stops being directly exposed and the turn completes empty. Negative: the tool-count fix introduces a silent-failure mode.","title":"codex exec can silently complete empty when configured MCP tools are deferred","publisher":"GitHub","claim_ids":["c11"]},{"id":"s23","type":"github","url":"https://github.com/kirodotdev/Kiro/issues/6553","author":"neJoe-ch","date":"2026-03-20","quote":"When an MCP server dynamically adds or removes tools at runtime and sends this notification per the MCP spec, Kiro IDE does not re-query tools/list, so the new tools never appear until the server is manually reconnected.","summary":"A reproducible report with a minimal dynamic-registration server: the spec-compliant path for adding a capability without a restart does nothing in one client while working in two others. Negative on the no-redeploy story.","title":"Kiro IDE does not handle MCP notifications/tools/list_changed","publisher":"GitHub","claim_ids":["c13"]},{"id":"s24","type":"github","url":"https://github.com/microsoft/wassette/issues/308","author":"asw101","date":"2025-09-29","quote":"Internal terminal testing shows the CLI never refreshes its tool list, unlike GitHub Copilot in VS Code which updates immediately.","summary":"A Microsoft engineer files that runtime-loaded tools never surface in one client until restart, with a repro video, while a sibling client updates immediately. Negative: hot-adding a capability is specified but unevenly implemented.","title":"GitHub Copilot CLI does not dynamically load tools via tools/list_changed","publisher":"GitHub","claim_ids":["c13"]},{"id":"s25","type":"hn","url":"https://news.ycombinator.com/item?id=46878126","author":"DonHopkins","date":"2026-02-03","quote":"I call this \"speed of light\" as opposed to \"carrier pigeon\".","summary":"Argues the architectural difference is round trips, not tokens: anything outside the completion costs a round trip, while in-context capability can iterate and compose inside one generation. Negative on out-of-context tool layers — which is a direct cost of this design's discovery step.","title":"Comment on: Agent Skills","publisher":"Hacker News","claim_ids":["c14"]},{"id":"s26","type":"hn","url":"https://news.ycombinator.com/item?id=47209810","author":"sophiabits","date":"2026-03-01","quote":"The main problem with this approach at the moment is it busts your prompt cache, because LLMs expect all tool definitions to be defined at the beginning of the context window.","summary":"The second-order cost of loading definitions lazily per skill: cache invalidation, on the input tokens that dominate the bill. Negative on naive lazy loading — and the reason the vendor's own deferral is careful to leave the system-prompt prefix untouched.","title":"Comment on: When does MCP make sense vs CLI?","publisher":"Hacker News","claim_ids":["c4"]},{"id":"s27","type":"hn","url":"https://news.ycombinator.com/item?id=47381282","author":"menix","date":"2026-03-14","quote":"Tool results from programmatic calls are not added to Claude's context window, only the final code output is. They report up to 98.7% token savings in some workflows.","summary":"Argues the contract itself is the value: input and output schemas let a code-writing agent plan one precise program instead of print-and-inspect loops. Positive on schema-as-contract and explicitly against reducing the debate to schema bloat.","title":"Comment on: MCP is dead; long live MCP","publisher":"Hacker News","claim_ids":["c9"]},{"id":"s28","type":"runtime_receipt","url":"https://miscsubjects.com/api/dispatch?map=1","title":"First-party: the same catalogue exposed three ways","publisher":"miscsubjects.com","author":"Opus 5 (Claude Code)","date":"2026-07-25","quote":"856 definitions / 149,187 input tokens / $0.02852109  ·  9 + search / 14,109 / $0.00443075  ·  protocol only / 14,071 / $0.00456265","summary":"Gateway log rows and wire captures, 2026-07-25, claude-cli 2.1.165 against @cf/moonshotai/kimi-k2.7-code through Cloudflare AI Gateway. One build, one directory table, three exposures: per-row MCP tools, the same tools with ENABLE_TOOL_SEARCH=true, and no MCP server attached. Method reproducible by toggling the single environment variable and re-reading the gateway log.","claim_ids":["c1","c17","c18","c6","c7"]},{"id":"s29","type":"runtime_receipt","url":"https://miscsubjects.com/api/dispatch?registry=1","title":"First-party: four counts of the same table, taken live","publisher":"miscsubjects.com","author":"Opus 5 (Claude Code)","date":"2026-07-26","quote":"rows_total 892 · enabled 879 · mcp_projected 832  (2026-07-26T04:37:42Z);  ?map=1 total 879;  ?registry=1 count 877","summary":"Command published in the article: npx wrangler d1 execute loop-content-spine --remote --command \"SELECT COUNT(*) ... FROM directory;\" --json, run from the repository root, plus two GETs. Each number is a different predicate named in the code. Rerunning returns the count at that instant because other writers touch the table.","claim_ids":["c6"]},{"id":"s30","type":"runtime_receipt","url":"https://miscsubjects.com/api/mcp","title":"First-party: weighing the MCP projection with tools/list","publisher":"miscsubjects.com","author":"Opus 5 (Claude Code)","date":"2026-07-26","quote":"tools 831 · bytes of tools array 451,197 · mean 543 bytes per definition","summary":"POST /api/mcp with method tools/list and an Authorization: Bearer <MCP_TOKEN> header, response measured with len(json.dumps(tools)). This is the array a client transmits on every request even when every definition is deferred.","claim_ids":["c3","c5"]},{"id":"s31","type":"runtime_receipt","url":"https://miscsubjects.com/api/dispatch?confirm=inv_z77vqe1qi6","title":"First-party: adding, invoking and removing a capability with no deploy","publisher":"miscsubjects.com","author":"Opus 5 (Claude Code)","date":"2026-07-26","quote":"{\"ok\":true,\"key\":\"__DOC_PROBE\",\"updated_at\":\"2026-07-26T04:36:08.555Z\"} — HTTP 201 in 0.437946s; invocation inv_z77vqe1qi6 confirmed: true","summary":"A probe row was created by POST /api/directory, given a self-describing contract automatically, invoked successfully, and deleted. The public confirmation receipt outlives the row. Every command and response is printed in the article so the round trip can be repeated with a different key.","claim_ids":["c12","c19"]},{"id":"s32","type":"runtime_receipt","url":"https://miscsubjects.com/api/dispatch?ask=what%20time%20is%20it","title":"First-party: the resolver's twelve results for one query","publisher":"miscsubjects.com","author":"Opus 5 (Claude Code)","date":"2026-07-26","quote":"NOW, GITHUB_LIST_ISSUES, GITHUB_GET_ISSUE, GITHUB_ADD_ISSUE_COMMENT, GITHUB_CREATE_ISSUE, GITHUB_CLOSE_ISSUE, LOCAL_EDIT, LOCAL_WRITE, CLI_GIT, WRITER_AGENT, BLOOIO_LIST_CONTACT_IDENTITIES, STRIPE_INVOICE_ITEMS_LIST","summary":"A single GET, output pasted unedited. The pinned answer is right and the tail is noise because the two-character term 'is' matches inside 'issue'. Published as a defect of the design, not an illustration of it.","claim_ids":["c8"]},{"id":"s33","type":"repository","url":"https://github.com/massoumicyrus/miscsubjects-architecture/blob/main/docs/tooling/directory-row.md","title":"massoumicyrus/miscsubjects-architecture — docs/tooling/directory-row.md","publisher":"GitHub","author":"massoumicyrus","date":"2026-07-25","quote":"Canonical version, with every source card and claim: https://miscsubjects.com/a/tooling-as-data","summary":"The public mirror of this volume, MIT-licensed, five documents covering the row contract, the dispatch loop, the MCP projection and the tool-search comparison. Useful for diffing the design description against its revisions.","claim_ids":["c20"]}]}