{"slug":"tool-search-vs-catalogue-as-data","verification":{"valid":false,"broken_at":0,"reason":"prev mismatch"},"count":28,"sources":[{"id":"s1","type":"publisher_documentation","url":"https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool","title":"Tool search tool — Claude API documentation","quote":"Internally, the API excludes deferred tools from the system-prompt prefix. When Claude discovers a deferred tool through tool search, the API appends a `tool_reference` block inline in the conversation, then expands it into the full tool definition before passing it to Claude. The prefix is untouched, so prompt caching is preserved.","summary":"The mechanism, both variants (regex and BM25), the model compatibility list, the 10,000-deferred-tool ceiling, the five-result default, and the vendor's own 'when not to use it' list. Positive on the feature and the single most important source for the server-side behaviour that makes the structural claim on this page falsifiable.","publisher":"Anthropic","date":"2026-07-25","claim_ids":["c1","c10","c11","c13","c2","c6"]},{"id":"s2","type":"publisher_documentation","url":"https://code.claude.com/docs/en/mcp","title":"Claude Code — Scale with MCP tool search","quote":"Tool search keeps MCP context usage low by deferring tool definitions until Claude needs them. Only tool names and server instructions load at session start, so adding more MCP servers has minimal impact on your context window.","summary":"The client-side implementation: what ENABLE_TOOL_SEARCH true/auto/auto:N/false each do, the alwaysLoad escape hatch that exempts a server from deferral, and the statement that names load at session start — the sentence that makes the per-name scaling on this page a documented property rather than an inference. Positive on the feature, and the source of the exemption caveat.","publisher":"Anthropic","date":"2026-07-25","claim_ids":["c2","c3","c7"]},{"id":"s3","type":"publisher_documentation","url":"https://code.claude.com/docs/en/prompt-caching","title":"Claude Code — what invalidates the prompt cache","quote":"This happens when tool search is unavailable or disabled, such as on Google Cloud's Agent Platform or with a custom `ANTHROPIC_BASE_URL` gateway.","summary":"Documents that tool definitions fall back into the cached prefix behind a custom gateway, and that a server connecting or disconnecting mid-session then invalidates the whole conversation. This is the first of the two pages the Kimi measurement on this page contradicts. Negative for anyone running behind a proxy.","publisher":"Anthropic","date":"2026-07-25","claim_ids":["c3","c9"]},{"id":"s4","type":"publisher_documentation","url":"https://platform.kimi.ai/docs/guide/claude-code-kimi","title":"Moonshot — Use Kimi with Claude Code","quote":"The Kimi endpoint does not support this feature yet; it must be set to `false`, otherwise tool calls misbehave","summary":"Moonshot's environment-variable table instructs ENABLE_TOOL_SEARCH=\"false\" for Claude Code against the Kimi endpoint. The second of the two vendor pages contradicted by the measurement on this page, in which a Kimi model drove the deferred loop successfully through a translator. Negative.","publisher":"Moonshot AI","date":"2026-07-25","claim_ids":["c9"]},{"id":"s5","type":"specification","url":"https://modelcontextprotocol.io/specification/2025-06-18/server/tools","title":"Model Context Protocol specification — Tools","quote":"To discover available tools, clients send a `tools/list` request.","summary":"The protocol both designs are measured against: a server publishes name, description and inputSchema per tool, a client lists them, and a listChanged notification tells the client the set has moved. Establishes that the definition list is the unit of discovery in MCP, which is what deferral works around and what catalogue-as-data replaces.","publisher":"Model Context Protocol","date":"2025-06-18","claim_ids":["c4","c8"]},{"id":"s6","type":"specification","url":"https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-use-with-prompt-caching","title":"Tool use with prompt caching — defer_loading and cache preservation","quote":"Deferred tools are not included in the system-prompt prefix. When the model discovers a deferred tool through tool search, the definition is appended inline as a `tool_reference` block in the conversation history. The prefix is untouched, so prompt caching is preserved.","summary":"The API reference for the cache behaviour of deferred tools, and the direct counterweight to the field reports that lazy loading busts the cache. Both are true of different implementations, which is why this page publishes them side by side rather than picking one.","publisher":"Anthropic","date":"2026-07-25","claim_ids":["c10"]},{"id":"s7","type":"repository","url":"https://github.com/massoumicyrus/claude-code-cloudflare-gateway","title":"claude-code-cloudflare-gateway — the translator and the capture harness","quote":"tools/capture-gateway.mjs","summary":"MIT-licensed Worker that answers the Anthropic Messages API and forwards to Cloudflare AI Gateway, plus the capture server whose log fields (n_tools, tool_names, system_chars) produced the wire measurements on this page. What a reader gets: a runnable way to reproduce every byte count here against their own client.","publisher":"GitHub","date":"2026-07-25","claim_ids":["c12","c5"]},{"id":"s8","type":"github","url":"https://github.com/anthropics/claude-code/issues/76372","title":"Desktop built-in MCP tool schemas load non-deferred with no opt-out","quote":"With tool search active (`ENABLE_TOOL_SEARCH` unset), third-party MCP tools correctly defer to names-only. But three Desktop built-in servers load complete schemas upfront every session","summary":"Audits session-start context across six sessions and three projects from session JSONL message.usage and isolates ~3.9k tokens of first-party server schemas that escape deferral with no setting to remove them. Negative: deferral works for the servers you added and not for the ones you did not.","author":"NAJEMWEHBE","publisher":"GitHub (anthropics/claude-code)","date":"2026-07-10","claim_ids":["c7"]},{"id":"s9","type":"github","url":"https://github.com/anthropics/claude-code/issues/57033","title":"ToolSearch does not index tools from claude.ai-hosted MCP servers","quote":"the `ToolSearch` deferred-tool discovery mechanism does NOT include them in its index. Any `ToolSearch` query that should match a claude.ai MCP tool returns zero results","summary":"On Claude Code 2.1.114, servers added at claude.ai/settings/connectors show as Connected in /mcp yet are invisible to deferred discovery, while local .mcp.json and plugin servers index fine. Negative, and the clearest example of a capability that exists and cannot be reached.","author":"brasscats","publisher":"GitHub (anthropics/claude-code)","date":"2026-05-07","claim_ids":["c7"]},{"id":"s10","type":"github","url":"https://github.com/openai/codex/issues/24536","title":"codex exec can silently complete empty when configured MCP tools are deferred","quote":"`codex exec` can silently finish with no assistant message when an explicitly configured MCP tool is deferred behind `tool_search`.","summary":"On codex-cli 0.133.0 with a GitHub connector contributing 90 tools, the task-specific server stays healthy and registered but is only exposed as a deferred source; the model emits reasoning then completes and the harness accepts an empty turn. Negative, and evidence that deferral is not Anthropic-only.","author":"yanxiyue","publisher":"GitHub (openai/codex)","date":"2026-05-26","claim_ids":["c13","c7"]},{"id":"s11","type":"github","url":"https://github.com/NousResearch/hermes-agent/issues/57520","title":"tool_search auto-gate computes its threshold from the wrong model","quote":"For any session running a model *other than* the configured default (e.g. `--model qwen3.6-27b --provider llamacpp`, or a model switched via `/model` in the TUI), the gate is scaled to the wrong window.","summary":"Traces the deferral decision to a helper reading config instead of the session model, so a 98,304-token local model gets a threshold computed from a 256K cloud model — 25,600 instead of 9,830 — and receives the full schema payload inline. Negative, with the offending code quoted.","author":"JT-III","publisher":"GitHub (NousResearch/hermes-agent)","date":"2026-07-03","claim_ids":["c7"]},{"id":"s12","type":"github","url":"https://github.com/github/app/issues/1398","title":"Dynamic MCP tool loading to reduce context window consumption","quote":"Only include a tool's full schema in context when it's first needed. Use a lightweight tool registry (name + one-line description) for discovery","summary":"A feature request that independently describes the names-plus-descriptions registry this page measures, filed by someone watching hundreds of schemas consume tens of thousands of tokens before any conversation content. Positive on the pattern, negative on the status quo.","author":"amithegde","publisher":"GitHub (github/app)","date":"2026-06-25","claim_ids":["c4"]},{"id":"s13","type":"independent_measurement","url":"https://github.com/yonatangross/orchestkit/issues/885","title":"Context bloat: 16.9k tokens on MCP tool definitions loaded into the main agent","quote":"The main agent context burns **16.9k tokens (8.5% of 200k)** loading 68 MCP tool schemas at session start. Most are never called by the main agent — they're used by subagents.","summary":"A per-server token table (notebooklm-mcp 25 tools/~6.0k tokens, hq-knowledge 14/~3.3k, memory 9/~2.1k, tavily 5/~1.2k, context7 2/~0.5k) measured on a real project, putting MCP definitions at 16.9k of 54k fixed overhead. The most rigorous third-party measurement of the problem both designs address. Negative on the default.","author":"yonatangross","publisher":"GitHub","date":"2026-03-01","claim_ids":["c4"]},{"id":"s14","type":"hn","url":"https://news.ycombinator.com/item?id=47400262","title":"Apideck CLI — an AI-agent interface with much lower context consumption than MCP","quote":"We built a unified API with a large surface area and ran into a problem when building our MCP server: tool definitions alone burned 50,000+ tokens before the agent touched a single user message.","summary":"A vendor of a large unified API measured 50,000+ tokens of definitions before any user input, replaced the MCP server with a CLI at roughly 80 tokens of system prompt plus progressive discovery, and cites a 75-run comparison showing 4-32x token overhead for MCP versus CLI. Also honest about where the non-MCP route loses: streaming, delegated auth, distribution.","author":"gertjandewilde","publisher":"Hacker News","date":"2026-03-16","claim_ids":["c4","c8"]},{"id":"s15","type":"hn","url":"https://news.ycombinator.com/item?id=48332411","title":"Comment on: MCP is dead?","quote":"And it's not actually necessary for it to exist at the API level. It's a pattern. Making it API-side is just an optimization.","summary":"Notes OpenAI also supports defer_loading, then gives the four-step client-side recipe — one tool_search tool, names in context, match on call, append the matched definition — and observes Claude Code does it client side. Positive on deferral, and the clearest outside description of the client-side mechanism this page measures.","author":"red_hare","publisher":"Hacker News","date":"2026-05-30","claim_ids":["c2","c9"]},{"id":"s16","type":"hn","url":"https://news.ycombinator.com/item?id=48332962","title":"Comment on: MCP is dead?","quote":"Because Claude Code only loads the tools it needs now, so context bloat is pretty much solved for MCPs.","summary":"The strongest pro-tool-search position in the corpus: the source article retracted its context-bloat argument after deferred loading shipped, quoting an 85%+ context reduction. Positive, and the case this page has to beat rather than dismiss.","author":"didibus","publisher":"Hacker News","date":"2026-05-30","claim_ids":["c1","c6"]},{"id":"s17","type":"hn","url":"https://news.ycombinator.com/item?id=46039648","title":"Comment on: Claude Advanced Tool Use","quote":"Seems like we traded scalability for accuracy, then accuracy for scalability… but I guess maybe we've come out on top because whatever they are using for tool search is better than RAG?","summary":"The accuracy objection, and the sharpest thing said against the whole direction: the field abandoned retrieval for full definitions because retrieval was less accurate, and tool search walks that back. Negative, and it applies equally to a catalogue-as-data resolver.","author":"morelandjs","publisher":"Hacker News","date":"2025-11-24","claim_ids":["c6"]},{"id":"s18","type":"hn","url":"https://news.ycombinator.com/item?id=47719499","title":"Comment on: I still prefer MCP over skills","quote":"If the special tool search tool is available, then a client would not load the descriptions of the tools in advance, but only for the ones found via the search tool. But it's not widely supported yet.","summary":"Corrects the claim that MCP has no progressive disclosure by pointing at the tool search tool, while conceding client support is thin. Mixed, and the concession that makes 'requires host support' a real column on the comparison table rather than a theoretical one.","author":"medbrane","publisher":"Hacker News","date":"2026-04-10","claim_ids":["c3"]},{"id":"s19","type":"hn","url":"https://news.ycombinator.com/item?id=47719249","title":"Comment on: I still prefer MCP over skills","quote":"That first tool consumes only about 60 tokens. As long as the LLM doesn't need the tools, it takes almost no space.","summary":"Someone who hit the token problem with only a few tools and hand-rolled a gate: one bare-bones tool with a one-line description that unlocks the full set on call, plus a way back to the quiet state. Positive, and the outside figure this page's 1,440-byte ToolSearch measurement sits next to.","author":"BeetleB","publisher":"Hacker News","date":"2026-04-10","claim_ids":["c5"]},{"id":"s20","type":"hn","url":"https://news.ycombinator.com/item?id=47392361","title":"Comment on: Chrome DevTools MCP","quote":"And no, the tool search function recently introduced by Anthropic does not completely solve this problem.","summary":"A first-hand verdict that deferral is a partial mitigation: you still pay for servers you never invoke on a given turn. Negative, and the direct answer to the 'context bloat is pretty much solved' position quoted above it.","author":"cheema33","publisher":"Hacker News","date":"2026-03-15","claim_ids":["c7"]},{"id":"s21","type":"hn","url":"https://news.ycombinator.com/item?id=47209810","title":"Comment on: When does MCP make sense vs CLI?","quote":"The main problem with this approach at the moment is it busts your prompt cache, because LLMs expect all tool definitions to be defined at the beginning of the context window.","summary":"The prompt-cache objection to lazy loading, from an operator: input tokens are the main cost driver, so invalidating the prefix can cost more than the definitions saved. Negative, and in direct tension with Anthropic's documented server-side behaviour, which is why both are quoted on this page.","author":"sophiabits","publisher":"Hacker News","date":"2026-03-01","claim_ids":["c10"]},{"id":"s22","type":"hn","url":"https://news.ycombinator.com/item?id=48336021","title":"Comment on: MCP is dead?","quote":"It's like saying APIs are dead because you can just use HTTP. They're not the same thing, though of course you can hand-roll the higher layer in the lower one. It's just more work, less standard, less valuable.","summary":"The sharpest rebuttal of the just-use-HTTP framing, and the strongest argument against the catalogue-as-data side: a decorated index of features beats a bare endpoint list for a model, and hand-rolling the higher layer is less standard and less valuable. Positive on MCP.","author":"brookst","publisher":"Hacker News","date":"2026-05-30","claim_ids":["c8"]},{"id":"s23","type":"hn","url":"https://news.ycombinator.com/item?id=47381322","title":"Comment on: MCP is dead; long live MCP","quote":"MCP is effectively \"just another HTTP REST API\"; OAuth and everything.  The key parts of the protocol is the communication shape and sequence with the client, which most SDKs abstract for you","summary":"From a team that shipped one: the protocol reduces to a communication shape over HTTP that the SDKs hide, and deploying a server is no harder than a REST API. Mixed — deflationary about what MCP is while still recommending it, which is the honest middle of the argument this page sits inside.","author":"CharlieDigital","publisher":"Hacker News","date":"2026-03-14","claim_ids":["c8"]},{"id":"s24","type":"hn","url":"https://news.ycombinator.com/item?id=48331540","title":"Comment on: MCP is dead?","quote":"The idea that MCP tool definitions take up a certain number of tokens is laughable. That's an implementation detail of the agent harness.","summary":"The dissent from every measurement on this page: MCP is a specification, not a context-loading strategy, and harnesses have options beyond dumping definitions into the window. Contrarian and worth carrying, because it is the reason the comparison here is framed on what a specific client sends rather than on what the protocol requires.","author":"827a","publisher":"Hacker News","date":"2026-05-30","claim_ids":["c1"]},{"id":"s25","type":"runtime_receipt","url":"https://github.com/massoumicyrus/claude-code-cloudflare-gateway","title":"First-party wire capture: 9 definitions / 38,116 bytes against 858 / 522,746 bytes","quote":"9 38116\n858 522746","summary":"Taken for this page on 2026-07-25. claude-cli 2.1.165, one MCP server attached, same prompt and working directory, run once with ENABLE_TOOL_SEARCH=true and once false against a local server that logs the request body. Method and full script published in the article so the run is repeatable. Derived: 570.8 bytes per deferred definition.","publisher":"miscsubjects.com","date":"2026-07-25","claim_ids":["c11","c12","c5"]},{"id":"s26","type":"runtime_receipt","url":"https://github.com/massoumicyrus/claude-code-cloudflare-gateway","title":"First-party measurement: the ToolSearch definition is 1,440 bytes, and 833 deferred names are 30,551","quote":"ToolSearch 1440 bytes  ·  833 mcp__ names, 30551 bytes, mean 36.7 bytes per name","summary":"Same capture, 2026-07-25. The search tool's own definition is 3.8% of the nine-definition payload and does not grow with the catalogue; the residue that does grow is the name list, at a measured 36.7 bytes per capability per request. This is the measurement the structural claim on this page rests on.","publisher":"miscsubjects.com","date":"2026-07-25","claim_ids":["c2","c5"]},{"id":"s27","type":"runtime_receipt","url":"https://miscsubjects.com/receipt/inv_3wt7dcbp2c","title":"First-party timing: four round trips, 2.185 s, 47,409 bytes, with the public receipt","quote":"resolve 0.619s / contract 0.215s / invoke 1.047s / receipt 0.304s — total 2.185s, 47,409 B","summary":"Taken against production on 2026-07-25 with curl -w. The invocation that closes the loop is public and keyless at the linked receipt, with input and output hashes. Also records the resolver miss: ?ask=what time is it returned 12 candidates and recommended NOW rather than TIME_NOW.","publisher":"miscsubjects.com","date":"2026-07-25","claim_ids":["c12","c14"]},{"id":"s28","type":"runtime_receipt","url":"https://miscsubjects.com/api/articles/mcp-tool-search-cost","title":"Gateway log rows: 149,187 / 14,109 / 14,071 input tokens on the same catalogue","quote":"856 definitions 149,187 tokens $0.02852109 · 9 + ToolSearch 14,109 $0.00443075 · no MCP server 14,071 $0.00456265","summary":"Cloudflare AI Gateway log rows for @cf/moonshotai/kimi-k2.7-code on 2026-07-25, one build, one catalogue of 891 capabilities, three exposures. The pair that makes the headline finding: deferred search and no-server-at-all land 38 tokens apart. Cloudflare labels its cost column an estimation, so the argument here is built on token counts.","publisher":"miscsubjects.com","date":"2026-07-25","claim_ids":["c1","c11","c9"]}]}