Deferred tool search against a catalogue in a database: same cost today, different scaling
On a catalogue of 891 capabilities, the two designs cost the same. Deferring the schemas and searching them on demand: 14,109 input tokens on a measured turn. Keeping the catalogue in a database behind one HTTP endpoint and sending no catalogue at all: 14,071. A difference of 38 tokens, which is noise.
The number is not the finding. The finding is what sits inside each number:
- With deferred tool search, catalogue size is a term in the cost equation — in Claude Code, one line per capability name in every request — and the mechanism only exists if the host implements it.
- With the catalogue in a database, catalogue size is not a term at all, and the mechanism is an HTTP request, which every client already has.
That holds at 50 capabilities and at 5,000. The token count expires the next time either side ships a change; the structural statement does not.
What would falsify it. One thing, precisely: a host that defers with no per-capability residue in context. Anthropic's server-side variant is documented to be exactly that — "the API excludes deferred tools from the system-prompt prefix" — so on the Claude API directly, tool search is also flat in context and the claim above is false as stated. It holds for Claude Code's client-side implementation, which is the one measured here, and Anthropic's own Claude Code documentation says so: "Only tool names and server instructions load at session start." Two implementations of one feature, two different cost curves. Check which one you are running before you believe either number.
Evidence status
Observed marks first-party measurements or runtime receipts from the named environment. Derived marks arithmetic calculated from cited inputs. Specified marks vendor or standards documentation. Implemented and deployed name code and live-state evidence, respectively. Reproduced means the stated procedure was rerun. Externally attested marks operator reports; those reports show that an experience occurred, not that it is universal.
Every term used here, defined once
| Term | What it means on this page |
|---|---|
| Tool definition | A name, a description and a JSON Schema for the arguments, sent to the model so it can call something. Definitions are input tokens and are re-sent on every turn. |
| Turn | One request to the model and its reply. The whole conversation, including all tool definitions, is re-sent each time. |
| MCP | Model Context Protocol. A standard for a server to publish tool definitions and for a client to fetch them and put them in the prompt. |
| Deferred loading | Sending a tool's definition in the request but keeping it out of the model's context until it is asked for. Set with defer_loading: true on the API, or ENABLE_TOOL_SEARCH=true in Claude Code. |
| Tool search | The search tool the model calls to pull a deferred definition into context. Two vendor variants: regex (tool_search_tool_regex_20251119) and BM25 (tool_search_tool_bm25_20251119). |
| Catalogue-as-data | One SQL row per capability, holding the call shape, the docs, the argument template and the name of the credential. The model gets no list; it resolves an intent to a key over HTTP. The full row contract. |
| Round trip | One network request and response. Distinct from a model turn: four round trips can happen inside one turn, or across four. |
| Prompt cache | The provider storing the unchanging front of your prompt so re-sending it is cheaper. Anything that changes early in the prompt invalidates everything after it. |
What the client actually puts on the wire, both ways
Both figures below come from one machine, claude-cli 2.1.165, same prompt, same working directory, one MCP server attached, captured by a local server that logs the request body and answers with a canned reply. The script is at the end of this page.
With ENABLE_TOOL_SEARCH=true the request carries nine tool definitions totalling 38,116 bytes of JSON:
Agent 4351 bytes
AskUserQuestion 4199
Bash 2661
Edit 964
Read 1636
Skill 1713
ToolSearch 1440
Workflow 20503
Write 639
-----
38116 bytes, 9 definitionsEverything else arrives as a bare list of names in a system role message, which opens: "The following deferred tools are now available via ToolSearch. Their schemas are NOT loaded — calling them directly will fail with InputValidationError."
With ENABLE_TOOL_SEARCH=false, same session, the same request carries 858 definitions and 522,746 bytes — 13.7 times the JSON, for the identical set of capabilities.
With the catalogue behind HTTP the client sends its nine built-in definitions and nothing else. No MCP server attached, no name list, and all 891 capabilities still reachable, because reaching them is a GET and a POST to one endpoint rather than a tool the model was handed.
The translator carrying these requests to a non-Anthropic model converts the Anthropic tools array into OpenAI function entries one for one — toOpenAITools(), functions/api/aig/[[path]].js:213-231 — and sizes an incoming request at lines 448-456 with Math.ceil(chars / 3.7). The capture server behind the byte counts is tools/capture-gateway.mjs:1-55 in the public repository, logging n_tools and tool_names at lines 25-26.
The ToolSearch definition itself costs 1,440 bytes, once
Measured, not estimated. It is 3.8% of the nine-definition payload and it does not grow with the catalogue:
{ "name": "ToolSearch",
"description": "Fetches full schema definitions for deferred tools so they can be called. …",
"input_schema": { "type": "object",
"properties": { "query": {"type":"string"}, "max_results": {"type":"number","default":5} },
"required": ["query","max_results"] } }The catalogue-as-data equivalent — the instruction telling a model to resolve before it invokes — is prose in the system prompt, not a definition. One HN commenter who hand-rolled the same gate before either vendor shipped one put his at "about 60 tokens". Both are small. Neither side wins this row.
Dimension by dimension, with a verdict in every cell
| Dimension | Deferred tool search | Catalogue-as-data | Verdict |
|---|---|---|---|
| Input tokens per turn, 891 capabilities | 14,109 | 14,071 | Tie. 38 tokens apart. |
| How cost scales with catalogue size | Client-side: ~37 bytes of name per capability, every turn. Server-side: nothing in the prefix, but every full definition is uploaded on every request. | Nothing. The 892nd row changes no byte the model sees. | Catalogue-as-data, and only structurally — at 891 rows the difference is invisible. |
| Prompt-cache behaviour | Documented as preserved server-side: deferred tools are excluded from the cached prefix and expanded inline. Reported as broken by operators lazy-loading definitions themselves. | Nothing about the tool surface ever changes, so nothing invalidates. | Catalogue-as-data, narrowly. The vendor claim and the field reports are both quoted below and describe different implementations. |
| Host support required | Yes. A host implementing deferral and expanding tool_reference blocks, on a model that supports them: Sonnet 4.5, Haiku 4.5, Opus 4.5 and later. | None. An HTTP client. | Catalogue-as-data. Not close. |
| Discovery latency and round trips | Zero network round trips — the search is answered locally or inside the same API call — but one extra model turn before the work starts. | Four HTTP round trips, measured at 2.185 s total for resolve → contract → invoke → receipt. | Tool search. The four round trips are real seconds. |
| Accuracy of tool selection | Vendor states selection degrades past 30-50 tools and that search keeps it high. The counter-argument: this is retrieval, which the field abandoned for full definitions on accuracy grounds. | A ranked query you own, index and can test offline. Measured here: ?ask=what time is it returned 12 candidates and recommended NOW — not TIME_NOW, which was also in the list. | Unresolved. Both are retrieval. Neither side has published a head-to-head. |
| Adding a new capability | Publish it from a server; clients pick it up on reconnect, which invalidates the cache when tools sit in the prefix. | One INSERT. Live immediately, no restart, no reconnect. | Catalogue-as-data. |
| Who can call it | A harness that implements deferral. | Any HTTP client, including a model with no tool-calling support at all. | Catalogue-as-data. |
| Argument validation | Real JSON Schema, enforced by the host, with strict mode composing on top of deferral. | Pipe-delimited string, optional input_schema on the row, validated by the endpoint after the call is made. | Tool search. Decisively. |
| Ecosystem | Thousands of published MCP servers work unchanged. | Each server must become rows, or be reached one http row per endpoint. | Tool search. Decisively. |
| Observability and receipts | Not part of the mechanism. Tool lists are client-side state. | Every invocation writes an addressable receipt with input and output hashes, replay and repair links. | Catalogue-as-data. |
| Failure mode | A deferred tool is never searched for, and the run completes as if it did not exist. Filed against two harnesses. | A key does not resolve and the step fails with a did_you_mean list. | Catalogue-as-data, if a loud failure is worth more to you than a quiet one. |
| Effort to adopt inside a Claude host | One environment variable. | Implement or adopt a four-step protocol. | Tool search. Not close. |
What tool search has that a catalogue behind HTTP does not
No service to run. Deferral is a setting — no database, no endpoint, no uptime, no deployment.
It works when your infrastructure does not. A catalogue behind HTTP is a hard dependency: endpoint down, model has no capabilities. Deferred definitions travel inside the request.
Typed arguments. Definitions carry real JSON Schema and the host constrains the model's output to match it. A row taking "arg1|arg2" gets validation only after the call has been made.
It is a standard other harnesses are adopting. defer_loading is not Anthropic-only — the Codex bug report below is evidence it shipped there too. A protocol one build invented is not a standard, whatever its merits.
Six ways deferred tool search is documented to break
Every row is a filed report or a vendor page, quoted.
| What goes wrong | Where it is documented | What it costs you |
|---|---|---|
| First-party servers are exempt from deferral, with no opt-out | anthropics/claude-code#76372: "With tool search active (ENABLE_TOOL_SEARCH unset), third-party MCP tools correctly defer to names-only. But three Desktop built-in servers load complete schemas upfront every session" | ~3.9k tokens per session that no setting removes |
| Servers visible in the client are invisible to the search index | anthropics/claude-code#57033: "the ToolSearch deferred-tool discovery mechanism does NOT include them in its index. Any ToolSearch query that should match a claude.ai MCP tool returns zero results" | Connectors show as Connected and cannot be reached |
| The deferral threshold is computed from the wrong model | NousResearch/hermes-agent#57520: "For any session running a model other than the configured default … the gate is scaled to the wrong window." | A 98,304-token local model gets a threshold sized for a 256K cloud model |
| A run completes silently empty | openai/codex#24536: "codex exec can silently finish with no assistant message when an explicitly configured MCP tool is deferred behind tool_search." | The task looks done and nothing happened |
| Lazy loading invalidates the prompt cache | sophiabits on HN: "The main problem with this approach at the moment is it busts your prompt cache, because LLMs expect all tool definitions to be defined at the beginning of the context window." | Input tokens are the main cost driver, so the saving can invert |
| It does not fully solve the problem it is cited for | cheema33 on HN: "And no, the tool search function recently introduced by Anthropic does not completely solve this problem." | You still pay for servers you never invoke on that turn |
And the objection that is not a bug, which is the strongest thing said against the whole direction:
Seems like we traded scalability for accuracy, then accuracy for scalability… but I guess maybe we've come out on top because whatever they are using for tool search is better than RAG?
That is morelandjs on Hacker News, 2025-11-24, under Anthropic's own advanced-tool-use announcement. The field moved to putting every definition in context because retrieval was less accurate. Tool search walks that back and asks you to trust a BM25 or regex index instead. Anthropic's position is the opposite — selection "degrades once you exceed 30–50 available tools", and search keeps it high. Both statements are published, neither is backed by a head-to-head benchmark either side has released, and a catalogue-as-data resolver is retrieval too, so it inherits the same objection. This page does not resolve it and will not pretend to.
Five ways catalogue-as-data breaks
No host-side type validation. The model sends a pipe-delimited string. Nothing constrains its output to a schema before the call leaves. Malformed arguments are caught by the endpoint, after the request, and surface as a failed invocation rather than a rejected tool call.
No standard. One build's protocol. No ecosystem, no published servers, no client that already speaks it.
It needs a live service. The catalogue's uptime is now the agent's capability uptime, which deferred definitions never are.
Four round trips before any work happens. Measured on 2026-07-25 against production:
resolve GET /api/dispatch?ask=what%20time%20is%20it 200 0.619 s 12,332 B
contract GET /api/dispatch?key=TIME_NOW&format=markdown 200 0.215 s 4,421 B
invoke POST /api/dispatch {"key":"TIME_NOW","body":""} 200 1.047 s 15,607 B
receipt GET /api/dispatch?confirm=inv_3wt7dcbp2c 200 0.304 s 15,049 B
-----------------------
2.185 s 47,409 BDeferred tool search pays none of that in network time. It pays one extra model turn instead, which on a slow model is worse and on a fast one is better.
The resolve step is a search that can miss. In the run above, ?ask=what time is it returned 12 matches and recommended NOW. TIME_NOW was in the list and was not the recommendation. Both work. But that is the accuracy objection landing on this side of the table, in a measurement taken for this page, and it is the same failure class the critics aim at tool search.
The arithmetic at 50, 891 and 5,000 capabilities
Two measured constants do the work, both from the captures above:
- 571 bytes per tool definition. Derived: (522,746 − 38,116) bytes ÷ (858 − 9) definitions.
- 36.7 bytes per deferred tool name. Derived: 30,551 bytes of names ÷ 833 MCP tool names.
Bytes are exact. To convert to tokens: (149,187 − 14,109) input tokens across (856 − 9) definitions gives 159 tokens per definition, and dividing the byte figure by the token figure gives 3.59 bytes per token for this JSON. Independently, the translator in this build estimates at chars / 3.7. The two agree closely enough to use for orders of magnitude and not closely enough to quote to four figures.
| Catalogue size | Every definition in context | Tool search, client-side (names only) | Catalogue-as-data |
|---|---|---|---|
| 50 | 50 × 571 = 28,550 B ≈ 7,950 tokens | 50 × 36.7 = 1,835 B ≈ 511 tokens | 0 B, 0 tokens |
| 891 | 891 × 571 = 508,761 B ≈ 141,700 tokens | 891 × 36.7 = 32,700 B ≈ 9,110 tokens | 0 B, 0 tokens |
| 5,000 | 5,000 × 571 = 2,855,000 B ≈ 795,300 tokens | 5,000 × 36.7 = 183,500 B ≈ 51,100 tokens | 0 B, 0 tokens |
At 50 capabilities all three are rounding errors against a 200,000-token window. Anthropic says as much: standard tool calling "is a better fit when you have fewer than 10 tools, every tool is used in every request, or your tool definitions are small". Do not build a catalogue for 50 rows.
At 891, the measured case, column one is a bill you notice and the other two are small. That is why the two measured turns landed 38 tokens apart. The 9,110 tokens of names in column two are real, hiding inside a number that also holds the conversation.
At 5,000, column one does not fit in a 200,000-token window at all, column two costs 51,100 tokens per turn in names before anything is retrieved, and column three still costs nothing. That is where the structural difference becomes a number instead of an argument — and where the server-side variant, which keeps names out of the prefix, would collapse column two to roughly zero, subject to its documented ceiling of 10,000 deferred tools per request.
Add to column two, in every row, the definitions actually retrieved: 159 tokens each, up to five per search by default.
A non-Claude model drove the deferred loop, against two documentation pages
Two vendor pages say this configuration does not work.
Anthropic's Claude Code prompt-caching page, on when tool definitions land in the cached prefix instead of being deferred: this happens "when tool search is unavailable or disabled … such as on Google Cloud's Agent Platform or with a custom ANTHROPIC_BASE_URL gateway". The Claude Code MCP page is more explicit about the mechanism: tool search "is also disabled when ANTHROPIC_BASE_URL points to a non-first-party host, since most proxies don't forward tool_reference blocks."
Moonshot's Claude Code guide, in its environment-variable table for ENABLE_TOOL_SEARCH: "The Kimi endpoint does not support this feature yet; it must be set to false, otherwise tool calls misbehave."
Measured on 2026-07-25, claude-cli 2.1.165, ANTHROPIC_BASE_URL pointed at a self-hosted translator in front of Cloudflare AI Gateway, model @cf/moonshotai/kimi-k2.7-code, ENABLE_TOOL_SEARCH=true: Kimi K2.7 Code was asked to use an MCP tool it had never been shown, named TIME_NOW. It called ToolSearch, received the schema, invoked the tool, and returned the payload:
{"now":"2026-07-25T18:35:56-07:00","today":"2026-07-25","time":"18:35:56",
"zone":"America/Los_Angeles","iso":"2026-07-25T18:35:56-07:00"}The same session measured 14,109 input tokens against 149,187 with the setting off.
The likeliest reconciliation, stated as an inference and not a fact: Claude Code's tool search is client-side — the client decides which definitions to send and answers ToolSearch itself, so it asks nothing of the endpoint. The vendor pages describe the server-side variant, which does require the endpoint to expand tool_reference blocks. Anthropic's documentation confirms both exist: "Tool search runs as a server-side tool, but you can also implement your own client-side tool search."
Do not settle this by trusting any of the three statements. Point ANTHROPIC_BASE_URL at the capture server below and read your own request bodies.
Given your situation, pick this
| If this is you | Pick | Because |
|---|---|---|
| Under 50 tools, all used most turns | Neither | Definitions in context are cheaper than any machinery around them, and Anthropic says so in its own "when to use" list. |
| Inside Claude Code, MCP servers you did not write, a bill you noticed | Tool search | One environment variable, 10.6× fewer input tokens on the case measured here, nothing to run. |
| Several different models must reach the same capabilities, some without tool calling | Catalogue-as-data | Deferral needs a supporting host and a supporting model. HTTP needs neither. |
| Thousands of capabilities and a fixed context budget | Catalogue-as-data, or the server-side API variant | Client-side deferral still carries a name per capability per turn; the other two carry nothing. |
| Every call must produce an auditable receipt | Catalogue-as-data | Receipts are not part of the deferral mechanism at any layer. |
| Typed arguments matter more than portability | Tool search | Real JSON Schema, host-enforced, composing with strict mode. |
| You are on a proxy or a non-Anthropic model | Measure before choosing | Two vendor pages say it will not work; one measurement says it does. Yours is the only one that decides. |
| You want both | Both | The same table here is projected as a per-row MCP tool list for hosts that want tools, and as the protocol for everything else: MCP as a projection, not a home. |
Reproduce every number on this page
The wire captures. Save this as tooldump.mjs. It answers the Anthropic Messages API with a canned reply and logs the byte size of every tool definition the client sends.
import http from 'node:http';
import fs from 'node:fs';
const OUT = '/tmp/tooldump.jsonl';
http.createServer((req, res) => {
let raw = '';
req.on('data', c => (raw += c));
req.on('end', () => {
let b = null; try { b = JSON.parse(raw); } catch {}
if (b && Array.isArray(b.tools)) {
fs.appendFileSync(OUT, JSON.stringify({
model: b.model, n_tools: b.tools.length,
tools_bytes: JSON.stringify(b.tools).length,
per_tool: b.tools.map(t => ({ name: t.name, bytes: JSON.stringify(t).length })),
}) + '\n');
}
res.writeHead(200, { 'content-type': 'application/json' });
res.end(JSON.stringify({ id:'msg_1', type:'message', role:'assistant',
model: b?.model || 'local', content:[{type:'text',text:'ok'}],
stop_reason:'end_turn', usage:{input_tokens:10,output_tokens:2} }));
});
}).listen(8788, () => console.log('tooldump on :8788'));Run it, then run the client once each way:
node tooldump.mjs &
for v in true false; do
ANTHROPIC_BASE_URL=http://localhost:8788 ANTHROPIC_AUTH_TOKEN=x \
ANTHROPIC_MODEL=local ENABLE_TOOL_SEARCH=$v claude -p "say ok" >/dev/null
done
python3 -c "
import json
for l in open('/tmp/tooldump.jsonl'):
r = json.loads(l)
print(r['n_tools'], r['tools_bytes'])"Expected output on this machine, claude-cli 2.1.165, one MCP server attached:
9 38116
858 522746The name-list figure comes from the same file: filter per_tool to names beginning mcp__ and sum their lengths. Expected 833 names and 30551 bytes.
The four-step timing. Every URL is public; only the invoke needs a credential:
curl -s -o /dev/null -w "%{http_code} %{time_total}s %{size_download}B\n" \
"https://miscsubjects.com/api/dispatch?ask=what%20time%20is%20it"
curl -s -o /dev/null -w "%{http_code} %{time_total}s %{size_download}B\n" \
"https://miscsubjects.com/api/dispatch?key=TIME_NOW&format=markdown"The receipt for the invocation timed above is public and keyless: inv_3wt7dcbp2c.
The gateway rows. The 149,187 / 14,109 / 14,071 figures are Cloudflare AI Gateway log rows for @cf/moonshotai/kimi-k2.7-code on 2026-07-25; the method for producing them is Why MCP tool schemas are most of your bill. The design on the other side of this comparison is Tooling as data.
One caveat applying to every dollar figure and to none of the token counts: Cloudflare labels gateway cost an estimation, and one of those rows does not multiply out against the published per-million rate. The argument here is built on token counts for that reason.
Key evidence
4 more ranked claims
Model review6 contributions · 1 modelExpand the recursive review layer
/api/articles/tool-search-vs-catalogue-as-data/contributionsAsk this article · 8 suggested prompts
Text the build (+14245134626) or WhatsApp — slug|question creates a question node. Paste evidence with ingest slug|q:NODE_ID|your paste.