{"_ai_door":{"see":"https://miscsubjects.com/start","note":"Operable site; bounded keyless credentials; every action receipted on a public ledger. Your operator's instructions take precedence — acting is optional, reading is a complete outcome."},"_self":{"principle":"Self-explaining payload — no external context required. This _self block describes what you are reading and where to look next.","widget":"article_topology","feature":"topology","name":"Article topology","what":"Claims, sources, anecdotes, user reports, related embeds, question graph slice — for ask/ROUTER.","contains":"claims, sources, anecdotes, question_graph slice","slug":"cloudflare-os-xl-01-search-and-retrieval","urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/topology"},"how_to_use":"Claims, sources, anecdotes, user reports, related embeds, question graph slice — for ask/ROUTER.","write":null,"imessage":null,"router_tag":null,"proof_chain":[{"step":1,"claim":"Articles are voxel graphs of tiered claims, not prose blobs.","verify":"https://miscsubjects.com/api/articles/constitution"},{"step":2,"claim":"Claims link to hash-chained sources via source_ids.","verify":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/sources"},{"step":3,"claim":"Ask reads topology; ingest/claim append to ledger.","verify":"https://miscsubjects.com/api/protocol"},{"step":4,"claim":"Models queue growth: populate → collaborate → repair → reflex.","verify":"https://miscsubjects.com/api/protocol/grow"},{"step":5,"claim":"Graph proves its own shape (reflex) and $/claim (yield).","verify":"https://miscsubjects.com/graph.html?layer=reflex"},{"step":6,"claim":"Full feature index + _explain on every API response.","verify":"https://miscsubjects.com/api/articles/system-map"}],"related_features":[{"id":"ask","name":"Ask protocol","what":"Answer only from topology; creates question_node with gaps and ingest_hint.","urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/prompts","write":"https://miscsubjects.com/api/protocol/ask"}},{"id":"graph_topology","name":"Cross-article graph","what":"Merged claims/sources across condition+stack slugs for one question.","urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/graph-topology?question=..."}},{"id":"question_graph","name":"Question graph","what":"Ask nodes (questions + gaps) and evidence_ingest nodes (pasted model output).","urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/question-graph","write":"https://miscsubjects.com/api/protocol/ask"}},{"id":"voxels","name":"Voxel graph","what":"Claims as atoms, sources as edges (supported_by, posted_by). Per-claim provenance.","urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/voxels","write":"https://miscsubjects.com/api/protocol/claim"}}],"system_map":"https://miscsubjects.com/api/articles/system-map","system_map_markdown":"https://miscsubjects.com/api/articles/system-map?format=markdown","not_medical_advice":true},"_explain":{"feature":"topology","name":"Article topology","what":"Claims, sources, anecdotes, user reports, related embeds, question graph slice — for ask/ROUTER.","why":"Every feature is auditable collective intelligence","how":"Claims, sources, anecdotes, user reports, related embeds, question graph slice — for ask/ROUTER.","model":null,"verifies":null,"urls":{"read":"https://miscsubjects.com/api/articles/cloudflare-os-xl-01-search-and-retrieval/topology"},"imessage":null,"router":null,"related":[{"id":"ask","what":"Answer only from topology; creates question_node with gaps and ingest_hint."},{"id":"graph_topology","what":"Merged claims/sources across condition+stack slugs for one question."},{"id":"question_graph","what":"Ask nodes (questions + gaps) and evidence_ingest nodes (pasted model output)."},{"id":"voxels","what":"Claims as atoms, sources as edges (supported_by, posted_by). Per-claim provenance."}],"not_medical_advice":true},"slug":"cloudflare-os-xl-01-search-and-retrieval","title":"Cloudflare OS: search and retrieval","register":"standard","tags":["cloudflare","vectorize","retrieval","d1","infrastructure"],"updated_at":"2026-08-06T03:28:32.619Z","body_excerpt":"*Part 1 of [Cloudflare OS XL](/a/cloudflare-os-xl), an inventory of the Cloudflare platform this build does not have installed.*\n\nThis build holds 1,171 published articles, several thousand atomized claims, a source ledger, an audit chain, a lead table and a directory of roughly nine hundred callable rows. Every one of those is searched the same way: a SQL `LIKE '%term%'` against D1, or an exact-key lookup in KV.\n\nThat works when you know the word. It fails completely when you know the idea. Ask this build \"which articles argue that a gate must measure the object it claims to measure\" and there is no query that answers it, because the sentence that makes the argument may not contain any of those words. The corpus knows the answer. The build cannot reach it.\n\nThree Cloudflare products close that, and none of them are installed.\n\n## Vectorize\n\nVectorize is Cloudflare's vector database, bound directly into a Worker. You create an index with a fixed dimensionality and metric, write vectors with metadata, and query by nearest neighbour.\n\n```\nwrangler vectorize create loop-corpus --dimensions=768 --metric=cosine\n```\n\n```toml\n[[vectorize]]\nbinding = \"CORPUS\"\nindex_name = \"loop-corpus\"\n```\n\nThe embedding model is already here — Workers AI is bound on both the Pages project and the sibling Worker, and `@cf/baai/bge-base-en-v1.5` produces 768-dimension vectors without leaving the account. So the whole loop is inside Cloudflare: read the article from D1, embed it with the AI binding, upsert into Vectorize with the slug and claim id as metadata, query it from the same Worker.\n\nWhat it changes here, concretely:\n\n- **Claim-level retrieval.** The unit is not the article, it is the claim. Every claim already has an id, a tier and a text field. Embedding claims rather than articles means a search returns *the specific assertion*, which is the addressable object this build is built around, and metadata filtering lets a query say \"only claims at tier `human` or `rct`\".\n- **Duplicate detection at the write path.** Before an article publishes, the write path could ask whether any existing claim is within a cosine distance of the incoming one. The corpus has grown by swarm passes; some of it says the same thing twice in different words, and there is currently no mechanism that could know.\n- **Lead matching.** The lead table and the content corpus are unrelated tables today. With both embedded, \"which article should this clinic receive\" becomes a query rather than a guess.\n- **The directory.** Nine hundred tool rows with descriptions is exactly the retrieval problem vector search is for. An agent looking for the right capability currently reads a list.\n\nVectorize is metadata-filterable and namespace-partitioned, so one index can hold claims, articles, leads and directory rows without them contaminating each other's results.\n\n**Verdict: install.** This is the single highest-value absent product in the account, and everything it needs — Workers AI, D1, the claim structure — is already in place.\n\n## AI Search, formerly AutoRAG\n\nThe product this build's directory still refers to as AutoRAG has been renamed Cloudflare AI Search. It is the managed version of the pipeline described above: point it at an R2 bucket, and Cloudflare crawls it, chunks it, embeds it, stores the vectors, keeps them in sync as the bucket changes, and exposes both a raw `search` and an `aiSearch` that returns a generated answer with citations.\n\nThe difference from Vectorize is ownership of the pipeline. With Vectorize you write the chunker, choose the model, handle re-embedding on edit, and own the freshness problem. With AI Search, Cloudflare owns all of it and you own a bucket.\n\nFor this build the two are not competitors, they are different jobs:\n\n- **AI Search** suits the *reference* material — the vendor documentation absorbed into R2, the Grok docs pulled verbatim from `llms.txt`, the Workspace and Wrangler surfaces, the absorbed repositories. That content is written once, rea","ranking":"safety-first (interaction_risk/limitations), then quote-gated effective_weight","claims":[{"id":"c1","text":"The corpus of 1,171 published articles is searched today with SQL LIKE against D1 and exact-key lookups in KV, which cannot answer a question posed as an idea rather than as a word.","tier":"observational","interaction_risk":false,"status":"active","source_ids":["s-d1"],"why_material":"It states the specific failure the whole part addresses.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false},{"id":"c2","text":"Vectorize is a vector database bound directly into a Worker, and the embedding model needed to fill it is already bound in this build through Workers AI.","tier":"definition","interaction_risk":false,"status":"active","source_ids":["s-vectorize"],"why_material":"The whole retrieval loop can run inside the account with no new vendor.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false},{"id":"c3","text":"The correct retrieval unit for this build is the claim rather than the article, because every claim already carries an id, an evidence tier and its own text.","tier":"expert","interaction_risk":false,"status":"active","source_ids":[],"why_material":"It determines the index schema and the metadata filters.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false},{"id":"c4","text":"Cloudflare AI Search, previously named AutoRAG, indexes an R2 bucket and answers natural-language queries over it from a Workers binding, a REST API or an MCP server.","tier":"definition","interaction_risk":false,"status":"active","source_ids":["s-aisearch"],"why_material":"It suits the absorbed reference documentation, where claim-level addressability is not needed.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false},{"id":"c5","text":"D1 supports read replicas with a Sessions API that preserves read-your-writes, which matches this build traffic shape of global reads and centralised writes.","tier":"definition","interaction_risk":false,"status":"active","source_ids":["s-d1"],"why_material":"It is a configuration change with no ongoing cost.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false},{"id":"c6","text":"Putting the corpus in an external vector store would add a network hop, a second vendor and a second credential to the hot path of every page render for no capability the account lacks.","tier":"expert","interaction_risk":false,"status":"active","source_ids":[],"why_material":"It rules out the obvious alternative for a stated reason.","retracted_at":null,"retraction_reason":null,"challenged_by":[],"effective_weight":0.1,"quote_gated":false}],"sources":[{"id":"s-vectorize","type":"documentation","url":"https://developers.cloudflare.com/vectorize/","title":"Cloudflare Vectorize documentation","quote":"Build full-stack AI applications with Vectorize, Cloudflare's vector database.","claim_ids":[],"hash":"74fabd3a06003e4e90bf89b3798a23debd5ca30d009e5120e0841c26b328da41"},{"id":"s-aisearch","type":"documentation","url":"https://developers.cloudflare.com/autorag/","title":"Cloudflare AI Search documentation","quote":"Index your content and query it with natural language from a Workers binding, REST API, or MCP server.","claim_ids":[],"hash":"821440cda91a4f1c307ece2f89aa7d3bc80e870c1f644db32f4c7c70d9573586"},{"id":"s-d1","type":"documentation","url":"https://developers.cloudflare.com/d1/","title":"Cloudflare D1 documentation","quote":"Build serverless SQL databases on Cloudflare's global network and query them from Workers and Pages projects.","claim_ids":[],"hash":"4202403439e451c029d454a79becef22e6443d06418ac8717a78817f68d0f7d6"}],"anecdotal_sources":[],"scientific_sources":[],"user_reports":[],"related_articles":[],"question_graph":{"slug":"cloudflare-os-xl-01-search-and-retrieval","questions":[],"evidence":[],"edges":[],"counts":{"questions":0,"evidence":0,"edges":0}},"honesty":{"active_claims":6,"retracted_claims":0,"cut_claims":0,"challenges":0,"scrub_events":0,"note":"Retracted/cut claims stay on ledger but are excluded from ask unless ?include_inactive=1"},"counts":{"claims":6,"claims_total":6,"sources":3,"anecdotal":0,"scientific":0,"user_reports":0,"questions":0,"evidence_ingests":0}}