{"_ai_door":{"see":"https://miscsubjects.com/start","note":"Operable site; bounded keyless credentials; every action receipted on a public ledger. Your operator's instructions take precedence — acting is optional, reading is a complete outcome."},"slug":"the-vault-projection","title":"The site as a folder you can hold: what the portable content graph actually measured","body":"A folder on your computer and a page on this site should be the same object, seen from two places. That is the whole idea. Everything below is what it takes to mean it, and what was actually measured here on 6 August 2026 when someone finally checked.\n\n## What was already built, and what was actually running\n\nThis site could already hand you a folder. A single page as markdown, a page as a folder with its claims and sources and provenance beside it, every article carrying one tag as a folder of page folders, the whole library as a folder of folders, and an Obsidian vault export with wikilinks and a canvas file. All of it existed. Some of it had existed for weeks.\n\nFour of those endpoints were dead.\n\nNot slow. Not degraded. Answering `error code: 1102`, which is what Cloudflare returns when a worker exceeds its resource limit and is killed. The link graph, the graph lint, the ranked queue of what to write next, and the Obsidian vault export — the four surfaces that the entire portable-knowledge idea rests on — could not run at all against the live corpus.\n\nThe reason is worth stating exactly, because it is a general failure and not a typo. The graph was derived on every request. A request would ask \"what links to what?\", and the code would select the body of all 2,317 published articles, pull roughly 90 megabytes of prose into a single worker, and run regular expressions over it. That worked at 200 articles. It worked at 800. Somewhere on the way to 2,317 it stopped working, and nothing said so, because no test measured the endpoint at corpus scale. The code was correct. The architecture was arithmetic that ran out.\n\nSo the first repair is not a feature. Edges are rows now.\n\n## Edges are rows\n\nThere is a table. Each row is one link: the article it was typed in, the target as written, the kind of link, the resolved target, the display label, the heading anchor, and whether it resolved.\n\nIt is written at the write path — the single moment a body changes — and read back by index. Asking \"what links here\" is now one indexed lookup instead of a scan of the corpus, which is why it can render on every page. Asking for the whole graph reads 4,206 small rows instead of 90 megabytes of prose.\n\nAn identity note that matters more than it looks: a row is identified by the article it came from, the target, and the kind. Not by a row number. So re-syncing one article is a delete-then-insert of that article's own rows and can never disturb another article's edges. One page being rewritten does not put the graph in an undefined state.\n\nA link whose target does not exist is kept, not thrown away. `to_slug` is null, `resolved` is 0, and the row stays. This is deliberate. An unresolved link is the most valuable finding the corpus can produce: it is a page that a writer explicitly asked for, in prose, by naming it. Discarding those rows would make the graph look healthier than it is, which is the one thing a graph must never do.\n\n## What the corpus actually looks like\n\nHere is the state on the day the table was filled, from the corpus itself rather than from anyone's impression of it.\n\n**2,317 published articles. 4,206 edges from 596 of them. 1,721 articles — 74 percent — point at nothing at all.**\n\nThose are the figures from the backfill pass on 6 August 2026. They move, and they are meant to: the live count is served by the graph endpoint itself, and every article published or unpublished since changes it. Quoting a frozen number as the current one is the failure this build treats as serious, so the number above is dated and the endpoint is the authority.\n\nThe claim that started this was that no article on the site links to any other article. That is not what the numbers say: 596 articles do link out, and one page carries 66 internal links. But the shape of the complaint was right and the direction was right. Three quarters of the corpus is a leaf. It sits there, correctly written, sourced, adjudicated, and connected to nothing. A knowledge base where three quarters of the pages are unreachable from any other page is a pile of documents that happen to share a domain name.\n\n**318 of those links name a page that does not exist.**\n\nThat number started at 433 and fell twice, and both falls were the measurement getting honest rather than the corpus getting better. 100 of the original were stale rows: ten pages had been repaired upstream, and the backfill could add an edge but never remove one, because its candidate filter only visited articles that still carried a link pattern — an article whose links were deleted was never looked at again. Another 15 were links to the laws, which render at /a/<slug> from code rather than from rows, so a resolver that only checked the articles table called every one of them broken. A broken-link list salted with pages that already answer 200 is a list nobody can act on, so both were fixed at the mechanism: the backfill now reconciles before it extracts, and one resolver serves both the write path and the backfill.\n\nThose are worth reading rather than summarizing, because the list is diagnostic. The most-wanted missing page is requested by 21 different articles. Then one wanted by 14, one by 12. Then this run: `oip-convergence-edge-11`, `-22`, `-33`, `-44`, `-55`, `-66`, `-77`, `-88`, `-99`, and `-1010`, each requested by exactly 10 articles.\n\nThose are not ten missing pages. Doubled digits ending in `1010` is what a loop that writes an index twice produces. Some generator built those link targets by concatenating a number with itself, and 100 broken links have been sitting in the corpus ever since, pointing at pages that were never going to exist under those names. Nobody found it because nothing could compute the graph. The dead endpoint was hiding its own evidence.\n\n## A download that returns a different download\n\nWhile measuring the export routes, a worse thing surfaced. Asking for one tag returned the entire library.\n\nNot an error. HTTP 200, `content-type: text/markdown`, 9.9 megabytes, and no header anywhere in the response saying that what came back was not what was asked for. A person clicking \"download the peptide folder\" would receive the whole site and have no way to know.\n\nThree layers had to agree for that to happen, and they did.\n\nFirst, the export handler read the whole corpus and filtered afterward in code. Any scope wider than a single article pulled the same 90 megabytes into one worker and threw — the same arithmetic failure as the graph.\n\nSecond, this site has an always-up layer, and it earns its place: it keeps a snapshot of the last good response so a page mid-rewrite serves the last good version instead of a 500. Its snapshot key is the pathname. The query string is discarded. So every scope of `/api/articles/export` — one article, one tag, one category, the entire library — shared exactly one snapshot. When the live handler failed, the caller received whichever scope had most recently succeeded.\n\nThird, the eligibility rule that decides which paths may hold a snapshot was consulted before *writing* one and not before *serving* one. So a path forbidden from ever storing a snapshot could still be answered from one.\n\nThe comment sitting directly above the neighbouring rule already said a bare request only, \"so a parameterized response never overwrites the canonical snapshot.\" The reasoning was written down. It was applied to one pattern and not to the one next to it.\n\nAll three are fixed. Tag and category now select their own rows in SQL and read nothing else. The whole library reads in windows. Snapshot eligibility requires no query string, and now governs serving as well as writing.\n\nThe general rule this leaves behind is worth more than the fix: **a cache keyed on less than the request will eventually answer a different question than the one asked.** A route whose meaning lives in its query cannot be keyed on its path.\n\n## Where you are, and what you can take\n\nEvery download named above was reachable only by knowing its URL, or by three plain-text links at the very bottom of an article, below the sources. Which is why the owner asked twice for downloads that already existed. A capability nobody can find is indistinguishable from one that was never built.\n\nThere is now a trail across the top of every article:\n\n> Home / peptide / BPC-157\n\nEach segment is a link, and the trail carries one control that hands you any level of it as a folder:\n\n- **This article** — markdown, json, its folder as a zip, its skill file\n- **Its history** — the last 1, 10, or 100 revisions as markdown, or all of them as json\n- **Everything under its tag** — markdown, or the tag folder as a zip\n- **The whole site** — markdown, the folder of folders as a zip, the Obsidian vault, the link graph\n\nThe nesting the owner described is exactly the nesting that exists: a page is a folder, a tag is a folder of page folders, the site is a folder of folders. Unzip the site folder and it opens as a vault.\n\nClicking the middle crumb used to land on the homepage. /t/<tag> was not a route at all — /t/peptide and /t/zzzznotatag returned the same bytes — so the trail asserted a hierarchy the site could not honour. It is a page now: the articles under that tag newest first, the sibling tags they are also filed under, and the folder downloads at that level. An unknown tag returns a 404 that names where to go instead of a homepage that pretends.\n\nOne more thing had to be true before any of this was portable. Every note this build exported carried its YAML frontmatter a few hundred bytes into the file, because the self-explaining preamble was prepended in front of it. Obsidian reads properties only from the opening three dashes at byte 0, so the whole export looked correct and had no machine-readable identity at all: no properties panel, nothing a Bases view could query, nothing a round trip could compare. The frontmatter now leads the file and carries a durable ms:article:<slug> id, the revision count, the content hash and the permalink. It is the same defect shape as a slot composer burying an authored body — a wrapper winning over the content it was meant to introduce — and it is pinned by a test that asserts the first four bytes.\n\n### The middle crumb is a tag, and that was not a preference\n\nThe obvious thing to put between Home and the article is its category. That does not work here, and the reason is a fact about this corpus rather than a design opinion: **2,235 of 2,317 published articles have no category at all.** Of the 82 that do, some disagree with themselves — `Governance` and `governance`, `system` and `systems`, `peptides` and `compound`.\n\nTags cover 2,174 of 2,317. The folder builder already groups the corpus by tag. So the tag became the middle crumb, because it is the only level that both exists for nearly every page and names the same thing the download hands over. A breadcrumb that names a folder the download does not produce is a lie told in navigation.\n\n## What is inherited from Obsidian, and what is refused\n\nThe field this borrows from is inventoried in [[second-brain-register]] — who built which part, and which parts last. Obsidian is the right compatibility target for one structural reason: a vault is a folder of markdown files and attachments on a disk. There is no proprietary container to reverse-engineer. Shida Li and Erica Xu released the first beta on 30 March 2020 and 1.0.0 on 13 October 2022; the storage model has always been ordinary files.\n\nThat makes it something rare — a widely used knowledge tool whose native format is the format this site already stores. The primitives worth inheriting are few:\n\n**Files and folders.** A page is a file. A tag is a folder. Already true in the export.\n\n**Properties.** YAML at the top of the file. Every exported note needs a stable identifier, a revision, a content hash, its slug, its permalink, its tags, and its updated time.\n\n**Internal links and backlinks.** `[[double brackets]]`, and the reverse index derived from them. Backlinks are never authored — they are the edge table read the other way.\n\n**Canvas.** Obsidian's canvas format is an open specification, JSON Canvas 1.0, published 11 March 2024. Two optional top-level arrays, `nodes` and `edges`, with four node types: `text`, `file`, `link`, and `group`. A category canvas is a mechanical projection of the edge table into that shape — no drawing, no layout engine, just nodes and edges written to a file Obsidian already opens.\n\n**Bases.** A core plugin since Obsidian 1.9.0 on 21 May 2025, and a `.base` file is plain YAML with `filters`, `formulas`, `properties`, `summaries`, and `views`. That means a saved query over the corpus is a text file this build can generate — \"every article with an unsourced claim\", \"everything updated this week\" — and it opens as a table in the reader's own app.\n\nNow the refusals, which matter as much.\n\n**Do not treat visible wikilinks as the graph.** There are two different things here and merging them destroys both. An editorial link is one a writer placed for a reader, inside a sentence, because it belongs there. A typed relation is a machine-readable fact — this supports that, this contradicts that, this supersedes that — which may or may not deserve to interrupt a sentence. The edge table stores relations. The renderer decides how each one appears: an anchor on the site, `[[a title]]` in a vault, a relative path in portable markdown, a target id in json. One relation, four representations.\n\n**Do not let the filesystem become the source of truth.** A path, a title, a slug, and a category all change. An identity may not. Folders are a projection of collections, and moving a file changes its projection, not what it is. This matters concretely, not theoretically: an article can belong to peptides and to neurology at once, and a file can only sit in one folder without being duplicated.\n\n**Do not attempt to inherit every Obsidian feature.** Obsidian has a plugin ecosystem in the thousands. Cloning it is an unbounded project that would never finish and would produce a worse Obsidian. Inherit the data model and the handful of interactions above; let everything else be a view over those primitives, or a local tool the reader already owns.\n\n**Do not build automatic two-way sync yet, and be honest about why.** One-way — the site hands you a vault — is safe and works today. The reverse direction is where knowledge bases get destroyed. It needs a recorded base revision per file and a genuine three-way comparison: only the remote changed, update the file; only the file changed, propose a revision; both changed from the same base, refuse and raise a conflict. A missing file is never an inferred deletion. A model may propose a merge; the model that produced the merge may not be the thing that certifies it preserved everything. Until that state machine exists and is tested, shipping a push path would be shipping a way to lose work.\n\n## What a model can do with this that it could not do before\n\nThis is the part that pays for the rest. When the corpus is a graph you can query rather than prose you must read, the instruction \"organize this\" stops being a request for judgment and becomes a series of checkable operations.\n\n- **Find what nothing points at.** One query. 1,721 answers today.\n- **Find what has been asked for and never written.** One query over unresolved targets, ordered by how many articles want it. The top answer is wanted by 21 pages. That is a commission, not a suggestion.\n- **Find generator bugs.** The `-1010` run was invisible in 2,317 bodies and obvious in one sorted list.\n- **Propose links without writing them.** Candidate edges from shared sources, shared tags, and exact title matches, submitted as a patch — moves, new links, removed links, redirects, the articles affected, and the revisions expected. Deterministic checks confirm every target resolves before anything is applied.\n- **Reorganize as a filesystem operation.** Nested folders and linked markdown are a shape every coding model already knows how to manipulate. It needs no special protocol for this site.\n\nThe last one is the actual prize, and it is why the graph had to become a table before anything else could be built on it.\n\n## The invariants this leaves behind\n\nThese are the rules the repairs imply. They are stated so the next failure of the same class is recognizable.\n\n**An edge is a row.** A graph derived per request over the corpus will pass every test at small scale and die silently at large scale. If it cannot be answered by an index, it will eventually not be answered at all.\n\n**A cache key must cover the whole request.** Keyed on less, it will hand one caller another caller's answer, with a success code.\n\n**One gate, both directions.** A path forbidden from writing a cached copy must not be served one either. Two halves of one rule drift apart the moment they are written in two places.\n\n**An unresolved link is kept.** It is the strongest signal in the corpus about what to write next. Deleting it is deleting the evidence.\n\n**Identity is not a path.** Slugs, titles, folders and categories are projections. What a thing is survives being moved.\n\n**A model may propose a reorganization and may not certify its own.** The thing that generated the merge does not get to be the thing that confirms nothing was lost.\n\n## What is not done\n\nStated plainly, because a specification that hides its gaps is worse than no specification.\n\nThe 1,721 unlinked articles are still unlinked — the graph can now name them, which is what makes the work possible, but nothing has connected them yet. The 433 broken links are still broken, including the 100 from the doubled-digit generator. The vault export does not yet carry a full YAML property block with a stable identifier per note, and until it does, a round trip cannot be verified by hash. There is no push path, for the reasons above. There is no manifest binding an export to one content head, so two people downloading \"the whole site\" a minute apart cannot yet prove they hold the same thing.\n\nThe rules above are written the way [[coding-law]] and [[writing-law]] are written, in the surfaces models load, not in prose, and for the same reason: a correction that lives only in prose is a correction the next agent never sees.\n\nEach of those is a task row, not a paragraph in a report. The distinction is the point: this document describes what was measured and repaired. What remains is work, and work here exists only as a row that names its own acceptance test.\n","register":"standard","hero":null,"hero_brief":"","editorial_review":null,"tags":["knowledge-graph","obsidian","markdown","second-brain","architecture","export"],"category":"systems","style":{},"claims":[{"id":"c1","text":"The corpus link graph is now materialized as rows in an article_links table, written at the article write path and read by index, rather than derived per request by reading every article body.","tier":"mechanism","source_ids":["s1"],"why_material":"It is the difference between a graph that answers and a graph that cannot run: the derived version read ~90 MB per call and returned Cloudflare error 1102."},{"id":"c2","text":"Before 6 August 2026, /api/articles/graph-links, /api/articles/graph-lint, /api/articles/next-acts and /api/articles/obsidian-vault all returned HTTP 503 with Cloudflare error code 1102 against the live corpus; all four now return HTTP 200.","tier":"mechanism","source_ids":["s1"],"why_material":"Every downstream surface — backlinks, orphan detection, the write-next queue, the vault projection — depended on endpoints that could not execute, while still appearing to exist."},{"id":"c3","text":"At the backfill pass on 6 August 2026 the corpus held 2,317 published articles carrying 4,206 edges from 596 articles, leaving 1,721 articles (74 percent) with no outbound link, and 433 links naming a page that does not exist.","tier":"mechanism","source_ids":["s1"],"why_material":"It replaces an impression of the corpus with a measurement, and it sets the falling ceiling the linking work is judged against."},{"id":"c4","text":"/api/articles/export?tag=<tag> returned HTTP 200 with the entire 9.9 MB library instead of the requested tag, because the live handler failed and a snapshot layer keyed only on pathname served whichever scope had most recently succeeded.","tier":"mechanism","source_ids":["s6"],"why_material":"A download that silently substitutes a different scope and reports success is worse than an error, because the caller has no signal that the artifact is wrong."},{"id":"c5","text":"A cache or snapshot keyed on less than the full request will eventually answer a question other than the one asked; a route whose meaning lives in its query string cannot be keyed on its path alone.","tier":"mechanism","source_ids":["s6"],"why_material":"It generalizes the export defect into the invariant that prevents its recurrence on any other parameterized route under the same middleware."},{"id":"c6","text":"An Obsidian vault is a folder of Markdown files and attachments on disk, and its canvas and database layers are open plain-text formats — JSON Canvas 1.0 for canvases and YAML .base files for saved views — so a site can generate both without reverse-engineering a proprietary container.","tier":"definition","source_ids":["s4","s5"],"why_material":"It is the reason Obsidian is a viable compatibility target rather than an integration project: the export format is the format already stored."},{"id":"c7","text":"Karpathy's llm-wiki pattern, published as a GitHub gist in April 2026, proposes that a language model compile raw sources once into a persistent interlinked Markdown wiki and query the wiki thereafter, rather than re-reading raw sources per query.","tier":"expert","source_ids":["s2","s3"],"why_material":"It is the external statement of the architecture this build already names in its own code, and it is why a computable link graph is a prerequisite rather than a nicety."},{"id":"c8","text":"2,235 of 2,317 published articles carry no category value, so the navigational level between the site root and an article is derived from the article's primary tag, which covers 2,174 of 2,317.","tier":"mechanism","source_ids":["s1"],"why_material":"A breadcrumb level that names a grouping the download cannot produce would misdescribe the folder it hands over."}],"sources":[{"id":"s1","url":"https://miscsubjects.com/api/articles/graph-links","title":"miscsubjects link graph endpoint (live)","quote":"{\"articles\": 2297, \"edges\": 3681, \"unresolved\": 425}"},{"id":"s2","url":"https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f","title":"Karpathy, llm-wiki (GitHub gist, April 2026)","quote":"A pattern for building personal knowledge bases using LLMs. This is an idea file, it is designed to be copy pasted to your own LLM Agent."},{"id":"s3","url":"https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f","title":"Karpathy, llm-wiki — why a compiled wiki beats retrieval","quote":"the wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged."},{"id":"s4","url":"https://jsoncanvas.org/","title":"JSON Canvas — open file format for infinite canvas data","quote":"The JSON Canvas format was created to provide longevity, readability, interoperability, and extensibility to data created with infinite canvas apps."},{"id":"s5","url":"https://obsidian.md/help/bases/syntax","title":"Obsidian Bases syntax (core plugin, .base files)","quote":"filters:\n  or:\n    - file.hasTag(\"tag\")\n    - and:\n        - file.hasTag(\"book\")\n        - file.hasLink(\"Textbook\")"},{"id":"s6","url":"https://miscsubjects.com/api/articles/export?tag=peptide","title":"miscsubjects tag-scoped export (live, header reports the count)","quote":"x-articles-exported: 56 — the live handler answers the requested scope; before the repair this route returned 9,912,730 bytes of the whole library."}],"prov":{"model":"unattributed","action":"write"}}