# The site as a folder you can hold: what the portable content graph actually measured

slug: the-vault-projection · https://miscsubjects.com/a/the-vault-projection · category: systems · tags: knowledge-graph, obsidian, markdown, second-brain, architecture, export · updated 2026-08-06T08:24:32.106Z

A folder on your computer and a page on this site should be the same object, seen from two places. That is the whole idea. Everything below is what it takes to mean it, and what was actually measured here on 6 August 2026 when someone finally checked.

## What was already built, and what was actually running

This site could already hand you a folder. A single page as markdown, a page as a folder with its claims and sources and provenance beside it, every article carrying one tag as a folder of page folders, the whole library as a folder of folders, and an Obsidian vault export with wikilinks and a canvas file. All of it existed. Some of it had existed for weeks.

Four of those endpoints were dead.

Not slow. Not degraded. Answering `error code: 1102`, which is what Cloudflare returns when a worker exceeds its resource limit and is killed. The link graph, the graph lint, the ranked queue of what to write next, and the Obsidian vault export — the four surfaces that the entire portable-knowledge idea rests on — could not run at all against the live corpus.

The reason is worth stating exactly, because it is a general failure and not a typo. The graph was derived on every request. A request would ask "what links to what?", and the code would select the body of all 2,317 published articles, pull roughly 90 megabytes of prose into a single worker, and run regular expressions over it. That worked at 200 articles. It worked at 800. Somewhere on the way to 2,317 it stopped working, and nothing said so, because no test measured the endpoint at corpus scale. The code was correct. The architecture was arithmetic that ran out.

So the first repair is not a feature. Edges are rows now.

## Edges are rows

There is a table. Each row is one link: the article it was typed in, the target as written, the kind of link, the resolved target, the display label, the heading anchor, and whether it resolved.

It is written at the write path — the single moment a body changes — and read back by index. Asking "what links here" is now one indexed lookup instead of a scan of the corpus, which is why it can render on every page. Asking for the whole graph reads 4,206 small rows instead of 90 megabytes of prose.

An identity note that matters more than it looks: a row is identified by the article it came from, the target, and the kind. Not by a row number. So re-syncing one article is a delete-then-insert of that article's own rows and can never disturb another article's edges. One page being rewritten does not put the graph in an undefined state.

A link whose target does not exist is kept, not thrown away. `to_slug` is null, `resolved` is 0, and the row stays. This is deliberate. An unresolved link is the most valuable finding the corpus can produce: it is a page that a writer explicitly asked for, in prose, by naming it. Discarding those rows would make the graph look healthier than it is, which is the one thing a graph must never do.

## What the corpus actually looks like

Here is the state on the day the table was filled, from the corpus itself rather than from anyone's impression of it.

**2,317 published articles. 4,206 edges from 596 of them. 1,721 articles — 74 percent — point at nothing at all.**

Those are the figures from the backfill pass on 6 August 2026. They move, and they are meant to: the live count is served by the graph endpoint itself, and every article published or unpublished since changes it. Quoting a frozen number as the current one is the failure this build treats as serious, so the number above is dated and the endpoint is the authority.

The claim that started this was that no article on the site links to any other article. That is not what the numbers say: 596 articles do link out, and one page carries 66 internal links. But the shape of the complaint was right and the direction was right. Three quarters of the corpus is a leaf. It sits there, correctly written, sourced, adjudicated, and connected to nothing. A knowledge base where three quarters of the pages are unreachable from any other page is a pile of documents that happen to share a domain name.

**318 of those links name a page that does not exist.**

That number started at 433 and fell twice, and both falls were the measurement getting honest rather than the corpus getting better. 100 of the original were stale rows: ten pages had been repaired upstream, and the backfill could add an edge but never remove one, because its candidate filter only visited articles that still carried a link pattern — an article whose links were deleted was never looked at again. Another 15 were links to the laws, which render at /a/<slug> from code rather than from rows, so a resolver that only checked the articles table called every one of them broken. A broken-link list salted with pages that already answer 200 is a list nobody can act on, so both were fixed at the mechanism: the backfill now reconciles before it extracts, and one resolver serves both the write path and the backfill.

Those are worth reading rather than summarizing, because the list is diagnostic. The most-wanted missing page is requested by 21 different articles. Then one wanted by 14, one by 12. Then this run: `oip-convergence-edge-11`, `-22`, `-33`, `-44`, `-55`, `-66`, `-77`, `-88`, `-99`, and `-1010`, each requested by exactly 10 articles.

Those are not ten missing pages. Doubled digits ending in `1010` is what a loop that writes an index twice produces. Some generator built those link targets by concatenating a number with itself, and 100 broken links have been sitting in the corpus ever since, pointing at pages that were never going to exist under those names. Nobody found it because nothing could compute the graph. The dead endpoint was hiding its own evidence.

## A download that returns a different download

While measuring the export routes, a worse thing surfaced. Asking for one tag returned the entire library.

Not an error. HTTP 200, `content-type: text/markdown`, 9.9 megabytes, and no header anywhere in the response saying that what came back was not what was asked for. A person clicking "download the peptide folder" would receive the whole site and have no way to know.

Three layers had to agree for that to happen, and they did.

First, the export handler read the whole corpus and filtered afterward in code. Any scope wider than a single article pulled the same 90 megabytes into one worker and threw — the same arithmetic failure as the graph.

Second, this site has an always-up layer, and it earns its place: it keeps a snapshot of the last good response so a page mid-rewrite serves the last good version instead of a 500. Its snapshot key is the pathname. The query string is discarded. So every scope of `/api/articles/export` — one article, one tag, one category, the entire library — shared exactly one snapshot. When the live handler failed, the caller received whichever scope had most recently succeeded.

Third, the eligibility rule that decides which paths may hold a snapshot was consulted before *writing* one and not before *serving* one. So a path forbidden from ever storing a snapshot could still be answered from one.

The comment sitting directly above the neighbouring rule already said a bare request only, "so a parameterized response never overwrites the canonical snapshot." The reasoning was written down. It was applied to one pattern and not to the one next to it.

All three are fixed. Tag and category now select their own rows in SQL and read nothing else. The whole library reads in windows. Snapshot eligibility requires no query string, and now governs serving as well as writing.

The general rule this leaves behind is worth more than the fix: **a cache keyed on less than the request will eventually answer a different question than the one asked.** A route whose meaning lives in its query cannot be keyed on its path.

## Where you are, and what you can take

Every download named above was reachable only by knowing its URL, or by three plain-text links at the very bottom of an article, below the sources. Which is why the owner asked twice for downloads that already existed. A capability nobody can find is indistinguishable from one that was never built.

There is now a trail across the top of every article:

> Home / peptide / BPC-157

Each segment is a link, and the trail carries one control that hands you any level of it as a folder:

- **This article** — markdown, json, its folder as a zip, its skill file
- **Its history** — the last 1, 10, or 100 revisions as markdown, or all of them as json
- **Everything under its tag** — markdown, or the tag folder as a zip
- **The whole site** — markdown, the folder of folders as a zip, the Obsidian vault, the link graph

The nesting the owner described is exactly the nesting that exists: a page is a folder, a tag is a folder of page folders, the site is a folder of folders. Unzip the site folder and it opens as a vault.

Clicking the middle crumb used to land on the homepage. /t/<tag> was not a route at all — /t/peptide and /t/zzzznotatag returned the same bytes — so the trail asserted a hierarchy the site could not honour. It is a page now: the articles under that tag newest first, the sibling tags they are also filed under, and the folder downloads at that level. An unknown tag returns a 404 that names where to go instead of a homepage that pretends.

One more thing had to be true before any of this was portable. Every note this build exported carried its YAML frontmatter a few hundred bytes into the file, because the self-explaining preamble was prepended in front of it. Obsidian reads properties only from the opening three dashes at byte 0, so the whole export looked correct and had no machine-readable identity at all: no properties panel, nothing a Bases view could query, nothing a round trip could compare. The frontmatter now leads the file and carries a durable ms:article:<slug> id, the revision count, the content hash and the permalink. It is the same defect shape as a slot composer burying an authored body — a wrapper winning over the content it was meant to introduce — and it is pinned by a test that asserts the first four bytes.

### The middle crumb is a tag, and that was not a preference

The obvious thing to put between Home and the article is its category. That does not work here, and the reason is a fact about this corpus rather than a design opinion: **2,235 of 2,317 published articles have no category at all.** Of the 82 that do, some disagree with themselves — `Governance` and `governance`, `system` and `systems`, `peptides` and `compound`.

Tags cover 2,174 of 2,317. The folder builder already groups the corpus by tag. So the tag became the middle crumb, because it is the only level that both exists for nearly every page and names the same thing the download hands over. A breadcrumb that names a folder the download does not produce is a lie told in navigation.

## What is inherited from Obsidian, and what is refused

The field this borrows from is inventoried in [[second-brain-register]] — who built which part, and which parts last. Obsidian is the right compatibility target for one structural reason: a vault is a folder of markdown files and attachments on a disk. There is no proprietary container to reverse-engineer. Shida Li and Erica Xu released the first beta on 30 March 2020 and 1.0.0 on 13 October 2022; the storage model has always been ordinary files.

That makes it something rare — a widely used knowledge tool whose native format is the format this site already stores. The primitives worth inheriting are few:

**Files and folders.** A page is a file. A tag is a folder. Already true in the export.

**Properties.** YAML at the top of the file. Every exported note needs a stable identifier, a revision, a content hash, its slug, its permalink, its tags, and its updated time.

**Internal links and backlinks.** `[[double brackets]]`, and the reverse index derived from them. Backlinks are never authored — they are the edge table read the other way.

**Canvas.** Obsidian's canvas format is an open specification, JSON Canvas 1.0, published 11 March 2024. Two optional top-level arrays, `nodes` and `edges`, with four node types: `text`, `file`, `link`, and `group`. A category canvas is a mechanical projection of the edge table into that shape — no drawing, no layout engine, just nodes and edges written to a file Obsidian already opens.

**Bases.** A core plugin since Obsidian 1.9.0 on 21 May 2025, and a `.base` file is plain YAML with `filters`, `formulas`, `properties`, `summaries`, and `views`. That means a saved query over the corpus is a text file this build can generate — "every article with an unsourced claim", "everything updated this week" — and it opens as a table in the reader's own app.

Now the refusals, which matter as much.

**Do not treat visible wikilinks as the graph.** There are two different things here and merging them destroys both. An editorial link is one a writer placed for a reader, inside a sentence, because it belongs there. A typed relation is a machine-readable fact — this supports that, this contradicts that, this supersedes that — which may or may not deserve to interrupt a sentence. The edge table stores relations. The renderer decides how each one appears: an anchor on the site, `[[a title]]` in a vault, a relative path in portable markdown, a target id in json. One relation, four representations.

**Do not let the filesystem become the source of truth.** A path, a title, a slug, and a category all change. An identity may not. Folders are a projection of collections, and moving a file changes its projection, not what it is. This matters concretely, not theoretically: an article can belong to peptides and to neurology at once, and a file can only sit in one folder without being duplicated.

**Do not attempt to inherit every Obsidian feature.** Obsidian has a plugin ecosystem in the thousands. Cloning it is an unbounded project that would never finish and would produce a worse Obsidian. Inherit the data model and the handful of interactions above; let everything else be a view over those primitives, or a local tool the reader already owns.

**Do not build automatic two-way sync yet, and be honest about why.** One-way — the site hands you a vault — is safe and works today. The reverse direction is where knowledge bases get destroyed. It needs a recorded base revision per file and a genuine three-way comparison: only the remote changed, update the file; only the file changed, propose a revision; both changed from the same base, refuse and raise a conflict. A missing file is never an inferred deletion. A model may propose a merge; the model that produced the merge may not be the thing that certifies it preserved everything. Until that state machine exists and is tested, shipping a push path would be shipping a way to lose work.

## What a model can do with this that it could not do before

This is the part that pays for the rest. When the corpus is a graph you can query rather than prose you must read, the instruction "organize this" stops being a request for judgment and becomes a series of checkable operations.

- **Find what nothing points at.** One query. 1,721 answers today.
- **Find what has been asked for and never written.** One query over unresolved targets, ordered by how many articles want it. The top answer is wanted by 21 pages. That is a commission, not a suggestion.
- **Find generator bugs.** The `-1010` run was invisible in 2,317 bodies and obvious in one sorted list.
- **Propose links without writing them.** Candidate edges from shared sources, shared tags, and exact title matches, submitted as a patch — moves, new links, removed links, redirects, the articles affected, and the revisions expected. Deterministic checks confirm every target resolves before anything is applied.
- **Reorganize as a filesystem operation.** Nested folders and linked markdown are a shape every coding model already knows how to manipulate. It needs no special protocol for this site.

The last one is the actual prize, and it is why the graph had to become a table before anything else could be built on it.

## The invariants this leaves behind

These are the rules the repairs imply. They are stated so the next failure of the same class is recognizable.

**An edge is a row.** A graph derived per request over the corpus will pass every test at small scale and die silently at large scale. If it cannot be answered by an index, it will eventually not be answered at all.

**A cache key must cover the whole request.** Keyed on less, it will hand one caller another caller's answer, with a success code.

**One gate, both directions.** A path forbidden from writing a cached copy must not be served one either. Two halves of one rule drift apart the moment they are written in two places.

**An unresolved link is kept.** It is the strongest signal in the corpus about what to write next. Deleting it is deleting the evidence.

**Identity is not a path.** Slugs, titles, folders and categories are projections. What a thing is survives being moved.

**A model may propose a reorganization and may not certify its own.** The thing that generated the merge does not get to be the thing that confirms nothing was lost.

## What is not done

Stated plainly, because a specification that hides its gaps is worse than no specification.

The 1,721 unlinked articles are still unlinked — the graph can now name them, which is what makes the work possible, but nothing has connected them yet. The 433 broken links are still broken, including the 100 from the doubled-digit generator. The vault export does not yet carry a full YAML property block with a stable identifier per note, and until it does, a round trip cannot be verified by hash. There is no push path, for the reasons above. There is no manifest binding an export to one content head, so two people downloading "the whole site" a minute apart cannot yet prove they hold the same thing.

The rules above are written the way [[coding-law]] and [[writing-law]] are written, in the surfaces models load, not in prose, and for the same reason: a correction that lives only in prose is a correction the next agent never sees.

Each of those is a task row, not a paragraph in a report. The distinction is the point: this document describes what was measured and repaired. What remains is work, and work here exists only as a row that names its own acceptance test.


## Sources

1. miscsubjects link graph endpoint (live) — https://miscsubjects.com/api/articles/graph-links
2. Karpathy, llm-wiki (GitHub gist, April 2026) — https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
3. Karpathy, llm-wiki — why a compiled wiki beats retrieval — https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
4. JSON Canvas — open file format for infinite canvas data — https://jsoncanvas.org/
5. Obsidian Bases syntax (core plugin, .base files) — https://obsidian.md/help/bases/syntax
6. miscsubjects tag-scoped export (live, header reports the count) — https://miscsubjects.com/api/articles/export?tag=peptide


---

# The second brain register: who built the plain-text knowledge base, and which part of it lasts

slug: second-brain-register · https://miscsubjects.com/a/second-brain-register · category: systems · tags: second-brain, obsidian, markdown, knowledge-graph, reference, zettelkasten · updated 2026-08-06T07:20:07.306Z

This is a register of the people, formats and programs that built the idea of a personal knowledge base out of plain text files. It exists to be tracked rather than read once: when someone in this field publishes something that changes the shape of the problem, the entry for them is where it goes.

It is organised by what each entry actually contributed, because the field has a habit of crediting tools for ideas and ideas for tools. The formats outlast the apps. The apps outlast the methods. Most of the methods are the same three ideas in different packaging.

One correction first, because it is the thing people get wrong in their first hour.

## The syntax is two brackets, not three

A link between two notes is written with two square brackets on each side.

    [[bpc-157]]

Obsidian's own documentation lists that as the wikilink format. Two more forms matter. A pipe changes what the reader sees while leaving the target intact:

    [[bpc-157|the compound itself]]

And a hash points at a heading inside the target:

    [[the-obedience-gap#Part III]]

Three brackets is not a syntax. It is what people remember after seeing `[[ ]]` once, and it silently fails to link, which is the worst kind of mistake: the note looks written and the graph stays empty.

---

## The lineage

### Niklas Luhmann — the slip box, and the reason links come before folders

A German sociologist who worked from a physical card index for decades and published at a rate that made people ask how. The answer was in the box: each card carried one idea, a fixed identifier, and pointers to other cards. Ideas were retrieved by following links, not by remembering which drawer they were in.

The load-bearing part is not the cards. It is that **the address of a note is permanent and its position is not**. Every good system since has re-derived that rule, and every bad one has tried to replace it with a better folder hierarchy.

**Track:** the secondary literature; his own archive has been digitised and published by Bielefeld University.

### Sönke Ahrens — the method, written down

*How to Take Smart Notes* is the book that turned Luhmann's practice into something a person could follow, and it is the source most later systems are quietly restating. Its useful contribution is the separation of note types: fleeting capture, literature notes tied to a source, and permanent notes written in your own words. The distinction between "I saved this" and "I understood this" is the whole discipline.

**Track:** the book. There is a revised edition.

### Andy Matuschak — evergreen notes

The most rigorous thinking in the field, published as a working note site rather than a book. His definition: "*Evergreen notes* are written and organized to evolve, contribute, and accumulate over time, across projects."

Three principles follow, stated as titles of their own notes: "Evergreen notes should be atomic", "Evergreen notes should be concept-oriented", and "Evergreen notes should be densely linked".

Those three lines are the best available specification for what a note in a knowledge base should be, and they map directly onto machine operations. Atomic means one addressable claim. Concept-oriented means the title is the assertion. Densely linked means the graph is the product, not a visualisation of it.

**Track:** notes.andymatuschak.org. It updates in place, which is the point.

### Tiago Forte — the name, and PARA

The person who made "second brain" a category. His organisational scheme, PARA, is described in his own words as "a simple, comprehensive, yet extremely flexible system for organizing any type of digital information across any platform": Projects, Areas, Resources, Archives.

PARA is a folder scheme, and its honest reading is that it solves *actionability*, not knowledge. It sorts by how soon you need something. That is genuinely useful for work and close to useless for a corpus of research, where the same article is needed at unpredictable times forever. Take the vocabulary, notice what it optimises for, and do not adopt it as a taxonomy for a reference library.

**Track:** fortelabs.com. High output, commercially framed.

### Nick Milo — maps of content

The idea worth having from this corner: when a subject accumulates enough notes, you write a note whose job is to be a curated index of them, by hand, with the links in a deliberate order. A map of content is an authored entry point, not a generated one.

Generated indexes and authored maps are different objects and a good system has both. This site's ranked write-next queue is the generated kind; a map of content is the human kind.

**Track:** Linking Your Thinking.

### Maggie Appleton — the digital garden

The clearest articulation of publishing notes in an unfinished state, with the growth stage stated openly rather than implied. The relevant discipline is that a page declares its own confidence — seedling, budding, evergreen — so a reader knows what they are holding. Any system that publishes work in progress needs some version of this, or every page reads as a finished claim.

**Track:** maggieappleton.com.

---

## The formats — the layer that actually survives

Apps get acquired and shut down. Methods go out of fashion. A file format with a specification and multiple implementations is the only part of this field you can build on for a decade.

### CommonMark, and GitHub Flavored Markdown

Markdown had no precise specification for years, so every implementation disagreed at the edges. CommonMark is the formal one; GFM is the widely-implemented superset that adds tables, task lists and strikethrough. Anything intended to be portable should target CommonMark plus the specific GFM extensions it actually needs, and should say which.

### YAML frontmatter

A block of key–value metadata at the top of a Markdown file, delimited by `---`. It came from static site generators and became the universal way to attach properties to a note. Its real limitation matters for design: nested structures are awkward to edit and awkward to validate. Keep frontmatter flat — identifier, revision, hash, slug, tags, dates — and put anything structured in a sibling file.

### JSON Canvas

The most underrated entry here. Obsidian's canvas files are an open format, JSON Canvas 1.0, published 11 March 2024. In its own words, it "was created to provide longevity, readability, interoperability, and extensibility to data created with infinite canvas apps."

The whole spec is two optional top-level arrays — `nodes` and `edges` — with four node types: `text`, `file`, `link`, and `group`. That is small enough to generate from a link graph with no library and no layout engine. If you already hold edges, you can write a canvas.

### Obsidian Bases

A core plugin since Obsidian 1.9.0 on 21 May 2025, and the important thing is the file: a `.base` is plain YAML with `filters`, `formulas`, `properties`, `summaries` and `views`. A filter looks like this:

    filters:
      or:
        - file.hasTag("tag")
        - and:
            - file.hasTag("book")
            - file.hasLink("Textbook")

That means a saved query over a corpus is a text file a build can generate and a reader opens as a live table in their own app. It replaces the plugin that used to own this job.

### Zettelkasten identifiers, and Johnny Decimal

Two competing answers to naming. Zettelkasten practice uses an opaque permanent identifier, usually a timestamp, so the name never has to change. Johnny Decimal uses a strict numeric hierarchy so a human can hold the whole map in their head.

They are in genuine tension and the resolution is to use both for different things: an opaque identifier as the durable identity, a readable path as the projection. If the readable name is the identity, renaming breaks links.

---

## The programs

### Obsidian

Created by Shida Li and Erica Xu; first beta 30 March 2020, version 1.0.0 on 13 October 2022. Local Markdown files in a folder called a vault. Canvas arrived December 2022; Bases in 2025.

It is the right compatibility target for one structural reason, and it is not popularity: **there is no container to reverse-engineer.** A vault is files on a disk. Anything that can write Markdown and folders can produce one, and anything that can read them can consume one. The graph view, backlinks, properties and canvas are all derived from the files.

### Roam Research

The product that made bidirectional linking a mainstream expectation and, for a period, set the agenda the whole field responded to. Block-level references were its distinctive idea: the addressable unit is a paragraph, not a page. Its cautionary contribution is equally real — it was cloud-first, and the difficulty people had getting their material out of it is the strongest argument in this entire register for plain files on your own disk.

### Logseq

Open source, outliner-first, operating on local Markdown files. The main alternative for people who want blocks as the primary unit but not a hosted service.

### org-mode and org-roam

Emacs. Predates most of this field by decades and does more than any of it — notes, tasks, literate code, exported documents — in one plain-text format that is not Markdown. If you already live in Emacs it is the most powerful option available; if you do not, its format will not travel to the rest of these tools without conversion.

### TiddlyWiki

A self-contained wiki in a single HTML file, by Jeremy Ruston, and far older than the current wave. Worth knowing because it solved portability in the most literal way anyone has: the notes and the application are one file you can email.

### Dendron, Foam

Editor-native approaches. Foam builds a linked notes workflow inside VS Code; Dendron pursued hierarchical naming as the organising primitive. Both matter chiefly as evidence that wikilinks and backlinks are not an Obsidian feature — they are a pattern that reappears in any editor.

### Anytype, Tana, Capacities, Heptabase, Reflect

The current generation, each betting on a different primitive: local-first objects, supertags and structured queries, typed objects, spatial whiteboards, speed with AI assistance. Track them for ideas rather than as targets. None has an open format on the level of Markdown-in-folders, so exporting *to* them is a commitment and exporting *from* them is a risk.

### Zettlr, Joplin, SilverBullet, Athens

Also-rans and specialists worth one line each: Zettlr is academic-writing-first with citation support; Joplin is a sync-and-encryption-first notebook; SilverBullet is a self-hosted programmable notebook; Athens was an open-source Roam alternative that stopped. Athens is in the register because a dead project is a data point about which bets do not pay.

---

## The publishers

Turning a vault into a website is a solved problem with several good answers, and it matters here because the reverse direction — a website that emits a vault — is the same graph read the other way.

**Quartz**, by Jacky Zhao, is the strongest of them: "Quartz is a fast, batteries-included static-site generator that transforms Markdown content into fully functional websites", with full-text search, a graph view and Obsidian compatibility. Currently at v5.

**Obsidian Publish** is the first-party paid option. **Eleventy**, **Hugo** and **Astro** are general static site generators that people point at vaults with varying amounts of glue.

The thing to notice: every one of these treats the folder of Markdown as canonical and the site as the projection. This site runs that arrangement in reverse — the database owns identity and the folder is generated — which is a defensible choice, but it means the burden of proving a round trip is lossless sits here rather than with the tool.

---

## The Obsidian plugin layer, narrowed

The community plugin ecosystem is in the thousands and almost all of it is irrelevant to a build. These are the ones whose *ideas* are worth knowing, since the features either exist natively now or are worth reimplementing:

- **Dataview** — queries over frontmatter as if the vault were a database. This is the idea Bases has now absorbed into core, and its long popularity is the evidence that structured metadata is the feature people actually wanted.
- **Templater** — templates with logic, which is how a "new article of this kind" gesture becomes repeatable.
- **Excalidraw** — hand-drawn diagrams stored inside the vault rather than in a separate service.
- **Obsidian Git** — version control over the vault, which is what makes a vault a synchronisable artifact rather than a folder that hopefully still matches.
- **Smart Connections** — embedding-based related-note suggestions, i.e. semantic edges as a complement to authored ones.

The pattern worth extracting: the plugins that lasted all added *structure* over the same plain files. None of them added a new storage format.

---

## The LLM-era entries

### Karpathy's llm-wiki

Published as a GitHub gist in April 2026 and the most consequential recent entry in this register. It opens: "A pattern for building personal knowledge bases using LLMs. This is an idea file, it is designed to be copy pasted to your own LLM Agent."

The architecture is three layers — immutable raw sources, a wiki of model-written Markdown, and a schema file (`CLAUDE.md` or `AGENTS.md`) that tells the agent the rules — plus an `index.md` catalogue and a `log.md` history.

The argument is a direct attack on retrieval-per-query: instead of re-reading raw sources each time, the model compiles them once into a persistent interlinked wiki, because "the wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged."

Two things to note about it precisely, because both get misreported. It is a one-page idea file, not software. And it does use the phrase — it ends by telling you to open a local server and "start building your second brain."

**Track:** the gist, and the re-implementations. There are many; most add a runner and lose the point.

### Model Context Protocol servers over vaults

The mechanism by which an agent reads and writes a vault directly rather than being handed pasted text. Several Obsidian MCP servers exist. This is the layer to watch, because it determines whether "the model organises my notes" means a real filesystem operation or a chat transcript someone copies back by hand.

---

## What to take, and what to leave

How this build actually applies the list below — and what it measured when it did — is [[the-vault-projection]].

**Take the primitives.** Files, folders, flat properties, internal links, derived backlinks, assets, saved queries, canvases, history, portable export. Everything anyone has built in this field is a view over that list.

**Take two brackets and a stable identifier.** The link syntax and the permanent address are the only two decisions that are expensive to change later.

**Take the separation of authored from generated.** A map of content is written. An index is derived. A backlink list is derived. Conflating them produces a system where nobody can tell whether a page's connections were a judgment or a side effect.

**Leave the folder methods as taxonomy.** PARA and its relatives sort by actionability. A research corpus is not sorted by actionability; the same page is needed at unpredictable intervals forever.

**Leave the app-shaped features.** Cloning a plugin ecosystem is an unbounded project that ends in a worse copy of the original.

**Leave any format you cannot read without its application.** That is the whole lesson of the entries above that no longer exist.

---

## How to use this register

Each entry above is a thing to watch, not a thing to have read. The field moves in a specific way: a format gets specified, then apps implement it, then methods get written about it, then the methods get commercialised and the useful part gets buried. Watching the format layer is the highest-signal habit — JSON Canvas being published as an open spec did more for interoperability than any amount of writing about note-taking.

When a new entry appears, three questions decide whether it matters here:

1. **Does it define a format, or consume one?** Format definitions change what is possible. Consumers change what is convenient.
2. **Can its output be read with no software but a text editor?** If not, it is a destination, not a substrate.
3. **Does it make an operation mechanical that was previously judgment?** Those are the ones to copy immediately.

Everything else is a preference, and this register does not track preferences.


## Sources

1. Obsidian Help — Internal links (wikilink syntax) — https://obsidian.md/help/Linking%20notes%20and%20files/Internal%20links
2. Andy Matuschak — Evergreen notes — https://notes.andymatuschak.org/Evergreen_notes
3. Tiago Forte — The PARA Method — https://fortelabs.com/blog/para/
4. JSON Canvas — open file format for infinite canvas data — https://jsoncanvas.org/
5. Obsidian Bases syntax — .base files are plain YAML — https://obsidian.md/help/bases/syntax
6. Karpathy — llm-wiki (GitHub gist, April 2026) — https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
7. Karpathy — llm-wiki, the compiled-artifact argument — https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
8. Quartz — static-site generator for Markdown vaults — https://quartz.jzhao.xyz/
9. Obsidian (software) — creators, release dates, storage model — https://en.wikipedia.org/wiki/Obsidian_(software)


---

# This site now derives its own work queue — and exports itself as an Obsidian vault

slug: the-corpus-now-writes-its-own-work-queue · https://miscsubjects.com/a/the-corpus-now-writes-its-own-work-queue · category: canon · tags: canonical, loop, knowledge-graph, obsidian · updated 2026-08-03T05:15:04.556Z

On August 2, 2026, this system gained three connected capabilities in one session: article bodies now carry Obsidian-style `[[wikilinks]]` that render as ordinary links on the site and resolve natively in Obsidian; the entire corpus exports as a working Obsidian vault whose index, log, health report, and work queue are generated from the live database; and the system now computes, on demand, a ranked answer to the question every content operation struggles with — *what should be written next?* This page documents the mechanism with the receipts from the session that shipped it.

## The idea this merges

Andrej Karpathy published a pattern in 2026 he called the LLM wiki: instead of retrieving documents and re-synthesizing them on every question, keep a *compiled* knowledge base — markdown pages an LLM maintains — where every new source revises the existing pages rather than merely joining an archive. Three operations run it: **ingest** (new material updates many pages, not one), **query** (answers get filed back in), and **lint** (a standing health check for contradictions, orphan pages, stale claims, and missing cross-references). Two files anchor it: an `index.md` catalog and an append-only `log.md`. The closing point of his note reaches back to Vannevar Bush's 1945 Memex essay: a personal store of knowledge with associative trails between documents was designed eighty years ago, and the unsolved part was always who does the maintenance.

This system already had most of the hard parts: 2,264 articles in one database, claims with tiers and hash-chained sources, a public events ledger, an outreach pipeline with tracked sends and reply routing. What it did not have was the compiled-wiki discipline — no inbound-link index, no orphan detection, no mechanical answer to "what next," and an export that produced files without the graph. The merge kept this site as the canonical store and adopted exactly the parts of the wiki pattern that compound: the link graph, the lint pass, the queue, and the vault as a lossless projection.

## Links now round-trip between the site and Obsidian

An author — human or model — can now type `[[adjudication-calibration-study]]` or `[[adjudication-calibration-study|the calibration study]]` in any article body. On the reader's page it renders as a normal link to that article, with the article's real title as the text when no label is given. This paragraph does it live: the calibration numbers this system publishes are at [[adjudication-calibration-study]], and the full capability inventory is at [[the-build-end-to-end]].

Two details carry the weight:

A wikilink to a page that does not exist renders as a marked, unlinked span — and becomes a row in the system's gap report. A typed link to a missing page is treated as an authored request for that page. This is not hypothetical: the first run of the gap scan found **163 distinct missing pages already being requested by published bodies** — models writing the OIP corpus had been typing wikilinks to pages like `oip-total-structure` (requested by 171 different articles) that no one ever created. The queue found real, latent demand that had been invisible for weeks.

On export, the transform inverts. Canonical links (`/a/slug`) and typed wikilinks both become vault-resolvable wikilinks (`[[Peptides/bpc-157/README|label]]`), so Obsidian's graph view, backlinks pane, and unresolved-links report all work on the exported corpus with no plugins. A four-assertion round-trip test proves no link target is lost in either direction, and it runs in CI on every push.

## One derivation: the graph, the lint, the queue

Three public endpoints now serve one derivation over the whole corpus:

**`/api/articles/graph-links`** builds the full edge set from three sources: declared embeds, markdown links to `/a/` pages, and typed wikilinks. First live run: **3,386 edges across 2,264 articles**, with inbound links (backlinks) computed for every page — a structure that did not exist anywhere in the system before this session.

**`/api/articles/graph-lint`** is Karpathy's third operation, mechanized: orphans (no inbound links — 1,715 on first run, an honest number discussed below), missing pages (163), articles carrying claims with no source (494), articles with claims under active challenge (32), and stale hubs (heavily-linked pages that stopped moving — 0 at the current threshold). Every finding names the page and the defect. The pass is pure derivation: run it twice against an unchanged corpus and it returns identical output.

**`/api/articles/next-acts`** turns the lint into a ranked queue, and this is the piece that changes the loop. At generation time 2026-08-03T02:48:38Z the top of the live queue read: `write oip-total-structure (score 271)` — wikilinked from 171 articles, no page exists; `write oip-machine-plane (110)`; `write oip-ground (103)`. Below the writing acts sit `resolve` (challenged claims), `source` (unsourced claims), `revise` (stale hubs), `connect` (orphans) — and then the outward half: `respond` (unread replies outrank everything else in their band, because responses update priors), and `outreach` (high-fit audience classes that have gone quiet). Content work and outreach work come off the same queue, computed from the same graph. Performing the top act and re-running the derivation *is* the loop.

## The vault: the whole system as a folder of markdown

`GET /api/articles/obsidian-vault` (or `node scripts/obsidian_pull.mjs --all`) now produces a version-3 vault. The session's verification run pulled **370 files** for a two-article slice and verified **369 of 369 content hashes** against the `SHA256SUMS` manifest the server generates (the manifest does not hash itself).

What lands on disk, and why each file exists:

- **`index.md`** — Karpathy's catalog: every page, one line, with role, claim count, source count, backlink count, and last-updated date, grouped by ontology folder. The first file a model opens.
- **`log.md`** — the append-only chronology, projected from the live events ledger. The ledger stays canonical; the file is a read model.
- **`SCHEMA.md`** — the maintainer contract: the three operations, the authority rule, the conventions. **`CLAUDE.md`** and **`AGENTS.md`** are three-line pointers to it, so Claude Code, Codex, and any agentic CLI that auto-loads root files inherits the same contract from the same source.
- **`lint.md`** and **`next.md`** — the health report and the ranked queue, as files, regenerated on every pull from the same derivation the API serves.
- **`Misc.base`** — an Obsidian Bases file (the native database view that shipped in Obsidian 1.9): a table of every article grouped by role with claim/source sums, an Orphans view filtered to `backlinks == 0`, and a claim-atom view by tier. No Dataview plugin required.
- **`_MOC/graph.canvas`** — a JSON Canvas 1.0 map of the ontology folders and their top pages, edges drawn from declared embeds.
- **`.obsidian/`** — graph color groups per ontology folder, so the vault opens looking organized rather than gray.
- **Per-article folders** — README with the full body (links converted), a `Linked from` backlinks section, YAML properties typed to Obsidian's conventions (ISO dates, list tags, aliases), plus the existing claims, hash-chained sources, voxel topology, provenance, question graph, and yield files.

The authority rule is stated in the vault itself and enforced by the sync design: **the site is canonical; the vault is a lossless projection.** Local edits do not overwrite canon — the sync script posts annotations as *challenges* against specific claims on the live ledger, and the next pull shows the graph revised. Karpathy's separation of immutable sources from generated synthesis was already this system's architecture (hash-chained sources, append-only revisions); the vault just makes it visible in a folder.

## The doctrine is now a law object

The loop itself — how a model picks the next act, what form each artifact takes, which grammars are legal in a body, what a hero image may and may not be, how outreach is gated, what a finished rep looks like — is now written as a canonical law object at [The Loop Law](/a/loop-law), alongside the writing, design, outreach, and skill laws in the site footer. It serves a human page, a JSON object, and a model skill (`/api/articles/loop-law?format=skill`) from one source file.

Its repair clause is the reason it exists: **the documentation is the fix surface.** When a model produces a wrong output, the owner points at the behavior; the clause that allowed it is amended with the exhibit attached; the instance is fixed second. Every model that loads the object afterward inherits the fix. The object's amendment history already carries the failures that taught it: the render that silently replaced an authored body with a claims digest (2026-08-02), the art-styled hero images (2026-08-01), the 121 outreach drafts that converged on one template (2026-07-25), the unsigned post (2026-07-24).

## What is not satisfied

- **1,715 orphans is a real debt, not a bug in the counter.** Most are peptide and OIP articles that were published connected by folder convention but never linked by any parent page. The queue now surfaces them, but clearing them is weeks of `connect` acts.
- **The missing-pages queue is currently dominated by one corpus.** The OIP convergence series accounts for most of the 163 missing targets; the ranking is honest about demand but not yet about diversity of demand.
- **Stale detection is young.** Zero stale hubs at the 120-day threshold mostly reflects a corpus younger than 120 days, not perfect maintenance.
- **The outreach acts are derived, not executed.** External sends remain owner-gated by standing law; the queue can rank a quiet audience class, and a model may draft for it, but nothing sends itself.
- **Obsidian's official CLI requires the desktop app running.** Headless automation still goes through this site's own API; the vault's `SCHEMA.md` documents both paths.
- **The vault is one-way for structure.** Sync posts claims and challenges; it does not yet accept new pages authored in the vault. A page authored locally still enters canon through the article intake.

## Run it yourself

```bash
# the queue
curl -s https://miscsubjects.com/api/articles/next-acts?limit=10

# the health report
curl -s https://miscsubjects.com/api/articles/graph-lint

# backlinks for any article
curl -s 'https://miscsubjects.com/api/articles/graph-links?slug=bpc-157'

# the vault (JSON manifest of files + hashes)
curl -s 'https://miscsubjects.com/api/articles/obsidian-vault?slugs=protocol,bpc-157'
```

Then open the pulled folder in Obsidian and start at `index.md`. The graph you see is the same graph the queue is computed from — the site, the vault, the lint, and the queue are one derivation, read four ways.


## Every proposed addition, classified — KEEP, CHANGE, DELETE, BUILD

The 2026-08-03 merge of this corpus with the second-brain pattern (the Karpathy LLM-wiki concept and the Obsidian toolchain) proposed a set of additions. The logic law requires each to land on a terminal verdict with the component it attaches to and its measurable advantage — not a wishlist. This is that record.

| Verdict | Item | Component it attaches to | Measurable advantage |
|---|---|---|---|
| KEEP | `[[wikilink]]` round-trip grammar | article renderer + vault export | a link typed in prose to a page that does not exist becomes a ranked work order; the queue below is computed from exactly these |
| KEEP | Obsidian vault export | `GET /api/articles/obsidian-vault` | the whole corpus pulls as a hash-verified vault (370 files, 369/369 verified on the conformance run); any editor, any model, zero lock-in |
| KEEP | One-derivation loop surfaces | `graph-links` · `graph-lint` · `next-acts` | subjects are read off the graph instead of invented; the queue re-derives after every publish |
| KEEP | Stale-write protection | articles API (`body_hash` → `expected_hash`) | two agents can no longer silently overwrite each other; a moved head returns 409 with the current hash |
| KEEP | Loop law as a loadable object | [[loop-law]] (`?format=skill`) | a fresh agent with no context ran a full loop rep from the documentation alone (the misc conformance run, 2026-08-03) |
| CHANGE | Law clauses frozen in code | six `*_law_object.js` files, ~178 clauses | convert to `laws` rows: an owner correction becomes a row edit, not a deploy — the LOGIC_OVER_CODE law (2026-08-03) names doctrine-in-code a standing conversion debt |
| CHANGE | The governing constitution in a JS literal | `DECISION_CONSTITUTION` directory row | the row exists but points back at code; moving the text into the row makes version lineage rows, not git comments |
| CHANGE | Writer prompts as string constants | 4 prompts in the prose pipeline | prompts are directory rows by law (MODEL_CALL_LAW); these evade the inventory gate because they open "You write" instead of "You are" |
| DELETE | The draft-approval email round | outreach lane | replaced 2026-08-03 by owner order: letters in the settled format send directly, tracked, CC to the owner — the approval email itself had become the spam |
| BUILD | Local vault search | the pulled vault | sub-second full-text recall over the corpus without an API round-trip; the missing half of the second-brain pattern |
| BUILD | Web Clipper ingest | intake lane | external sources enter as claims with provenance instead of dying in bookmarks |
| BUILD | Vault-authored pages entering canon | obsidian sync lane | today the vault is read-only outbound; authored-in-Obsidian pages should arrive as drafts through the same claims gate |
| BUILT | The autonomy tick | automation 20 (`AGENT_SPAWN_CLI`, every 720 min) | the loop now fires its own reps: one rep per tick under [[loop-law]], sends tracked and CC'd, receipts ledgered — scheduled 2026-08-03 |

Verdicts follow the [[logic-law|logic law]]'s terminal states. Every CHANGE row is also filed on the amendment lane of the page it concerns, so the conversion debt is queue work, not a note.


## Sources

1. Live derivation receipts, 2026-08-03: graph-lint counts {articles:2264, edges:3386, orphans:1715, missing_pages:163, unsourced:494, contested:32}; next-acts head [write oip-total-structure 271, write oip-machine-plane 110, write oip-ground 103] at 2026-08-03T02:48:38Z; vault pull 370 files, SHA256SUMS verify 369 ok 0 fail — https://miscsubjects.com/api/articles/graph-lint
2. Karpathy — LLM Wiki (gist, 2026) — https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
3. Obsidian Bases syntax (official documentation) — https://obsidian.md/help/bases/syntax
4. JSON Canvas spec 1.0 — https://jsoncanvas.org/spec/1.0/
5. Obsidian CLI (official documentation) — https://obsidian.md/cli
6. Vannevar Bush — As We May Think, The Atlantic, July 1945 — https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/
7. kepano/obsidian-skills — Obsidian's agent-skill reference for markdown, Bases, JSON Canvas, and the CLI — https://github.com/kepano/obsidian-skills
8. Obsidian URI scheme (official documentation) — https://obsidian.md/help/uri