{"slug":"oip-what-is-pagination","title":"What Is Pagination?","body":"## What It Is\n\n**Pagination is the deterministic slicing of a large, ordered dataset into bounded, addressable windows.** Every window has a boundary. Every boundary has a name. That name — a cursor, an offset, a page number — lets any system request exactly the same slice twice and get the same result.\n\n## Why It Matters\n\nInfinite scroll feels good. It also destroys reproducibility.\n\nWhen a dataset has no boundaries, you cannot reference it. You cannot audit it. You cannot prove what you saw. Pagination draws lines. Those lines are contract surfaces. They turn a streaming river into a numbered archive.\n\nThe philosophical weight is this: **deterministic access is the foundation of trust.** If two observers cannot request the same slice and agree on its contents, you have opinion, not protocol. Pagination makes sequence explicit. It makes consensus possible.\n\nPractically, it protects systems. It caps memory. It bounds latency. It turns \"load everything\" into \"load exactly this.\" That transformation is the difference between a toy and infrastructure.\n\n## How It Works\n\nAt its core, pagination is a contract between a caller and a dataset. The caller asks for a window. The dataset returns the window plus a pointer to the next one.\n\nStep 1: The caller chooses a strategy.\n- **Offset/limit:** Start at position N, return M items. Simple. Dangerous on changing datasets.\n- **Cursor-based:** A opaque token marks the boundary. The dataset decodes it. Stable. Scalable.\n- **Keyset:** The boundary is a value in the ordering column. Fast. Requires an ordered index.\n\nStep 2: The caller sends a request.\n\n```\nGET /items?page[cursor]=abc123&page[size]=50\n```\n\nStep 3: The dataset evaluates the boundary.\n- Cursor \"abc123\" decodes to: last seen ID = 7,491, timestamp = 1698000000.\n- Query becomes: WHERE id > 7491 AND created_at >= 1698000000 ORDER BY created_at, id LIMIT 50.\n\nStep 4: The dataset returns the window plus the next pointer.\n\n```json\n{\n  \"data\": [ /* 50 items */ ],\n  \"links\": {\n    \"next\": \"/items?page[cursor]=def456\"\n  }\n}\n```\n\nStep 5: The caller decides. Stop? Or follow the next pointer? The dataset does not decide. The caller does. That separation of concerns is clean.\n\n## The Contract\n\n**The interface:**\n\n- Input: `limit` (max items per window, bounded by a global cap), `cursor` (opaque token, optional; omitted means \"first window\").\n- Output: `data` (array of items, length ≤ limit), `next_cursor` (opaque token, null if no further data).\n\n**The invariants:**\n\n1. **Determinism:** The same cursor, requested twice, returns the same ordered sequence.\n2. **Exclusivity:** No item appears in two windows for the same cursor sequence.\n3. **Completeness:** Every item in the ordered dataset appears in exactly one window, or is newly added after the cursor was minted.\n4. **Boundedness:** `limit` is always ≤ the global cap. The global cap is a hard ceiling.\n5. **Opacity:** The caller does not decode the cursor. The dataset encodes and decodes it. The cursor is a capability, not a coordinate.\n\n**The failure modes:**\n\n- Cursor invalid: 400. The token is corrupt or expired.\n- Limit exceeded: 400. The caller asked for more than the global cap.\n- Dataset empty: 200, data = [], next_cursor = null. Not an error. A boundary.\n\n## Real Examples\n\n**GitHub API commits:**\nGitHub returns 30 commits per page. The `Link` header contains `rel=\"next\"` with a URL carrying `page=2`, `page=3`. The cursor is the page number. The ordering is implicit: reverse chronological. The contract is simple because the dataset is append-only at the top.\n\n**Stripe API charges:**\nStripe uses cursor-based pagination. The `starting_after` parameter is an object ID. The response includes `has_more`. The ordering is creation time, stable because IDs are KSUIDs. The contract is strong: no charge ever changes position.\n\n**PostgreSQL keyset with LIMIT/OFFSET:**\nA query like `SELECT * FROM events WHERE id > $cursor ORDER BY id LIMIT 50` is keyset pagination at the database layer. The cursor is the last `id` of the previous batch. The database uses the primary key index to seek directly. No full table scan. No memory bloat.\n\n**Twitter/X timeline (historical):**\nThe timeline API once exposed `max_id` and `since_id`. These were tweet IDs, which are Snowflake timestamps in disguise. The cursor encoded both position and time. Two boundaries, one token. Elegant.\n\n**OIP ledger events:**\nThe OIP ledger appends events in strict order. Each event has a monotonic sequence number. Pagination requests `?after_seq=8471&limit=100`. The dataset returns events 8472-8571. The cursor is `8571`. The next request is `?after_seq=8571&limit=100`. Every auditor, every replica, every observer can request the exact same window. That is the point.\n\n## Common Mistakes\n\n**Using offset/limit on mutable datasets:**\nOffset says \"skip N rows.\" If a row is inserted at position 5, every offset shifts. The same request, run twice, returns different items. You have lost determinism. You have broken the contract.\n\n**Letting the caller set limit without a cap:**\nA caller asks for `limit=1000000`. The system allocates a million rows in memory. The database locks. The node dies. The cap is not a suggestion. It is a guardrail.\n\n**Returning the cursor as a raw database ID:**\nThe caller starts guessing IDs. They iterate backward. They probe gaps. The cursor is a capability. It should be opaque, signed, or encoded. Treat it like a session token.\n\n**Omitting `has_more` or `next_cursor` when the dataset is empty:**\nAn empty page is not an error. It is a valid boundary. The absence of a next cursor is the signal. Returning 404 or 204 turns a clean contract into an edge-case nightmare.\n\n**Paginating without a total order:**\nIf the dataset has no deterministic sort, page 2 is fiction. The database returns \"some 50 rows.\" Which 50? Undefined. Every page request is a dice roll.\n\n## Connection to OIP\n\nOIP is built on three principles: **open, deterministic, auditable.** Pagination is the mechanical expression of all three.\n\n**Open:** A paginated endpoint is a public contract. Any client can walk it. No hidden state. No \"you had to be there.\" The dataset is inspectable one window at a time.\n\n**Deterministic:** The cursor is a commitment. It binds a specific query to a specific result set. Two honest nodes, given the same cursor, agree. That is consensus material. That is what makes a protocol a protocol instead of a service.\n\n**Auditable:** An auditor does not need the full dataset. They request page 1, then page 2, then page 3. Each page is small, verifiable, and self-contained. The auditor hashes the page. They compare hashes. If the dataset drifts, the hash changes. Pagination turns audit from a memory-intensive nightmare into a bounded, parallelizable walk.\n\nIn OIP, pagination is not a convenience feature. It is a structural requirement. Without it, the ledger is a stream. With it, the ledger is an archive. And archives are what civilizations build on.\n\n## Connection to the Grain Philosophy\n\nThis protocol is part of the [Open Inventory Protocol](/a/philosophy) — a living system of self-describing voxels that serves the Grain philosophy. The OIP is the interface. The philosophy is the core.\n","hero":null,"images":[],"style":{},"tags":["oip","protocol"],"model":null,"ledger":null,"embeds":[],"widgets":[],"home":true,"claims":[],"sources":[],"reviews":[],"extra":{},"has_traversal":false,"register":"oip_protocol","status":"published","revisions":1,"contributions":[],"provenance":[],"energy":{"passes":0,"tokens_in":0,"tokens_out":0,"tokens_total":0,"cost_usd":0,"models":{},"head":"genesis"},"posted_at":"2026-07-04T18:30:34.380Z","created_at":"2026-07-04T18:30:34.380Z","updated_at":"2026-07-04T19:01:09.424Z","machine":{"shape":"article.machine/v1","slug":"oip-what-is-pagination","kind":"article","read":{"human":"https://miscsubjects.com/a/oip-what-is-pagination","json":"https://miscsubjects.com/api/articles/oip-what-is-pagination","bundle":"https://miscsubjects.com/api/articles/oip-what-is-pagination/bundle?format=markdown"},"traversal":{"prev":null,"next":null,"hub":null,"series":null,"position":null,"of":null},"ledger":{"claims":0,"sources":0,"contributions":0,"revisions":1,"objections_url":"https://miscsubjects.com/api/articles/oip-what-is-pagination/objections","thread_state_url":"https://miscsubjects.com/api/protocol/thread-state?target=oip-what-is-pagination","proof_rule":"An action is proven by its ledger receipt, never by a 200 or a description."},"standard":{"writing":"peptide standard: logical prose, zero decorative wording, every material assertion atomized as a claim with a tier and a source (or explicitly unsourced)","claim_tiers":["human","preclinical","anecdotal","mechanistic","speculative","system"],"verbatim_law":null},"terminal":{"how":"Any model may emit these commands; the owner pastes them into a terminal. $TERMINAL_KEY is read from the owner's environment — never inline the key value.","claim_append":"curl -s -X POST https://miscsubjects.com/api/protocol/claim -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"oip-what-is-pagination\",\"text\":\"<one atomized claim>\",\"tier\":\"<human|preclinical|anecdotal|mechanistic|speculative|system>\",\"source_ids\":[],\"who_claims\":\"<model>\",\"rationale\":\"<why material>\"}'","source_append":"curl -s -X POST https://miscsubjects.com/api/protocol/sources -H \"x-terminal-key: $TERMINAL_KEY\" -H 'content-type: application/json' -d '{\"slug\":\"oip-what-is-pagination\",\"sources\":[{\"type\":\"review\",\"url\":\"<url>\",\"title\":\"<title>\",\"quote\":\"<verbatim quote>\",\"summary\":\"<one line>\"}]}'","objection":"curl -s -X POST https://miscsubjects.com/api/articles/oip-what-is-pagination/objections -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"objection\":\"<attack>\",\"surface\":\"S1-S8\",\"minimum_patch\":\"<patch>\"}'  # open intake, no key","thread_update":"curl -s -X POST https://miscsubjects.com/api/protocol/thread-update -H 'content-type: application/json' -d '{\"actor\":\"<model>\",\"target\":\"oip-what-is-pagination\",\"raw_text\":\"<material delta>\"}'  # open intake, no key","read_back":"curl -s https://miscsubjects.com/api/articles/oip-what-is-pagination | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps(d[\"claims\"][-3:], indent=1))'"}}}