@zosmaai/pi-llm-wiki

Self-maintaining, Obsidian-compatible knowledge base for pi. Follows Andrej Karpathy's LLM Wiki pattern.

Turn raw sources (URLs, PDFs, markdown, JSON, XML) into a durable, interlinked, LLM-maintained wiki that compounds over time.

---

Quick Start

pi install npm:@zosmaai/pi-llm-wiki

The extension will proactively suggest creating a wiki on your first session. Alternatively:

/wiki-init "AI Engineering"
/wiki-ingest
/wiki-query What are the key patterns?

---

Why This Package?

Most file-based LLM workflows behave like one-shot RAG: the model searches raw documents every time you ask a question. Synthesis is ephemeral.

pi-llm-wiki creates a middle layer:

• Raw source packets preserve source-of-truth inputs
• Source pages summarize what each source says
• Canonical wiki pages track what the wiki currently believes
• Generated metadata keeps everything searchable and navigable

The result is a wiki that compounds as you capture sources, ask questions, and file durable analyses.

---

Features

Capability	Description
🔗 Immutable source capture	URLs, local files (PDF/md/txt/html/XML/JSON), or pasted text → structured source packets
🧠 Automated ingestion	wiki_ingest batch-processes sources into concept, entity, synthesis & analysis pages
🔍 Full-text search	Generated registry with keyword lookup across all pages and sources
🩺 Mechanical linting	Orphans, broken links, duplicate aliases, coverage gaps, stale captures
📊 Dashboard	wiki_status — counts, source states, recent activity
🤖 Auto-update watch	wiki_watch — schedule periodic discovery + ingest
🧠 Auto-recall	Wiki searched automatically before every turn — relevant pages injected into context
📝 Auto-bootstrap	Extension suggests creating a wiki when none exists in the current directory
💾 Auto-capture	wiki_retro — save atomic insights from completed tasks with one call
🌐 MCP Server	Use with Claude Code, Cursor, Windsurf via stdio MCP transport
📝 Obsidian-friendly	Folder-qualified wikilinks, stable source-ID citations, compatible vault
🛡️ Guardrails	Blocks direct edits to raw sources and generated metadata
🔧 Configurable PDF extraction	MarkItDown timeout via WIKI_MARKITDOWN_TIMEOUT_MS env var
🧪 38+ tests, CI, CodeQL	TypeScript, Vitest, Biome, Codecov

---

Tools

Tool	Description
wiki_bootstrap	Initialize a new wiki vault with config, templates, schema, and metadata
wiki_capture_source	Capture a URL, local file, or pasted text into an immutable source packet
wiki_recall	🔄 Auto-called at turn start — search wiki for task-relevant pages
wiki_retro	Save atomic insights from completed tasks into the wiki
wiki_ingest	Process uningested source packets into wiki pages (batch)
wiki_ensure_page	Resolve or safely create entity / concept / synthesis / analysis pages
wiki_search	Search the generated wiki registry
wiki_lint	Deterministic health checks (orphans, gaps, contradictions, auto-fix)
wiki_status	Show counts, source states, and recent activity
wiki_rebuild_meta	Force a full metadata rebuild (registry, backlinks, index, log)
wiki_log_event	Append a structured event to the wiki activity log
wiki_watch	Schedule automatic wiki updates (daily / weekly / hourly)

Slash Commands

Command	Description
/wiki-init <topic>	Initialize a new LLM Wiki vault
/wiki-ingest [path]	Process new source files and update the wiki
/wiki-query <question>	Ask questions against the wiki with citations
/wiki-discover [--topic <topic>]	Auto-discover new sources from the web
/wiki-run [--schedule daily|weekly]	Full cycle: discover → ingest → lint
/wiki-lint [--fix]	Health check (orphans, contradictions, gaps)
/wiki-status	Show a concise operational summary
/wiki-digest [--period daily|weekly]	Generate a digest of recent activity
/wiki-retro	Save atomic insights from completed tasks

---

Quick Start (Detailed)

1) Create a new wiki

mkdir my-wiki
cd my-wiki
pi

Ask pi:

Initialize an llm wiki here for AI research.

This calls wiki_bootstrap and creates:

.llm-wiki/
├── config.json
├── templates/
├── raw/
├── wiki/
├── meta/
└── WIKI_SCHEMA.md

2) Capture a source

Capture this article into the wiki: https://example.com/some-article

Capture this PDF into the wiki: ./papers/context-windows.pdf

Capture these notes into the wiki: ...pasted text...

3) Integrate the source

1. Capture the source
2. Read .llm-wiki/wiki/sources/SRC-*.md
3. Update that source page
4. Search for impacted canonical pages with wiki_search
5. Create missing pages with wiki_ensure_page
6. Update concept / entity / synthesis pages with citations
7. Mark the integration with wiki_log_event kind=integrate

4) Query the wiki

Based on the wiki, what are the main tradeoffs between long-context models and RAG?

By default, query mode is read-only. To file a durable answer:

Answer the question and file the result as an analysis page.

---

Vault Layout

my-wiki/
└─ .llm-wiki/
   ├─ config.json               # Vault config
   ├─ templates/                 # Page templates
   ├─ raw/
   │  └─ sources/
   │     └─ SRC-2026-05-11-001/
   │        ├─ manifest.json
   │        ├─ original/           # Original artifact
   │        ├─ extracted.md        # Normalized text
   │        └─ attachments/
   ├─ wiki/
   │  ├─ sources/                  # Source pages (what each source says)
   │  ├─ concepts/                 # Concepts and recurring ideas
   │  ├─ entities/                 # People, orgs, products, papers, systems
   │  ├─ syntheses/                # Cross-source theses and tensions
   │  └─ analyses/                 # Durable filed answers from queries
   ├─ meta/
   │  ├─ registry.json             # Auto-generated search index
   │  ├─ backlinks.json
   │  ├─ index.md
   │  ├─ events.jsonl              # Append-only event log
   │  ├─ log.md
   │  └─ lint-report.md
   └─ WIKI_SCHEMA.md               # Operating manual

Ownership Model

Path	Owner	Rule
Path	Owner	Rule
------	-------	------
.llm-wiki/raw/**	Extension tools	Immutable after capture
.llm-wiki/wiki/**	Model + user	Editable knowledge pages
.llm-wiki/meta/registry.json	Extension	Generated
.llm-wiki/meta/backlinks.json	Extension	Generated
.llm-wiki/meta/index.md	Extension	Generated
.llm-wiki/meta/events.jsonl	Extension / tool	Append-only
.llm-wiki/meta/log.md	Extension	Generated from events
.llm-wiki/meta/lint-report.md	Extension	Generated
.llm-wiki/WIKI_SCHEMA.md	Human + explicit request	Operating manual

---

Linking & Citation Style

Internal Navigation

[[concepts/retrieval-augmented-generation]]
[[entities/openai|OpenAI]]
[[syntheses/long-context-vs-rag]]

Factual Citations

[[sources/SRC-2026-04-04-001|SRC-2026-04-04-001]]

Stable source-page IDs keep provenance stable even if titles change.

---

Guardrails

The extension blocks direct tool-call edits to:

• .llm-wiki/raw/** — immutable source artifacts
• .llm-wiki/meta/registry.json
• .llm-wiki/meta/backlinks.json
• .llm-wiki/meta/events.jsonl
• .llm-wiki/meta/index.md
• .llm-wiki/meta/log.md
• .llm-wiki/meta/lint-report.md

If the model directly edits .llm-wiki/wiki/** using Pi's built-in write or edit tools, the extension automatically rebuilds generated metadata at the end of the agent turn.

---

Source Packet Format

Each captured source is stored as a structured packet:

.llm-wiki/raw/sources/SRC-YYYY-MM-DD-NNN/
├─ manifest.json     # Capture metadata (title, URL, format, timestamp)
├─ original/         # Original artifact (preserved as-is)
├─ extracted.md      # Normalized text (PDF→md, XML→md, JSON→md, etc.)
└─ attachments/      # Future attachment downloads

This preserves both the original artifact and a normalized extracted view for reading.

---

MCP Server

Use the wiki from any MCP-compatible tool — Claude Code, Cursor, Windsurf, and others.

The package ships a standalone MCP server exposing 5 wiki tools over stdio:

Tool	Description
wiki_recall	Search wiki for task-relevant pages
wiki_search	Full registry search
wiki_status	Wiki stats (page counts, type breakdown)
wiki_retro	Save atomic insights
wiki_capture_source	Capture text as a source packet

Usage

# Auto-discovered by pi:
pi install npm:@zosmaai/pi-llm-wiki

# Standalone with any MCP client:
WIKI_ROOT=~/my-wiki node node_modules/@zosmaai/pi-llm-wiki/mcp/index.js

Set WIKI_ROOT to your wiki vault directory. If unset, the server auto-detects from the current working directory.

---

Skill Behavior

The bundled llm-wiki skill teaches the model to:

• ❌ Never edit raw sources directly
• ❌ Never edit generated metadata files
• ✅ Capture first, integrate second
• ✅ Search before creating new canonical pages
• ✅ Cite facts using source-page IDs
• ✅ Keep query mode read-only by default
• ✅ Use "Tensions / caveats" and "Open questions" when evidence is mixed

---

Architecture

Four layers with clear ownership:

.llm-wiki/raw/sources/SRC-*/     # Immutable source packets (extension-owned)
.llm-wiki/wiki/                   # Editable knowledge pages (you + LLM)
.llm-wiki/meta/                   # Auto-generated registry, backlinks, index, log
.llm-wiki/                        # Config and templates

Read docs/architecture.md for the full design document.

---

Documentation

Document	What it covers
Architecture	How the four layers work, ownership model
Commands	All slash commands and tool reference
Obsidian Integration	Vault setup and recommended plugins
Configuration	Wiki modes, topics, environment variables
API	Extension tool parameter reference

---

Contributing

See CONTRIBUTING.md for development setup, test patterns, and PR workflow.

---

Star History

Contributors

---

License

MIT