The LLM Wiki: How to Build an AI Knowledge Base That Never Forgets

📋 Overview

• Type: Lecture / Tutorial (with a live build-along demo)
• Main Topic: How to build a persistent, self-organizing "LLM Wiki" — a concept proposed by Andrej Karpathy — using Obsidian and an AI coding agent, to replace the limitations of traditional document Q&A (RAG).
• Speakers: Jamie (host of the "Teachers Tech" YouTube channel).

🎯 Core Purpose & Context

• The video addresses a specific, widely-experienced frustration: AI tools like ChatGPT and NotebookLM treat every question as a fresh start, never accumulating or building on prior work.
• The goal is educational and practical: explain why the standard approach is broken, introduce Karpathy's alternative concept, and then walk viewers step-by-step through building their own working LLM Wiki (demonstrated via a Japan trip-planning example).
• Target audience: non-technical users. Jamie explicitly states: "If you can create a folder on your computer, you can do this."

---

🧠 Key Concepts & Steps

Core Concepts

Figure 1 — RAG resets on every query; the LLM Wiki accumulates and compounds knowledge with each new source added.

1. The Problem with RAG (Retrieval Augmented Generation)

• RAG = you upload files, ask a question, the AI searches, grabs relevant chunks, and generates an answer.
• The flaw: "Ask a similar question tomorrow and the AI does all of that work again from scratch. Nothing was saved, nothing was built up. Every single question starts from zero."
• Analogy given: A researcher reading papers for weeks — but with RAG, "every time you ask the AI a question, it's like it's never read any of those papers before."
• The bottleneck: no memory, no accumulation, nothing compounds.

2. The LLM Wiki (Karpathy's Solution)

• Instead of searching raw documents every time, the AI reads your documents once and builds a structured, persistent Wiki of interlinked markdown files.
• When you add a new source, the AI: reads it, extracts key ideas, integrates them, updates existing pages, creates new pages, links related ideas, and flags contradictions.
• Over time the Wiki "keeps growing and getting richer" — synthesis and connections are pre-built before you even ask.
• Karpathy's framing: "Think of Obsidian as the IDE, the LLM as the programmer, and the Wiki as the code base." You rarely write the Wiki yourself — you focus on what goes in and what questions to ask.

3. The Three-Layer System

• Layer 1 — Raw Sources: Original documents (PDFs, articles, notes). Read-only — the AI reads but never changes them. This is your "source of truth."
• Layer 2 — The Wiki: A folder of AI-created/maintained markdown files (index page, concept pages, entity pages, summaries, comparisons — all interlinked).
• Layer 3 — The Schema: A rules document telling the AI how to structure the Wiki, handle new sources, and format everything. In Claude Code, this is the claude.md file.

Important Distinctions

• RAG vs. LLM Wiki: RAG re-searches raw files every time (ephemeral); the Wiki builds a persistent, compounding knowledge base (permanent, cumulative).
• Obsidian vs. any text editor: Obsidian is chosen for its graph view (visual connections), but the system is fundamentally just a folder of markdown files — VS Code or any editor works.
• The AI is the engine, not Obsidian: Obsidian alone does nothing; you need a coding agent (Claude Code, Codex, Cursor) to read/write the files.

Figure 2 — The three-layer system: raw sources stay untouched, the Wiki grows automatically, and the schema defines all the rules the AI follows.

---

🛠️ Step-by-Step Build Guide

Prerequisites

1. Obsidian — free markdown note-taking app (from obsidian.md), used as the viewer.
2. An AI coding agent — Jamie uses Claude Code; alternatives: OpenAI Codex, Cursor, or similar tools that can read/write local files.

The Build Process

1. [00:04:40] Open Obsidian → create a new Vault (a "fancy name for folder") → named LLM Wiki, saved in Documents.
2. [00:05:10] Create three folders inside:
   • raw — AI reads from here, never edits.
   • wiki — where the AI builds and maintains pages.
   • templates — optional (only needed for manual note formatting; not used here since Claude generates everything).
3. [00:06:10] Create the schema file claude.md in the root of the vault. Claude Code reads this automatically on opening the project. (Jamie provides a starter template in the description.)
4. [00:06:40] Customize the claude.md. The five components:
   • Purpose: The one line you MUST customize (demo = "planning a trip to Japan").
   • Folder structure: Where raw sources and Wiki output live.
   • Ingest workflow: read doc → extract concepts → create/update Wiki pages → update index → log changes.
   • Page formatting rules: summary at top of every page, every claim references its source, pages link to related concepts.
   • Question-answering behavior: consult Wiki first, cite sources, flag uncertainty.
5. [00:08:10] Install the Obsidian Web Clipper Chrome extension (free) — converts web articles into markdown files.
6. [00:08:40] Add first source: a Tokyo travel blog post → saved as markdown → dragged into the raw folder. (Note: PDFs and text files can be dropped in directly — Claude reads PDFs natively.)
7. [00:09:40] Point Claude Code to the correct directory (the vault location), then open Claude.
8. [00:10:20] Command: "I just added a new source to the raw folder. Please read it and update the wiki." → Claude reads, summarizes, and proposes Wiki pages (e.g., neighborhood pages). You can review scope before approving. Build took ~3 minutes.
9. [00:10:39] Verify in Obsidian: structured, interlinked pages appear. Graph view visually shows the connections forming from a single document.
10. [00:11:21] Add second source (a Japan food guide) → same ingest command → Claude updates existing neighborhood pages AND creates new ones. Graph view gains more nodes and connections.
11. [00:12:18] Ask a cross-source question: "What neighborhood should I stay at if I want to be close to the best food and still near the major temples?" → Claude answers by pulling from Wiki pages (neighborhoods, food, temples) and citing specific Wiki pages, not raw articles.
12. [00:13:19] Lint the Wiki: Command "Please lint the wiki." → Claude checks for contradictions, outdated claims, orphaned pages (no incoming links), and mentioned-but-missing concepts, then returns a health report.

Figure 3 — The compounding advantage: each source ingested enriches the entire Wiki, while traditional AI usage produces no lasting intellectual capital.

---

🧭 Strategic Analysis & "Game Changers"

• Hidden Connections: The video quietly reframes AI from a "search tool" into a "knowledge worker/librarian." The mental shift — treating the AI as a programmer maintaining a codebase — implies your intellectual output becomes a version-controlled, inspectable asset rather than an ephemeral chat log. This is a fundamentally different relationship with AI.

• The "So What?": The compounding effect is the real value. Traditional AI usage is a treadmill — effort doesn't accumulate. The LLM Wiki turns every hour of reading/ingesting into a permanent, self-improving asset that appreciates over time. For knowledge workers, students, and researchers, this converts scattered effort into durable capital.

Figure 4 — Know the limits: the LLM Wiki excels at personal knowledge management but requires curation, an AI agent, and periodic linting to stay healthy.

• Game Changer: The persistence + synthesis-at-ingest model. By doing the "hard work" of connecting ideas when a source is added (rather than at query time), the system front-loads synthesis. The single most valuable shift is: stop asking the AI to think from scratch, and instead have it maintain a living structure that already contains the thinking. Bonus underrated feature: contradiction flagging — the Wiki doesn't just store, it actively reconciles conflicting information across sources, something RAG cannot do.

• Data ownership angle: Everything stays local, in plain-text markdown files that you own — a strategic advantage for privacy and portability that Jamie explicitly highlights as a selling point over cloud tools.

---

⚖️ Limitations & Honest Caveats (from [00:15:14])

Limitation	Detail
Scale	Best at personal scale (~100 articles per Karpathy). Tens of thousands of pages need more infrastructure than markdown files.
Garbage in, garbage out	Wiki quality depends entirely on curated sources; you must vet what goes in.
Requires a coding agent	Obsidian alone does nothing — you need Claude Code, Codex, or similar.
AI can make mistakes	May miscategorize or miss connections; hence the lint feature and human review (especially early on).

📊 Detailed Breakdown

• [00:00:00] Hook: the "unseeable" problem with how most people use AI — every question starts from zero.
• [00:00:30] Introduction of Andrej Karpathy (OpenAI co-founder, former Tesla AI Director) and his "LLM Wiki" idea.
• [00:01:10] Jamie introduces himself and Teachers Tech; deep dive into RAG and its lack of memory.
• [00:02:10] The Wiki solution explained — read once, build a persistent interlinked knowledge base.
• [00:02:40] Karpathy's IDE/programmer/codebase analogy.
• [00:03:10] The three-layer architecture (Raw / Wiki / Schema) laid out.
• [00:04:10] Setup requirements — Obsidian + AI coding agent (Claude Code recommended).
• [00:04:40]–[00:05:40] Creating the vault and the three folders.
• [00:06:10]–[00:07:40] The claude.md schema file and its five components explained; purpose line = the one customization needed.
• [00:08:10] Installing the Obsidian Web Clipper.
• [00:08:40]–[00:09:10] Ingesting the first source (Tokyo travel blog).
• [00:09:40]–[00:10:39] Running the first ingest command; verifying structured pages and graph view.
• [00:11:21]–[00:11:54] Ingesting a second source (food guide); Claude updates existing pages + graph grows.
• [00:12:18]–[00:12:43] Cross-source question answered with citations from the Wiki.
• [00:13:19]–[00:14:02] The "lint the wiki" concept and demo report (orphan pages, broken links, un-ingested gaps).
• [00:14:02]–[00:15:14] Use cases: students/researchers, teachers, businesses, curious readers.
• [00:15:14]–[00:16:21] Limitations laid out honestly, followed by a strong endorsement.
• [00:16:21] Wrap-up: schema/template links in description; plug for beginner Claude Code guide.

🔑 Key Takeaways

1. Traditional AI Q&A (RAG) has no memory — effort never compounds; every query restarts from zero.
2. The LLM Wiki flips this: the AI reads sources once and maintains a persistent, interlinked markdown knowledge base that gets richer with each addition.
3. Three simple layers — read-only Raw sources, the AI-maintained Wiki, and a Schema (claude.md) that defines the rules.
4. The "lint" feature keeps the Wiki healthy by catching contradictions, orphaned pages, broken links, and missing concept pages.
5. It's free, local, and non-technical to set up — you own your data in plain-text files, and the workflow applies to any domain where knowledge accumulates over time.

❓ Unresolved Questions / Follow-up

• How does the system scale beyond ~100 articles? Jamie notes it breaks down but doesn't specify what infrastructure to add.
• How often should you lint? "Periodically" is vague — no concrete cadence given.
• How do you handle version conflicts or roll back a bad ingest? The AI editing pages en masse (during lint) carries risk, unaddressed here.
• What's the exact content of the claude.md template? Only summarized verbally — full details are in the (external) description link.
• Cost/token implications of re-reading and re-writing the whole Wiki during ingests and lints are not discussed.
• Multi-user / collaboration: Business use case is mentioned, but syncing a shared Wiki across a team isn't explained.