How Andrej Karpathy’s Thought Is Altering AI

Take into consideration revisiting objects you’ve saved to Pocket, Notion or your bookmarks. Most individuals don’t have the time to re-read all of these items after they’ve saved them to those numerous apps, except they’ve a necessity. We’re glorious at amassing tons of knowledge. Nonetheless, we’re simply not superb at making any of these locations work together with one another or add a cumulative layer that connects them collectively. 

In April of 2026, Andrej Karpathy (former AI Director of Tesla and co-founder of OpenAI) prompt an answer to this challenge: use a big language mannequin (LLM) to construct your wiki in real-time.

This concept grew to become viral and was finally adopted with a GitHub gist describing how to do that utilizing an LLM. This information will present all the help (with instance code) for constructing your individual residing, evolving private wiki. 

The RAG Drawback: Rediscovering Information from Scratch, Each Single Time

Numerous trendy AI data instruments use Retrieval-Augmented Era (RAG). In easy phrases, you add paperwork, ask a query, and the system retrieves related textual content to assist the AI generate a solution. Instruments like NotebookLM, ChatGPT (file uploads), and most enterprise AI methods observe this method.

On the floor, this is smart. However as Andrej Karpathy factors out, there’s a key flaw: the mannequin doesn’t accumulate data. Every question begins from scratch.

If a query requires synthesizing 5 paperwork, RAG pulls and combines them for that one response. Ask the identical query once more tomorrow, and it repeats the complete course of. It additionally struggles to attach data throughout time, like linking an article from March with one from October.

Briefly, RAG produces solutions, but it surely doesn’t construct lasting data.

Wiki Answer: Compile Information As soon as, Question Ceaselessly

Karpathy’s method will change the way in which we take a look at fashions. Somewhat than processing uncooked paperwork after our queries, we are going to now course of these paperwork on the time of ingestion. The outcomes of this processing might be a everlasting, structured wiki-like product (which can permit you to retailer and retrieve paperwork with a excessive diploma of management over their location). 

While you add a brand new doc supply to the LLM, the LLM doesn’t merely create an index of that supply for later retrieval. As a substitute, the LLM reads, understands, and integrates that supply into the data base, updating all related present pages (the place obligatory). It notes down any contradictions between the brand new and present claims or data, creating any obligatory new idea pages, and reinforcing the advanced relationships throughout the complete wiki. 

In line with Karpathy, “With LLMs, data is created and maintained repeatedly and constantly somewhat than with the ability to use particular person queries to create data.” Right here is an easy comparability that illustrates this distinction additional. 

How It Truly Works: A Step-by-Step Information

Let’s evaluate how a person would develop considered one of these wikis. 

Step 1: Acquire your sources

That you must accumulate the whole lot – articles that you’ve got saved, books loved, notes you’ve gotten created, transcripts from discussions, and even your very personal historic conversations. All these supplies are your uncooked supplies, simply as ore should bear refining earlier than use. 

Top-of-the-line practices from this neighborhood is to not deal with all paperwork in the identical vogue. For instance, a 50-page analysis white paper requires extraction on a section-by-section foundation whereas a tweet or social media thread solely requires a main perception and corresponding context. Likewise, a gathering transcript requires extraction of selections that have been made, motion objects which are to be carried out and key quotations. By first classifying the kind of doc will assist extract the suitable kind of knowledge to the correct quantity of element. 

Step 3: The AI writes wiki pages (Question)

You’ll feed your supply supplies into your AI’s LLM by way of a structured question. It can permit the LLM to supply a number of wiki pages that conform to the established template of getting: a frontmatter block (YAML), a TLDR sentence, the physique of the content material, and the counterarguments/knowledge gaps. 

Step 4: Create an index

A central index.md will function a desk of contents, and hyperlink straight to every web page of the wiki. That is how an AI agent can effectively traverse the complete data base; it begins on the index, reads by the tldr’s, then drills down into solely these pages which are related to its particular query. 

Step 5: Document your questions

This is without doubt one of the most under-appreciated options of the system. While you ask the LLM a well-formed query and obtain a response that gives useful perception. For instance, a comparability between two frameworks, or a proof of how two ideas are associated, you save that response as a brand new wiki web page tagged with the label query-result. As time goes on, your finest pondering has been collected somewhat than misplaced in chat logs.

Step 6: Conduct lint passes

At applicable intervals, you ask the LLM to audit the complete wiki for contradictions or inconsistencies between pages, and to point these statements which have been rendered out of date by a newer supply. Moreover, the LLM will present enter on figuring out orphan pages (i.e., pages that don’t have any hyperlinks pointing to them), and for offering a listing of ideas which are referenced inside the present content material however usually are not but represented by their very own respective pages.

Karpathy talks about numerous particular instruments in his Gist. Beneath you will discover what every software does and the way they match into his general workflow. 

1. Obsidian – Your Wiki IDE

Obsidian is a free markdown data administration software which makes use of a neighborhood listing as a vault. For Karpathy, that is the viewing interface used for the wiki as a result of it has three distinct options that matter for his system: 

1.  The Graph View offers a graph of all wiki pages represented as nodes, and as well as, each wiki hyperlink ( [[wiki-links]] ) might be represented as edges connecting all nodes collectively. Hub pages might be related to many nodes, and so might be represented as bigger than common nodes. Orphan pages might be represented as remoted nodes. This enables for fast visible illustration of the density of data and gaps inside an individual’s data. No different doc view or file browser can present this illustration visually. 
2. The Dataview Plugin permits customers to show their wiki right into a database that may be queried. All pages will need to have yaml frontmatter, so the combination specification is glad and subsequently permits the person to run SQL-like queries in opposition to all pages within the wiki. 

# In any Obsidian be aware, Dataview renders this dynamically:

# Record all idea pages ordered by variety of sources
TABLE size(sources) AS "Supply Depend", confidence, up to date
FROM "wiki/ideas"
SORT size(sources) DESC


# Discover low-confidence pages that want evaluate
TABLE title, sources
FROM "wiki"
WHERE confidence = "low"
SORT file.mtime ASC


# Discover pages not up to date within the final 2 weeks
LIST
FROM "wiki"
WHERE up to date 

3. The Internet Clipper browser extension (accessible for Chrome, Firefox, Safari, Edge, Arc, and Courageous) converts net articles to scrub Markdown with YAML frontmatter in a single click on, saving on to your uncooked folder. You obtain all photographs to your laptop by urgent the hotkey Ctrl+Shift+D after you end clipping as a result of the LLM requires entry to the photographs. 

2. Qmd: Search at Scale

The LLM can use the index.md file to entry the wiki content material with out issues at small scale. The index turns into unreadable in a single context window when you’ve gotten greater than 100 pages as a result of it reaches extreme dimension. 

The native search engine qmd allows Markdown file searches by three search strategies which Tobi Lutke (CEO of Shopify) developed. The system operates completely in your gadget as a result of it makes use of node-llama-cpp with GGUF fashions which require no API connections and defend your knowledge from leaving your laptop. 

# Set up qmd globally
npm set up -g @tobilu/qmd 

# Register your wiki as a searchable assortment
qmd assortment add ./wiki --name my-research 

# Primary key phrase search (BM25)
qmd search "combination of consultants routing effectivity" 

# Semantic search, finds associated ideas even with completely different phrases
qmd vsearch "how do sparse fashions deal with load balancing" 

# Hybrid search with LLM re-ranking, highest high quality outcomes
qmd question "what are the tradeoffs between top-k and expert-choice routing" 

# JSON output for piping into agent workflows
qmd question "scaling legal guidelines" --json | jq '.outcomes[].title' 

# Expose qmd as an MCP server so Claude Code can use it as a local software
qmd mcp

The MCP server mode allows Claude Code to make use of qmd straight as a built-in software which ends up in smoother workflow integration all through your knowledge ingestion and question processing duties. 

3. Git: Model Management for Information

As a result of your total wiki is a folder of plain Markdown information, you get model historical past branching and collaboration at no cost with Git. That is fairly highly effective: 

# Initialize the repo while you begin
cd my-research && git init 

# Commit after each ingest session
git add .
git commit -m "ingest: MoE effectivity article — flags dense-vs-sparse contradiction" 

# See precisely what modified in any ingest
git diff HEAD~1 

# Roll again a nasty compilation go
git revert abc1234 

# See how your data developed over time
git log --oneline wiki/ideas/mixture-of-experts.md 

# Share with a group by way of GitHub (the wiki turns into collaborative)
git distant add origin https://github.com/yourname/research-wiki
git push -u origin major

Getting Began: Your First LLM Wiki in Three Steps

If you happen to’re enthusiastic about this idea there’s a straightforward option to start: 

1. Choose one space of curiosity you’re at present exploring and provides the AI 5-10 of your finest sources. Don’t try to put the whole lot you’ve completed digitally into one place on the primary day however as an alternative learn the way the system works and methods to apply it to a small scale. 
2. Create the essential framework quickly. Create a wiki/listing in your wiki and have an index.md file there. Write down what your frontmatter is (title, kind, supply, created, up to date, tags), and be constant in naming your information e.g., concept-name.md or firstname-lastname.md. If this isn’t completed it will likely be troublesome to rectify later. 
3. Spend a number of time creating your preliminary immediate. That is essentially the most vital step. Create guidelines for Classifying, writing TLDRs, writing the frontmatter in addition to guides for when to create Pages and when to edit the pages. Ensure that to maintain updating the immediate as you utilize it. 

Use Claude with Claude Code, or any AI with file entry, to construct and preserve the wiki. Begin at your index file when querying. Let the agent navigate. 

The Sensible Challenges (And The best way to Deal with Them)

Let’s be sensible, Setting up an LLM powered wiki isn’t any straightforward activity because it comes with a number of obstacles as properly:  

1. Constructing an LLM-powered wiki is troublesome: It entails a number of challenges throughout setup, construction, and long-term upkeep.
2. Immediate engineering is the primary problem: You want clear directions for structuring pages, deciding when to create vs replace them, and resolving conflicting data, which requires iteration and refinement.
3. Scalability is a hidden issue: Easy setups break down past just a few hundred pages, so that you want tagging, folders, and search methods deliberate prematurely.
4. Consistency over time issues: With out common upkeep, your wiki will accumulate outdated data, contradictions, and orphaned pages.
5. Agent proficiency is a key ability: Successfully guiding AI by prompts and construction takes follow, and people who spend money on studying this get considerably higher outcomes.

Conclusion 

An important recommendation for constructing your first LLM wiki is identical recommendation Karpathy offers in his gist: don’t overthink the setup. The schema template from this information will be simply copied after which you’ll create the listing construction by executing the bash instructions.  

The system achieves its magical impact by a number of architectural enhancements which develop from the primary day onwards. The wiki turns into extra useful with every new supply materials you embody. The info belongs to you. The information exist in codecs which can be utilized by any system. You need to use any AI you wish to question it. The LLM takes care of all upkeep duties as an alternative of you needing to deal with them which creates a special expertise from different productiveness instruments. 

Your data, lastly, working as onerous for you as you labored to amass it. 

Continuously Requested Questions

Knowledge Science Trainee at Analytics Vidhya
I’m at present working as a Knowledge Science Trainee at Analytics Vidhya, the place I give attention to constructing data-driven options and making use of AI/ML strategies to resolve real-world enterprise issues. My work permits me to discover superior analytics, machine studying, and AI functions that empower organizations to make smarter, evidence-based selections.
With a robust basis in laptop science, software program improvement, and knowledge analytics, I’m captivated with leveraging AI to create impactful, scalable options that bridge the hole between expertise and enterprise.
📩 You may also attain out to me at [email protected]