What if the best way we construct AI doc chatbots immediately is flawed? Most methods use RAG. They break up paperwork into chunks, create embeddings, and retrieve solutions utilizing similarity search. It really works in demos however usually fails in actual use. It misses apparent solutions or picks the incorrect context. Now there’s a new strategy known as PageIndex. It doesn’t use chunking, embeddings, or vector databases. But it reaches as much as 98.7% accuracy on robust doc Q&A duties. On this article, we’ll break down how PageIndex works, why it performs higher on structured paperwork, and how one can construct your individual chatbot utilizing it.

The Downside with Conventional RAG

Right here’s the basic RAG pipeline you’ve most likely seen 100 instances.

• You’re taking your doc – might be a PDF, a report, a contract – and also you chop it into chunks. Possibly 512 tokens every, perhaps with some overlap.
• You run every chunk by means of an embedding mannequin to show it right into a vector — an extended listing of numbers that represents the “which means” of that chunk.
• You retailer all these vectors in a vector database — Pinecone, Weaviate, Chroma, no matter your flavour is.
• When the consumer asks a query, you embed the query the identical approach, and also you do a cosine similarity search to search out the chunks whose vectors are closest to the query vector.
• You hand these chunks to the LLM as context, and it writes the reply.

Easy. Elegant. And completely riddled with failure modes.

Downside 1: Arbitrary chunking destroys context

Whenever you slice a doc at 512 tokens, you’re not respecting the doc’s precise construction. A single desk would possibly get break up throughout three chunks. A footnote that’s crucial to understanding the primary textual content leads to a totally completely different chunk. The reply you want would possibly actually span two adjoining chunks that the retriever picks solely certainly one of.

Downside 2: Similarity just isn’t the identical as relevance

That is the large one. Vector similarity finds textual content that feels like your query. However paperwork usually don’t repeat the query’s phrasing once they reply it. Ask “What’s the termination clause?” and the contract would possibly simply say “Part 14.3 — Dissolution of Settlement.” Low cosine similarity. Missed fully.

Downside 3: It’s a black field

You get three chunks again. Why these three? You don’t have any thought. It’s pure math. There’s no reasoning, no rationalization, no audit path. For monetary paperwork, authorized contracts, and medical information? That opacity is a major problem.

Downside 4: It doesn’t scale to lengthy paperwork

A 300-page technical guide with complicated cross-references? The sheer variety of chunks makes retrieval noisy. You find yourself getting chunks which might be vaguely associated as an alternative of the precise part you want.

These aren’t edge circumstances. These are the on a regular basis failures that RAG engineers spend most of their time preventing. And the rationale they occur is definitely fairly easy — all the structure is borrowed from engines like google, not from how people truly learn and perceive paperwork.

When a human knowledgeable must reply a query from a doc, they don’t scan each sentence searching for the one which sounds most much like the query. They open the desk of contents, skim the chapter headings, navigate, and motive about the place the reply ought to be earlier than they even begin studying.

That’s the perception behind PageIndex.

What’s PageIndex?

PageIndex was constructed by VectifyAI and open-sourced on GitHub. The core thought is deceptively easy:

As an alternative of looking a doc, navigate it: the best way a human knowledgeable would.

Right here’s the important thing psychological shift. Conventional RAG asks: “Which chunks look most much like my query?”

PageIndex asks: “The place on this doc would a wise human search for the reply to this query?”

These are two very completely different questions. And the second seems to supply dramatically higher outcomes.

PageIndex does this by constructing what it calls a Reasoning Tree. It’s basically an clever, AI-generated desk of contents to your doc.

Right here’s the best way to visualize it. On the prime, you could have a root node that represents all the doc. Beneath that, you could have nodes for every main part or chapter. Every of these branches into subsections. Every subsection branches into particular subjects or paragraphs. Each single node on this tree has two issues:

1. A title: what this part is about
2. A abstract: a concise AI-generated description of what’s on this part

This tree is constructed as soon as, if you first submit the doc. It’s your index.

Now right here’s the place it will get intelligent. Whenever you ask a query, PageIndex does two issues:

1. Tree Search (Navigation)

It sends the query to an LLM together with the tree, however simply the titles and summaries, not the total textual content. The LLM reads by means of the tree like a human reads a desk of contents, and it causes: “Okay, given this query, which branches of the tree are more than likely to comprise the reply?”

The LLM returns an inventory of particular node IDs, and you may see its reasoning. It actually tells you why it selected these sections. Full transparency.

PageIndex fetches solely the total textual content of these chosen nodes, fingers it to the LLM as context, and the LLM writes the ultimate reply grounded fully in the actual doc textual content.

Two LLM calls. No embeddings. No vector database. Simply reasoning.

And since each reply is tied to particular nodes within the tree, you all the time know precisely which web page, which part, which a part of the doc the reply got here from. Full audit path. Full explainability.

The way it Works: Deep Dive

Let me go deeper into the mechanics, as a result of that is the actually attention-grabbing half.

The Tree Index – Constructing Section

Whenever you name submit_document(), PageIndex reads your PDF or textual content file and does one thing exceptional. It doesn’t simply extract textual content but additionally understands the construction. Utilizing a mixture of format evaluation and LLM reasoning, it identifies:

• What are the pure sections and subsections?
• The place does one matter finish and one other start?
• How do the items relate to one another hierarchically?

It then constructs the tree and generates a abstract for each node. Not only a title. An precise condensed description of what’s in that part. That is what permits the good navigation later.

The tree makes use of a numeric node ID system that mirrors actual doc construction: 0001 may be Chapter 1, 0002 Chapter 2, 0003 the primary part inside Chapter 1, and so forth. The hierarchy is preserved.

Why This Beats Chunking

Take into consideration what chunking does to a 50-page monetary report. You get perhaps 300 chunks, every with zero consciousness of whether or not it’s from the chief abstract or a footnote on web page 47. The embedder treats all of them equally.

The PageIndex tree, then again, is aware of that node 0012 is the “Income Breakdown” subsection below the “Q3 Monetary Outcomes” part below “Annual Report 2024.” That structural consciousness is enormously beneficial if you’re looking for one thing particular.

The Search Section – Reasoning, Not Math

Right here’s the opposite factor that makes PageIndex particular. The search step just isn’t a mathematical operation. It’s a cognitive operation carried out by an LLM.

Whenever you ask, “What have been the primary danger elements disclosed on this report?”, the LLM doesn’t measure cosine distance. It reads the tree, acknowledges that the “Threat Elements” part is precisely what’s wanted, and selects these nodes, identical to you’d.

This implies PageIndex handles semantic mismatch naturally. That is the sort of mismatch that kills vector search. The doc calls it “Threat Elements.” Your query calls it “major risks.” A vector search would possibly miss it. An LLM studying the tree construction is not going to.

The Numbers

PageIndex powered Mafin 2.5, VectifyAI’s monetary RAG system, which achieved 98.7% accuracy on FinanceBench. For these unaware, it is a benchmark particularly designed to check AI methods on monetary doc questions, the place the paperwork are lengthy, complicated, and filled with tables and cross-references. That’s the toughest setting for conventional RAG. It’s the place PageIndex shines most.

What’s it Greatest For?

PageIndex is especially highly effective for:

• Monetary studies: earnings statements, SEC filings, 10-Ks
• Authorized contracts: the place each clause issues and context is every thing
• Technical manuals: complicated cross-referenced documentation
• Coverage paperwork: HR insurance policies, compliance paperwork, regulatory filings
• Analysis papers: structured tutorial content material

Mainly: anyplace your doc has actual construction that chunking would destroy.

And the actually thrilling factor? You should use it with any LLM. OpenAI, Anthropic, Gemini — the tree search and reply technology steps are simply prompts. You’re in full management.

Arms-on With Jupyter Pocket book

Okay. You now know the idea. You already know why PageIndex exists, what it does, and the way it works below the hood. Now let’s truly construct one thing with it.

I’m going to open a Jupyter pocket book and stroll you thru the entire PageIndex pipeline: importing a doc, getting the reasoning tree again, navigating it with an LLM, and asking questions. Each line of code is defined. No hand-waving.

Set up PageIndex

%pip set up -q --upgrade pageindex

 First issues first. We set up the pageindex Python library. One line, executed. No vector database to arrange. No embedding mannequin to obtain. That is already easier than any conventional RAG setup.

Imports & API Setup

import os
from pageindex import PageIndexClient
import pageindex.utils as utils
from dotenv import load_dotenv
load_dotenv()
PAGEINDEX_API_KEY = os.getenv("PAGEINDEX_API_KEY")
pi_client = PageIndexClient(api_key=PAGEINDEX_API_KEY)

We import the PageIndexClient. That is our connection to the PageIndex API. All of the heavy lifting of constructing the tree occurs on their finish, so we don’t want a beefy machine. We additionally load API keys from a .env file — all the time preserve your keys out of your code.

OpenAI Setup

import openai 
async def call_llm(immediate, mannequin="gpt-4.1-mini", temperature=0): 
    shopper = openai.AsyncOpenAI(api_key=OPENAI_API_KEY) 
    response = await shopper.chat.completions.create(...) 
    return response.decisions[0].message.content material.strip()

Right here we outline our LLM helper perform. We’re utilizing GPT-4.1-mini for value effectivity — however this works with any OpenAI mannequin, and you would swap in Claude or Gemini with a one-line change. Temperature zero retains the solutions factual and constant.

Submit the Doc

pdf_path = "/Customers/soumil/Desktop/PageIndex/HR Insurance policies-1.pdf" 
doc_id = pi_client.submit_document(pdf_path)["doc_id"] 
print('Doc Submitted:', doc_id)

That is the magic line. We level to our PDF — on this case an HR coverage doc — and submit it. PageIndex takes the file, reads its construction, and begins constructing the reasoning tree within the background. We get again a doc_id, a novel identifier for this doc that we’ll use in each subsequent name. Discover there’s no chunking code, no embedding name, no vector database connection.

Look ahead to Processing & Get the Tree

whereas not pi_client.is_retrieval_ready(doc_id): 
    print("Nonetheless processing... retrying in 10 seconds") 
    time.sleep(10) 
tree = pi_client.get_tree(doc_id, node_summary=True)['result'] 
utils.print_tree(tree)

PageIndex processes the doc asynchronously — we simply ballot each 10 seconds till it’s prepared. Then we name get_tree() with node_summary=True, which supplies us the total tree construction together with summaries.

Have a look at this output. That is the reasoning tree. You’ll be able to see the hierarchy — the top-level HR Insurance policies node, then Digital Communication Coverage, Sexual Harassment Coverage, Grievance Redressal Coverage, every branching into its subsections. Each node has an ID, a title, and a abstract of what’s in it.

That is what conventional RAG throws away. The construction. The relationships. The hierarchy. PageIndex retains all of it.

Tree Search with the LLM

question = "What are the important thing HR insurance policies and worker tips?" 
tree_without_text = utils.remove_fields(tree.copy(), fields=['text']) 
search_prompt = f""" 
You might be given a query and a tree construction of a doc... 
Query: {question} 
Doc tree construction: {json.dumps(tree_without_text, indent=2)} 
Reply in JSON: {{ "considering": "...", "node_list": [...] }} 
""" 
tree_search_result = await call_llm(search_prompt)

Now we search. For this, we construct a immediate that features the query and all the tree — however crucially, with out the total textual content content material of every node. Simply the titles and summaries. This retains the immediate manageable whereas giving the LLM every thing it must navigate.

The LLM is instructed to return a JSON object with two issues: its considering course of and the listing of related node IDs.

Have a look at the output. The LLM tells us precisely why it selected every part. It reasoned by means of the tree like a human would. And it gave us an inventory of 30 node IDs — each part of this HR doc, as a result of the query is broad.

This transparency is one thing you merely can’t get with cosine similarity.

Fetch Textual content and Generate Reply

node_list = tree_search_result_json["node_list"] 
relevant_content = "nn".be part of(node_map[node_id]["text"] for node_id in node_list) 
answer_prompt = f"""Reply the query primarily based on the context: 
Query: {question} 
Context: {relevant_content}""" 
reply = await call_llm(answer_prompt) 
utils.print_wrapped(reply)

Step two. Now that we all know which nodes are related, we fetch their full textual content — solely these nodes, nothing else. We be part of the textual content and construct a clear context immediate. Yet another LLM name, and we get our reply.

Have a look at this reply. Detailed, structured, correct. And each single declare could be traced again to a particular node within the tree, which maps to a particular web page within the PDF. Full audit path. Full explainability.

The ask() Perform

async def ask(question): 
    # Full pipeline: tree search → textual content retrieval → reply technology 
    ... 
 
user_query = enter("Enter your question: ") 
await ask(user_query)

Now we package deal all the pipeline right into a single ask() perform. Submit a query, get a solution — the tree search, retrieval, and technology all occur below the hood. Let me present you a few dwell examples.

Sort a query: e.g., “What are the penalties for sexual harassment?”

Watch what occurs. It searches the tree, identifies the Sexual Harassment Coverage nodes particularly, pulls their textual content, and offers us a exact, cited reply in seconds. That is the expertise you wish to ship to your customers.

One other one. Once more, it finds precisely the suitable part. No confusion, no noise, no hallucination. Simply the reply, from the doc, with a transparent path displaying the place it got here from.

Conclusion

Let’s carry this collectively. Conventional RAG finds textual content that appears much like a query. However the actual objective is to search out the suitable reply in a structured doc. PageIndex solves this higher. It builds a reasoning tree and lets the mannequin navigate it intelligently. The result’s correct and explainable solutions, with as much as 98.7% accuracy on FinanceBench. It isn’t good for each use case. Vector search nonetheless works effectively for giant scale semantic search. However for lengthy, structured paperwork, PageIndex is a stronger strategy. You will discover all of the code within the description. Add your API keys and get began.