RAG, GraphRAG, and LLM Wikis: Three Ways to Give AI Access to Knowledge

A large language model knows only what its training and current context make available. It does not automatically know a company’s latest policies, a customer’s current order status, or the contents of a private document repository.

Three related approaches try to close that gap:

1. Retrieval-Augmented Generation (RAG) finds relevant source passages when a question is asked.
2. GraphRAG adds an explicit model of entities, relationships, and larger themes.
3. An LLM wiki uses an LLM to incrementally compile source material into a persistent, navigable wiki.

These terms are often mixed together, but they solve different problems. RAG is primarily a retrieval pattern. GraphRAG is a family of graph-assisted retrieval patterns. “LLM wiki” is an emerging design pattern, not an established industry standard.

What Is RAG? #

Retrieval-Augmented Generation combines information retrieval with text generation. Instead of asking a model to answer entirely from its learned parameters, the application first retrieves evidence from an external collection and places that evidence in the model’s context.

A typical RAG pipeline looks like this:

Documents
   |
parse and normalize
   |
split into chunks
   |
create search indexes
   |
   +-------------------------------+
                                   |
User question -> retrieve passages -> LLM -> grounded answer + citations

During ingestion, the system usually:

• extracts text, tables, headings, page numbers, and metadata;
• divides documents into chunks;
• creates embeddings for semantic similarity;
• stores lexical and vector indexes;
• retains source identifiers, permissions, versions, and locations.

At query time, it:

• interprets or rewrites the question;
• filters candidates by tenant, user, product, locale, or permissions;
• performs keyword, vector, or hybrid search;
• reranks the best candidates;
• assembles a limited evidence context;
• asks an LLM to answer using that context;
• attaches citations back to the original sources.

RAG is more than vector search #

Early descriptions often reduce RAG to “put documents in a vector database.” Production systems normally use hybrid retrieval:

• Lexical search is strong for exact names, product codes, error messages, policy wording, and identifiers.
• Vector search is strong when the question paraphrases the source.
• Metadata filtering enforces scope, permissions, dates, and business categories.
• Reranking evaluates a smaller result set more precisely than the first-stage search.

The database is only one component. Parsing quality, chunk boundaries, permissions, freshness, citation handling, and evaluation often matter more than the choice of vector store.

What RAG is good at #

RAG works well when an answer is supported by a small number of identifiable passages:

• “What is the return period for an unopened product?”
• “Which configuration option enables audit logging?”
• “Summarize the warranty exclusions in this manual.”
• “According to the current policy, when should this case be escalated?”

It is particularly useful for customer support because approved policies and documentation can be updated without retraining the model.

Where ordinary RAG struggles #

Ordinary chunk retrieval can fail when:

• the answer requires combining many weakly related documents;
• relationships between entities are more important than textual similarity;
• the user asks for themes across an entire corpus;
• retrieved chunks omit necessary surrounding structure;
• several versions of a fact coexist;
• the answer depends on live customer data rather than documents.

RAG also does not guarantee truth. If the source is wrong, stale, unauthorized, or malicious, retrieval can faithfully supply the wrong evidence.

Famous RAG software #

Amazon Bedrock Knowledge Bases #

Amazon Bedrock Knowledge Bases is a managed RAG service. It connects data sources, handles ingestion and indexing, retrieves relevant information, supports reranking, and can generate answers with citations. AWS also offers customer-managed configurations for teams that need control over the underlying vector or graph store.

Google RAG Engine #

Google’s RAG Engine, now documented as part of Gemini Enterprise Agent Platform, covers ingestion, transformation, chunking, embedding, corpus indexing, retrieval, and grounded generation.

Azure AI Search #

Azure AI Search supports classic and agentic RAG patterns over enterprise search indexes. It is notable for combining mature keyword search, vector search, semantic ranking, filters, and Azure security integration.

Dify, RAGFlow, and Onyx #

• Dify is an open-source AI application platform with knowledge bases, workflows, agents, model integrations, and deployment APIs.
• RAGFlow focuses on document understanding and configurable RAG pipelines, including difficult PDFs, tables, and knowledge-graph-oriented retrieval.
• Onyx is an open-source enterprise search and AI assistant platform with connectors to workplace knowledge sources.

These products package more of the application around RAG than a vector database or framework alone.

LlamaIndex and Haystack #

LlamaIndex and Haystack are code-first frameworks. They provide composable ingestion, indexing, retrieval, reranking, prompting, and agent components, but teams still build and operate the customer-facing application around them.

What Is GraphRAG? #

GraphRAG augments retrieval with a graph of entities and relationships. Instead of representing knowledge only as isolated text chunks, it may represent facts such as:

(Customer) --placed--> (Order)
(Order) --contains--> (Product)
(Product) --covered_by--> (Warranty Policy)
(Warranty Policy) --has_exception--> (Commercial Use)

There is no single GraphRAG architecture. The name covers several patterns:

• retrieve entities with vectors, then traverse their graph neighborhood;
• convert questions into graph queries;
• combine graph results with ordinary text retrieval;
• embed graph nodes, edges, or subject-predicate-object triplets;
• cluster a graph and generate summaries for each community;
• use a manually curated domain graph rather than an automatically extracted one.

Microsoft’s GraphRAG pattern #

Microsoft GraphRAG popularized a particular approach for extracting structure from unstructured text. Its indexing pipeline:

1. extracts entities, relationships, and claims;
2. performs community detection over the entity graph;
3. generates summaries at multiple levels;
4. embeds text and graph-derived artifacts for retrieval.

This architecture supports two broad kinds of questions:

• Local questions, about a particular entity or nearby facts.
• Global questions, about themes, patterns, or the overall contents of a large corpus.

The global case is a major motivation for GraphRAG. A vector search for “What are the main concerns across these reports?” may return a handful of locally similar chunks. A hierarchy of graph-community summaries can provide a broader view.

What GraphRAG is good at #

GraphRAG becomes attractive when:

• a result requires multiple hops across entities;
• relationships have business meaning;
• information is distributed across many documents;
• users ask corpus-wide or thematic questions;
• provenance paths improve explainability;
• the company already maintains a useful knowledge graph.

Examples include fraud analysis, supply chains, scientific literature, legal matters, software dependencies, customer-account relationships, and organizational ownership.

The cost of GraphRAG #

GraphRAG is not an automatic upgrade over RAG. It introduces:

• extra LLM calls during entity and relation extraction;
• entity-resolution and deduplication problems;
• schema and ontology decisions;
• graph storage or graph-derived tables;
• more complicated incremental updates and deletion;
• more retrieval strategies to evaluate;
• the possibility of hallucinated graph edges.

An automatically extracted relationship is not necessarily an authoritative fact. Systems should retain source provenance and distinguish curated records from LLM-derived interpretations.

For many customer-support questions, hybrid RAG is cheaper, faster, and easier to debug. GraphRAG should be adopted because measured queries need relationship-aware or global retrieval, not because a graph visualization looks impressive.

Famous GraphRAG software #

Microsoft GraphRAG #

Microsoft GraphRAG is the best-known open-source implementation of community-based GraphRAG over unstructured text. It is a toolkit and reference architecture rather than a complete customer-service product.

Neo4j GraphRAG #

Neo4j’s GraphRAG ecosystem combines graph queries, vector indexes, knowledge-graph construction, and integrations with common LLM frameworks. It is especially relevant when Neo4j already stores authoritative domain relationships.

GraphRAG Toolkit for Amazon Neptune #

AWS publishes the open-source GraphRAG Toolkit for building graph-enhanced retrieval with Amazon Neptune. It targets teams already using AWS graph infrastructure.

Cognee and Graphiti #

• Cognee builds graph, vector, and relational representations from documents and agent interactions.
• Graphiti builds temporal context graphs that retain entities, relationships, provenance, and changing facts.

They overlap with GraphRAG but also position themselves as agent memory or context infrastructure. They should not automatically become the authoritative company database.

What Is an LLM Wiki? #

“LLM wiki” has two common meanings, and confusing them causes trouble.

Meaning 1: a wiki with AI search #

Most commercial “AI wikis” are existing knowledge products enhanced with RAG:

Human-authored wiki pages
          |
      search index
          |
Question -> retrieve -> LLM answer

Humans still author and approve the canonical pages. The LLM helps users find, summarize, translate, or draft content.

Examples include:

• Notion Enterprise Search, which searches Notion and connected sources such as Slack, Google Drive, GitHub, Jira, Teams, and SharePoint;
• Atlassian Rovo in Confluence, which adds AI search, chat, summaries, and agent features around Confluence and connected Atlassian knowledge;
• Glean, which provides enterprise search and assistants across company applications;
• Guru, which combines a company knowledge base with verification workflows and AI assistance;
• Slite, GitBook, and similar documentation products with AI question-answering features.

Architecturally, these are usually RAG over a wiki, not a new knowledge-storage model.

Meaning 2: a wiki maintained by an LLM #

In the stricter pattern, the LLM performs significant work at ingestion time:

Raw source documents
          |
LLM analyzes, reconciles, and organizes
          |
Persistent generated wiki
  - topic pages
  - entity pages
  - summaries
  - backlinks
  - source references
          |
Users and agents navigate or query the compiled wiki

The model does not merely retrieve raw chunks and regenerate an answer from scratch for every question. It incrementally creates and updates a durable intermediate knowledge representation.

Andrej Karpathy’s LLM Wiki pattern describes this as three layers:

1. immutable raw sources;
2. an LLM-generated wiki;
3. a schema that tells the LLM how to organize and maintain it.

The key operations are ingestion, querying, and linting. Humans curate; the LLM performs much of the maintenance.

A concrete LLM-wiki implementation #

nashsu/llm_wiki is a cross-platform desktop implementation of this pattern. It reads imported documents, creates interlinked Markdown pages with YAML front matter and source references, maintains indexes and logs, provides graph and vector search, and exposes local API and MCP interfaces.

This is much closer to a strict LLM wiki than products that merely add chat to a conventional document store.

Why compile a wiki? #

The approach can offer:

• a navigable body of durable pages rather than transient answers;
• shared summaries reused across many queries;
• explicit links among topics and entities;
• opportunities to detect duplication, contradiction, and missing coverage;
• human review of the intermediate knowledge representation;
• lower repeated reasoning cost when the same sources are queried frequently.

Why an LLM wiki is risky #

The generated wiki is a lossy, model-produced interpretation of the sources. It can:

• omit important qualifications;
• merge entities incorrectly;
• convert ambiguity into false certainty;
• preserve outdated synthesis after a source changes;
• create circular citations among generated pages;
• make generated prose look more authoritative than it is.

A safe design preserves immutable sources, records source references on every generated page, supports rebuilding, separates generated and human-approved content, and requires review before consequential knowledge is promoted to “truth.”

RAG vs. GraphRAG vs. LLM Wiki #

Which One Should You Use? #

Start with RAG when: #

• most questions can be answered from one to five passages;
• your knowledge consists mainly of policies, manuals, FAQs, and product documentation;
• you need the shortest path to a customer-facing assistant;
• citations, permissions, cost, and predictable latency matter.

A sensible default is structure-aware parsing, hybrid lexical/vector retrieval, metadata filters, reranking, and explicit citations.

Add GraphRAG when: #

• evaluation shows repeated failures on relationship or multi-hop questions;
• the domain already has useful entities and typed relationships;
• users ask global questions across large collections;
• graph paths provide real audit or analytical value.

Do not begin by extracting a giant graph from everything. Start with a bounded domain and compare it against a strong hybrid-RAG baseline.

Consider an LLM wiki when: #

• users benefit from browsing a synthesized knowledge space, not only asking questions;
• the same corpus is repeatedly analyzed;
• topic pages and cross-links are valuable artifacts;
• humans can review generated structure;
• all generated claims remain traceable to immutable sources.

An LLM wiki is particularly interesting for research collections, technical archives, personal knowledge, and internal intelligence. It is less suitable as the sole authority for prices, account state, legal obligations, or rapidly changing transactional data.

A Customer-Service Agent Usually Needs More Than RAG #

Consider three requests:

1. “What is your cancellation policy?”
   Retrieve and cite the current policy with RAG.

2. “Which of my subscriptions are covered by that policy?”
   Retrieve the policy, then call an authorized account API. Graph retrieval may help if product and policy relationships are complex.

3. “Cancel the eligible subscription.”
   Use a typed transactional tool with identity verification, policy checks, confirmation, idempotency, and audit logging.

The resulting architecture is:

Approved documents --------> RAG or GraphRAG index -----+
                                                         |
Customer identity ---------> authorization policy -------+--> bounded agent
                                                         |
CRM / orders / billing ----> typed read and action tools-+
                                                         |
Conversation state --------> short-term session memory --+

Neither RAG, GraphRAG, nor an LLM wiki should replace the systems of record. They help the agent find and organize knowledge. Live customer state and consequential actions still belong behind authorized APIs.

Final Perspective #

RAG is the general-purpose starting point: retrieve current evidence and let the model answer from it. GraphRAG is a specialized extension for connected and corpus-wide questions. An LLM wiki changes the timing of synthesis by compiling source material into a durable, linked knowledge space.

The best production architecture is rarely the most fashionable one. Start with authoritative sources, strong permissions, high-quality parsing, hybrid retrieval, citations, and evaluation. Add graphs or persistent LLM-generated pages only when they solve failures you can measure.