If you want an AI system to answer questions about your own documents — your contracts, your handbook, your product documentation — you need something beyond a chat window. The standard approach is called retrieval-augmented generation, usually shortened to RAG. The name is worse than the idea.
The problem it solves
A language model knows what it absorbed during training. It does not know your company's internal documents, and asking it about them produces invention rather than information.
The obvious fix — paste the documents into the question — works until the documents exceed what fits, which happens almost immediately for any real collection.
The idea
Instead of giving the model everything, find the parts relevant to the question and give it those. The question becomes "here are four passages from our documentation; answer using them" rather than "what do you know about our returns policy".
The task shifts from recall to reading comprehension, which these systems do considerably better.
How it works
Preparation, done once. Documents are split into sections of manageable size. Each section is converted into a numerical representation capturing its meaning — an embedding. These are stored in a searchable index.
At question time. The question is converted the same way. The system finds the stored sections whose representations are most similar in meaning, retrieves the best few, and includes them with the question. The model answers from that material.
The important property is that matching happens on meaning rather than wording. A question about "sending goods back" can retrieve a section headed "returns and refunds" without sharing a single word.
Why it is worth the effort
Answers are grounded in your actual documents. Sources can be cited, so a reader can verify a claim rather than trusting it. Updating a document updates the answers — no retraining. And access control can be applied at retrieval, so people only get answers from material they are permitted to see.
That last point is easy to overlook and important. A well-built system will not answer an ordinary employee's question using the salary spreadsheet.
Where it goes wrong
Retrieval misses. If the right passage is not retrieved, the answer will be wrong or absent, however capable the model. Most quality problems in RAG systems are retrieval problems, not model problems — a fact that surprises people who assume a better model is the fix.
Bad splitting. If a section is cut mid-argument, retrieved fragments lack the context needed to interpret them. How documents are divided has a large effect on quality and is worth deliberate attention.
Questions spanning many documents. "How many contracts include this clause" requires reading everything, not retrieving the top few. RAG suits targeted questions, not exhaustive analysis.
Stale or contradictory sources. If three versions of a policy exist, retrieval may surface the wrong one. The system inherits the state of your documentation — it does not repair it.
What to ask about it
If a provider proposes this approach, useful questions are: how are documents split, what happens when nothing relevant is found, are sources cited, how are permissions enforced at retrieval, and how quickly does a document change take effect.
Answers to those tell you more about the system's quality than any statement about which model is underneath.
