What is RAG?
RAG, retrieval-augmented generation, is a method where AI first retrieves relevant passages from your documents and then writes the answer from those passages.
A language model answering from memory can only use what it learned in training. With RAG it reads your own material before it answers. The answer then builds on current documents instead of general knowledge.
It works in two steps. First the system finds the passages that match the question. Then the model writes an answer from them and can show which documents it comes from.
Take a colleague who wants to know how to report an incident. With RAG the answer comes from your own routine, with a reference to the right section. When the routine changes, so does the answer, without retraining the model.
How it works
Before the system can answer, the documents are read in and split into passages. Each passage is indexed so it can be found by meaning with vector search, and often by exact words too. When a question arrives it is compared with the passages, and the best matches are sent to the model along with the question.
Why it matters
RAG lets you use a language model on your organisation's own knowledge without retraining it. It can then answer questions about procedures, contracts and decisions it has never seen. Permissions can carry through, so each user only gets answers from documents they are allowed to read.
What to keep in mind
Retrieval decides the result. If the system finds the wrong passages, the answer will be wrong however good the model is. Vague headings, scanned pages without text and several versions of the same procedure all make it harder.
Tidying the material often helps more than changing the model. Test with real questions from staff and look at which passages come back.
What it means for you
With RAG the answers are as good as your documents. Make sure the right versions are included and old ones are removed.
Common questions
Is RAG the same as training a model?
No. RAG leaves the model unchanged. It reads relevant passages from your documents when the question is asked, whereas fine-tuning trains the model further.
How do we know the answer is right?
A good RAG answer shows the passages it is based on. With a source citation, the reader can open the document and check for themselves.
What happens when a document changes?
The new content is used as soon as the document has been read in again. No model needs retraining.
Does RAG remove the risk of wrong answers?
It reduces the risk, because the answer is grounded in your documents. It does not remove it entirely. That is why answers should show their sources and the assistant should say when the material is missing.
Sources: Lewis et al. 2020 on arXiv
Related terms
Read more
All terms Updated