Train AI on your documents or let it read them?
Many organisations want an AI that knows their procedures, contracts and manuals. The usual idea is to train a model on the documents. There is another way, in which the model reads the right documents every time someone asks.
iSearch
Contents
The question comes up early in almost every conversation about AI in an organisation. Can we train an AI on our own documents, so it can answer questions about how things work here? The answer is yes, but it is seldom the best way to get there. It often works better to leave the model as it is and give it the right documents to read when the question is asked. The choice affects how current the answers are, whether they can be checked and who gets to see what.
Training the model or letting it read
With fine-tuning, an existing language model is trained further on your own material. The model changes, and what it has learned is then built into it. The method suits teaching a model a tone, a format or a well-defined task that recurs.
With RAG, retrieval-augmented generation, the model is left untouched. When someone asks a question, the system first finds the passages in your documents that best answer it. The model reads them and writes its answer from there. Researchers described the method in 2020, with the specific aim of giving language models access to knowledge that can be updated and traced.
Current answers with sources
A fine-tuned model knows what the material said when it was trained. When a procedure changes, the old version lives on in the model until it is retrained. With RAG, the model reads the document that applies today. A new document can be used as soon as it has been added, and a document that is removed no longer affects the answers.
Sources also set the methods apart. A trained model can rarely show where a particular piece of information came from, because the knowledge has been absorbed into the model. When the answer is based on documents that have been read, it can show which ones. The reader can open the source and check, and the risk of the model making things up falls when the answer has something to stand on.
Permissions and personal data
In an organisation, not everyone may read everything. With RAG, the search can follow the same permissions as your other systems, so each person only gets answers from the documents they are allowed to see. A model trained on all the material cannot keep it apart in the same way. What it has learned could in principle surface in an answer to anyone.
Personal data makes this more important. Research has shown that language models can reproduce parts of what they were trained on, sometimes word for word. In December 2024 the European Data Protection Board concluded that an AI model trained on personal data cannot in all cases be considered anonymous. Removing a single piece of information from a trained model is difficult. When the model reads the documents, it is enough to change or delete the document.
Fine-tuning still has its place. It can give a model a specialist vocabulary, a fixed structure for its answers or better accuracy in a narrow task, and the methods can be combined. For questions about what applies in your organisation, it is reading your documents that gives the answer its foundation.
How it works in iSearch
The assistants in iSearch answer from your own knowledge banks and show which documents were read. Your questions and documents are never used to train models. Role-based access controls who reaches what, so answers are based only on what the user is allowed to see. Every customer gets a separate installation of its own, and the whole chain can run in your own environment.
Read more about how to stop AI making things up and permissions in AI, or book a walkthrough.
Sources: Lewis et al., Retrieval-Augmented Generation (2020)Carlini et al., Extracting Training Data from Large Language Models (2020)EDPB, Opinion 28/2024 on AI models (2024)
Want to see how it would work for you?
We’ll show you iSearch with your own documents.