Offline AI: the whole chain in your own environment

When most people think of AI, they think of a service in the browser. You type a question, press enter and an answer comes back. It feels simple. But on the way, the question has left the organisation, and parts of your documents have often gone with it.

For many organisations that hardly matters. For others, it is exactly where the conversation ends. Healthcare, the public sector and companies with sensitive commercial data need to know where their data lives and who handles it. For them, AI where the whole chain sits in their own environment is what makes getting started possible at all.

What offline means

When we say offline, we mean that you own the whole chain. The software, your documents and the language model itself run in your own environment. Questions and excerpts are not sent to an external AI provider unless you choose to.

That includes what protects the environment. Passwords and keys are stored in your own installation. If you want to use an existing service for keys, you can, but it is not a requirement. The environment can be connected like your other internal systems, and when required it can also run fully isolated with an air gap.

The difference from solutions described as private can be bigger than it sounds. Some keep your documents with you but still send the question, and the excerpts needed for the answer, to an external model. The data leaves anyway, just in smaller pieces. When the whole chain is with you, the question, the material and the answer stay in your environment.

What it looks like day to day

Take a case handler who needs to know how a certain type of case should be handled. She types the question the way she would ask an experienced colleague. The assistant searches the organisation’s own routines, finds the passages that apply and writes an answer. Beneath it are the documents it is based on, so she can open them and read for herself.

All of that happens on servers you control. For the person using the assistant, nothing changes. It answers just as quickly and shows its sources in the same way.

It also simplifies a lot that otherwise takes time. When the data stays with you, there is no need to investigate transfers to other countries or how an external supplier treats your questions. Your security team can review one installation instead of a cloud service with many unknown parts.

What to ask for

The most important question for a supplier is also the simplest. Can the whole chain run in your own environment, including the language model? The answer should be yes, with no caveats about features that only exist in an outside service.

Then ask where passwords and keys are stored, and whether your data is used to train models. The answer to the last one should be no. Ask how sign-in and permissions work, whether there is an audit log that cannot be changed after the fact, and whether every answer shows which documents it is based on. AI in your own environment that answers without sources has only solved half the problem.

Think about who should run it, too. Some want everything in-house. Others prefer the supplier to run a separate installation for them. Both can be right. What matters is that you choose, and that the choice is not dictated by what the technology can handle.

How it works in iSearch

In iSearch, the whole chain can run in your own environment, the language model included. Every customer gets a separate installation of its own, and passwords and keys are stored there. If you would rather we ran it, we do so in Sweden, in an environment of its own for your organisation.

Sign-in uses SSO and MFA, access is controlled with RBAC and everything that happens is written to a tamper-evident audit log that can be connected to your SIEM. Every answer shows which documents were read, and your data is never used to train models.

Read more about security, see how iSearch is used or book a walkthrough.