What is explainable AI?
Explainable AI is methods and systems that make it possible to understand why an AI arrived at a particular result.
Many AI models are hard to inspect from the inside. You can rarely follow exactly how an answer came about. Explainability is about still giving people something to test the result against.
For a language model answering from documents, the source is the most useful explanation. The reader can see which passage the answer builds on and judge whether it holds.
The requirement is highest when AI is used for decisions about people. The GDPR has specific rules for decisions based solely on automated processing.
How it works
There are two main routes. One is to choose models simple enough to follow, such as rules or decision trees. The other is to explain a complex model from the outside, for example by showing which inputs influenced the result most or which material an answer rests on. For language models the second route is the usual one.
Why it matters
Explanations make it possible to spot mistakes, correct them and justify a decision to the person affected. In the public sector this ties in with duties to give reasons and to administer well. In any organisation it makes AI easier to trust.
What to keep in mind
An explanation the model writes itself is not always an account of how the answer came about. It can be a plausible story told after the fact.
So rely more on what can be checked, such as citations and logs, than on the model's own description of its reasoning.
What it means for you
Require every AI answer used at work to be traceable back to its material.
Common questions
Does AI have to explain its decisions?
When a decision is fully automated and has legal or similarly significant effects on a person, they have a right to meaningful information about the logic involved. That follows from GDPR.
Is a source citation an explanation?
It shows what material the answer is based on, which is often what is needed in practice. It does not explain how the model phrased the answer.
Sources: GDPR, Article 22, on EUR-Lex
Related terms
Read more
All terms Updated