What is bias in AI?
Bias is a systematic skew in what AI produces, often because the training data reflects distortions that were already there.
A model learns from historical examples, including the bad ones. If earlier decisions favoured a certain group, the model can do the same without anyone asking it to.
The risk is highest when AI is used to assess people, for example in recruitment. Such use also counts as high-risk in the EU AI Act.
When AI answers from your own documents, the answers are as good or as poor as the documents. Old or one-sided material gives old or one-sided answers.
How it works
Bias can enter in several places. In the data, when some groups or cases are missing or under-represented. In the labelling, when whoever set the answers made skewed judgements. And in use, when a system built for one purpose is applied to another. A model often amplifies the pattern it has seen most of.
Why it matters
A skewed model can give poorer service or wrong assessments to some groups, without anyone noticing in a single case. That can breach equal treatment rules and damage trust in the whole organisation. The more decisions AI influences, the larger the effect.
What to keep in mind
Bias rarely shows in a single answer. It shows when you compare many. Decide in advance what you will monitor.
Watch for language too. A model that is stronger in English may give weaker answers to someone writing in Swedish or another language.
What it means for you
Do not let AI assess individuals without human review, and keep the material up to date.
Common questions
How do you detect bias?
By comparing results across groups and cases that should be treated alike. That takes testing before the system is used and follow-up afterwards.
Can bias become a legal problem?
Yes. A system that disadvantages certain groups may breach anti-discrimination rules. For high-risk AI, the AI Act also requires training data to be examined for possible biases.
Does bias affect an assistant that answers from our documents?
Less so, because the answer is based on your texts. Skews in the material still carry over, and the model shapes how the answer is phrased.
Sources: NIST SP 1270 on bias in AI
Related terms
Read more
All terms Updated