What is an embedding?
An embedding is a long series of numbers that describes what a piece of text means, so that texts with similar content end up close to each other.
A model reads the text and calculates the numbers. Two sentences about the same thing get similar numbers, even when they use different words. That makes it possible to compare meaning rather than spelling.
Embeddings are the basis of vector search. Documents are split into passages, each passage gets an embedding and the question gets one of its own. The search retrieves the passages closest to the question.
The technique spread widely after research on word representations in 2013. Today it is used in search, in recommendations and in RAG, where it helps an AI assistant find the right material.
How it works
The numbers can be seen as a position in a space with a great many dimensions. Texts about holidays end up in one area and texts about invoices in another. When someone asks about annual leave, the question lands near the passages on holidays, even if the word leave does not appear in them.
The same model must be used for documents and questions. If the model changes, every document needs to be processed again.
Why it matters
Embeddings let an AI assistant find the right passage when the question is phrased differently from the text. That is a precondition for good answers with RAG, because the model can answer only from what the search finds.
What to keep in mind
Embeddings need to follow the documents' permissions. Otherwise the search can find passages the user is not allowed to see. Read more about vector search.
Research has shown that parts of a text can be reconstructed from its embedding. Treat embeddings as part of the content.
What it means for you
Embeddings are calculated from your texts and can reveal part of their content. Protect them with the same permissions and retention rules as the documents.
Common questions
Is an embedding the same as a summary?
No. It cannot be read as text. It describes the content with numbers that a computer can compare with others.
Do we need to understand embeddings to use AI search?
No. It is enough to know that they exist and that they should be protected like the documents. Read more about RAG.
Sources: Mikolov et al. 2013 on arXivMorris et al. 2023 on arXiv
Related terms
Read more
All terms Updated