Embedding

An embedding is a list of numbers — a vector — that stands in for a piece of text, an image, a product, or a user, chosen so that the position of that vector carries meaning. Instead of treating the word 'sneaker' as an opaque token, a model maps it to a few hundred coordinates, and it places 'trainer' and 'running shoe' nearby because they mean nearly the same thing. The vector is dense, meaning almost every number carries signal, unlike a sparse list that is mostly zeros. Distance in this space is the whole point: two embeddings that sit close together represent things that are similar, and two that sit far apart represent things that are unrelated. That single property — nearness equals likeness — is what makes embeddings so useful across search, recommendation, and language systems.

Embeddings are produced by models trained on large amounts of data, which learn to place related things together and unrelated things apart. A modern language model, for instance, learns embeddings for words and sentences as a by-product of predicting text, so 'invoice' and 'receipt' end up near each other while 'invoice' and 'volcano' do not. Once you have embeddings, you can measure similarity with a simple calculation such as cosine distance, cluster items that group naturally, or feed the vectors into another model. This is why embeddings sit underneath so much practical machine learning: semantic search that matches meaning rather than exact words, recommendation engines that surface similar products, and retrieval-augmented generation, where an assistant fetches the passages whose embeddings are closest to your question before it answers.

The clearest way to understand an embedding is to contrast it with the crude representations it replaced. A one-hot encoding gives every distinct word its own slot and marks a single one, so 'cat' and 'kitten' are as different from each other as 'cat' and 'spreadsheet' — the representation holds no notion of meaning, only identity. It is also sparse and huge, one dimension per vocabulary item. An embedding is the opposite: compact, dense, and meaning-aware, with 'cat' and 'kitten' landing close together because the model learned they behave alike. Where one-hot encoding can only tell whether two things are literally the same token, an embedding can tell how similar they are in meaning, which is exactly the question most applications actually care about.

The same gap separates embeddings from keyword matching. Classic keyword search looks for the literal words you typed, so a query for 'laptop bag' misses a product listed as 'notebook sleeve' even though they are the same thing. Semantic search built on embeddings compares the meaning of the query with the meaning of each document, so it retrieves the sleeve because its embedding sits near the query's. That is a real advantage, but it comes with trade-offs worth naming: embeddings can return things that are related yet wrong, they are only as good as the model that produced them, and they are harder to inspect than a keyword rule. The strongest systems often blend both — keyword matching for precision on exact terms, embeddings for recall on meaning — rather than treating one as a full replacement for the other.

Using embeddings well starts with picking the right model for your domain and language, because an embedding is only as good as the training behind it — a model tuned on general web text may misjudge medical or legal wording. Normalize and clean the text you embed, decide the unit you are representing (a word, a sentence, a whole document, a product), and store the vectors in a system built for similarity search so lookups stay fast as the collection grows. When you retrieve by similarity, tune how many neighbors you return and set a threshold, because the closest match is not always relevant. For retrieval-augmented generation especially, the quality of the answer depends on the quality of the passages the embeddings pull back, so this layer deserves real attention.

The common mistakes are treating similarity as truth, when nearby vectors can be topically related but factually wrong; embedding text with a model that never saw your domain's vocabulary; and comparing embeddings that came from different models, which live in incompatible spaces and cannot be measured against one another. Teams also forget that embeddings drift — retrain or re-embed when the underlying model or content changes, or the space quietly goes stale. The discipline is to choose a fitting model, keep every vector in one consistent space, verify that 'close' actually means 'relevant' for your task, and combine embedding similarity with exact-match and business rules where precision matters, rather than trusting distance alone to make decisions.

The term embedding comes from mathematics, where one space is embedded within another, and in machine learning it names mapping words or items into a continuous vector space.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where embedding is a core concern: