Search stopped looking for your words. It looks for a point in space that sits near them

Twenty years ago a search engine looked for your words. If you typed "how do I stop my dosa sticking", it found pages containing those words, ranked by how unusual each word was and how many other pages linked in. Today you can type a description of a half-remembered concept without using any of its actual vocabulary and get the right document. Something in between has changed, and it is not that the machine understands you. It is that your sentence has been converted into a list of numbers, and the search is now geometry.

Meaning as coordinates

An embedding model takes a piece of text and returns a vector — an ordered list of a few hundred to a few thousand numbers. That vector is a position in a space with as many dimensions as there are numbers. The model has been trained so that texts used in similar ways end up in similar positions.

The idea underneath is old and linguistic: a word is characterised by the company it keeps. If two words consistently appear surrounded by the same other words, they are doing similar work in the language, whatever a dictionary says. Train a model on enough text with an objective that forces it to predict context, and this similarity becomes distance. The famous early demonstration was that arithmetic on word vectors produced sensible results — the vector for king, minus man, plus woman, landed near queen — which was startling because nobody had programmed any of that in. It fell out of counting which words occur near which.

Modern systems embed whole sentences and documents rather than single words, using transformers trained specifically so that a question and the passage that answers it land close together. That last detail matters more than it sounds: the model is optimised for a particular notion of "related", chosen by whoever trained it.

Why closeness is measured as an angle

Given two vectors, the standard measure of similarity is cosine similarity — the cosine of the angle between them, running from 1 for identical direction to 0 for unrelated to −1 for opposite.

Angle rather than straight-line distance, because the length of an embedding vector tends to reflect uninteresting things such as document length or word frequency, while the direction is what carries the semantic content. Two paragraphs on the same subject, one three times longer, point the same way.

A search is then: embed the query, find the stored vectors with the smallest angle to it, return those documents.

Searching a million points without looking at them

Comparing a query against every stored vector is exact and hopeless at scale. Ten million documents times a thousand dimensions is ten billion multiplications per search.

So real systems use approximate nearest neighbour methods, and the word approximate is doing real work. One family builds a navigable graph in which each vector is linked to its neighbours, so a search walks from an entry point towards the query, hopping closer at each step and visiting a tiny fraction of the data. Another partitions the space into regions and searches only the few regions nearest the query. A third compresses vectors into compact codes so that many more fit in memory, at some cost in precision.

All of these trade recall for speed: they will sometimes fail to return a document that genuinely was among the closest. That is a deliberate engineering decision, tuned by a parameter, and it is why a vector database has settings that quietly change what your users find. A system can be missing five per cent of the right answers and look entirely healthy.

Embedding similarity is not a measure of meaning. It is a measure of how similarly two things were used in the data the model was trained on — which is often the same thing, and sometimes exactly the opposite.

What "similar" actually gets you

That distinction produces the failure modes, and they are consistent enough to plan around.

Antonyms sit close together. Hot and cold appear in almost identical sentences — you turn the tap, the room is, the food went. Distributionally they are near-twins, so an embedding places them near each other. A search for reviews praising a product will happily surface reviews condemning it.

Negation is barely represented. "A study that found no association" and "a study that found an association" differ by one word carrying the entire meaning, and embeddings compress that difference into almost nothing. For anyone retrieving scientific or legal text, this is not an edge case.

Domain mismatch is invisible. A model trained mostly on general web English will place technical vocabulary crudely, and performs far worse on languages it saw little of. Embedding quality for most Indian languages remains well behind English, and a retrieval system built on a weak multilingual model fails quietly — it still returns ten results, they are just worse ones.

Bias comes along for the ride. Since the geometry is built from co-occurrence statistics in human text, whatever associations exist in that text become directions in the space. This has been demonstrated repeatedly, and it matters because embeddings are increasingly used to screen documents, match candidates and rank content.

Why this is now the substrate under everything

Once text is a vector, several different problems become the same problem. Semantic search is nearest-neighbour lookup. Recommendation is placing users and items in one space and finding what is near a user. Deduplication and clustering are grouping by proximity. Anomaly detection is finding points far from everything. Multimodal models put images and text into a shared space, which is how you search photographs with a sentence.

And retrieval-augmented generation — the standard way of making an AI assistant answer from your documents rather than from memory — is an embedding search followed by a language model. This deserves emphasis because it is widely misunderstood: in most RAG systems that give poor answers, the language model is not the problem. The retrieval step handed it the wrong passages, and no amount of prompt engineering repairs a context window filled with irrelevant text. Retrieval quality is a ceiling on the whole system, and it is the part nobody measures.

The practical answer is not to choose

Keyword search was never actually beaten; it was joined. Classical term-matching remains better than embeddings at precisely the things embeddings are worst at: rare exact strings, product codes, error messages, surnames, an ISSN, a section number. A vector model will return something like your part number. A keyword index returns your part number.

Serious systems therefore run hybrid retrieval — a keyword search and a vector search in parallel, results merged, then a slower and more accurate model re-ranking the top candidates by reading query and document together rather than comparing precomputed points. The first stage is fast and approximate, the second is expensive and precise, and it is only applied to the few dozen documents that survived.

Why it matters for students and researchers

Embeddings are the most useful representational idea in applied machine learning right now, and understanding them changes what you build. They turn unstructured text into something you can do arithmetic on, which is why they show up in search, recommendation, clustering, deduplication and every retrieval pipeline.

They are also the clearest available lesson in reading a metric honestly. Cosine similarity produces a confident number between −1 and 1 for any two texts, including two that have nothing to do with each other, and that number describes distributional usage rather than meaning. Knowing when those two things diverge — negation, antonyms, technical vocabulary, an unfamiliar language — is the difference between a retrieval system that works and one that looks like it works.

There is real research room here too, particularly for Indian languages, where the gap between English and everything else is not a small tuning problem but an open one. A retrieval system that works properly in Hindi, Tamil or Bengali is a more valuable thing to build than another wrapper around an English model.

Frequently asked questions

What is an embedding?

It is a list of numbers representing a piece of text, produced by a model trained so that texts used in similar contexts get similar lists. The numbers act as coordinates, so comparing two texts becomes measuring the distance between two points.

How does semantic search differ from keyword search?

Keyword search finds documents containing your words. Semantic search converts your query into a vector and finds documents whose vectors point in a similar direction, so it can match a description that shares no vocabulary with the document.

Why do vector searches sometimes miss obvious results?

Because searching millions of vectors exactly is too slow, so systems use approximate methods that trade recall for speed and can skip genuinely close matches. Embeddings also handle negation poorly and place opposites near each other, since antonyms appear in similar contexts.

What is retrieval-augmented generation?

It is a system that searches a document collection for relevant passages and gives them to a language model to answer from. Its accuracy depends heavily on the retrieval step — if the wrong passages are found, the model cannot recover.

Should a search system use keywords or embeddings?

Both. Keyword matching is better for exact strings such as codes, names and identifiers, while embeddings handle paraphrase and description. Production systems typically run both and then re-rank the combined results with a more accurate model.