Last week I had a conversation with a client who wanted search on their website. Not the kind where you type a product name and get an exact match — they wanted users to describe what they were looking for in plain English and get relevant results back. “Something for a rainy day with the kids” should surface indoor activities, even though those words appear nowhere in the listing.
I told them this was a solved problem. Then I tried to explain how it was solved, and realised that embedding models — the thing that makes it all work — are genuinely difficult to explain without either oversimplifying or losing people in linear algebra.
This is my attempt at the middle ground.
The Problem: Computers Don’t Understand Meaning
A traditional database is brilliant at exact comparisons. Is this string equal to that string? Is this number greater than that number? These operations are fast, well-understood, and the backbone of every search box that matches what you typed against what’s stored.
But language doesn’t work that way. “Affordable” and “budget-friendly” mean the same thing. “Bank” means completely different things depending on context. “I love this product” and “This product is not what I expected” are both five words long, share three of them, and express opposite sentiments.
Keyword search handles some of this with tricks — stemming, synonyms, fuzzy matching. But these are patches over a fundamental limitation: the computer is matching characters, not meaning.
Embedding models bridge this gap. They translate human language into a format where meaning is measurable.
What an Embedding Actually Is
An embedding is a list of numbers — typically between 384 and 3,072 of them — that represents the meaning of a piece of text. Not the words. Not the grammar. The meaning.
Think of it as coordinates. A physical address gives you two numbers (latitude, longitude) that pin a location in 2D space. Nearby addresses have similar coordinates. An embedding gives you hundreds of numbers that pin a piece of text in meaning-space. Texts with similar meaning have similar embeddings.
flowchart TD
subgraph INPUT["Text inputs"]
S1["'The restaurant closes at 9pm'"]
S2["'Last orders are at 8:30 in the evening'"]
S3["'The chef trained in Lyon for three years'"]
end
subgraph MODEL["Embedding Model"]
M["Transforms text into<br/>high-dimensional vectors"]
end
subgraph OUTPUT["Embedding vectors (simplified to 3D)"]
V1["[0.82, 0.91, 0.15]"]
V2["[0.79, 0.88, 0.18]"]
V3["[0.23, 0.11, 0.87]"]
end
S1 --> M --> V1
S2 --> M --> V2
S3 --> M --> V3
V1 -.- |"Close together:<br/>similar meaning"| V2
V3 -.- |"Far apart:<br/>different topic"| V1
style V1 fill:#e2f4e6,stroke:#27ae60
style V2 fill:#e2f4e6,stroke:#27ae60
style V3 fill:#fff3d6,stroke:#e08e0bThe first two sentences are about the same thing — when the restaurant stops serving. The embedding model maps them to nearby points, despite sharing almost no words. The third sentence is about the chef’s background — semantically unrelated — and lands far away.
Real embeddings have hundreds of dimensions, not three. Each dimension captures some aspect of meaning that we can’t easily name. Some dimensions might correlate with tense, others with topic, others with formality. The model learns these dimensions during training, and no human ever specifies what they should be.
How Embedding Models Learn
An embedding model is a neural network that’s been trained on vast amounts of text — billions of sentence pairs, paragraphs, and documents. During training, it learns to push similar texts close together and dissimilar texts apart.
The training process looks roughly like this:
- Show the model two pieces of text that are related (a question and its answer, a paragraph and its summary, two paraphrases of the same idea).
- The model produces embeddings for both.
- Measure how close the embeddings are.
- If they’re not close enough, adjust the model’s weights to bring them closer.
- Repeat with billions of examples.
After enough training, the model has internalised patterns about language that let it place any new text — even text it’s never seen before — in the right neighbourhood of meaning-space.
flowchart LR
subgraph TRAINING["Training (billions of examples)"]
direction TB
PAIR["Text pair:<br/>'How do I reset my password?'<br/>'Steps to change your login credentials'"]
PAIR --> ENCODE["Model encodes both texts"]
ENCODE --> COMPARE["Measure distance"]
COMPARE --> UPDATE["Adjust weights to<br/>bring similar texts closer"]
end
subgraph RESULT["After training"]
direction TB
NEW["Any new text"] --> EMBED["Model produces embedding"]
EMBED --> SPACE["Lands in the right<br/>neighbourhood of meaning-space"]
end
TRAINING --> RESULT
style TRAINING fill:#f0f4ff,stroke:#4f46e5
style RESULT fill:#e2f4e6,stroke:#27ae60This is fundamentally different from a keyword index. A keyword index is constructed mechanically from the exact words in your documents. An embedding model has learned what words and phrases mean in relation to each other. It generalises to text it’s never encountered.
The Models You’ll Actually Choose Between
Not all embedding models are equal. They differ in quality, speed, dimension count, and what they’re good at. Here’s the landscape as of mid-2026:
| Model | Dimensions | Max Tokens | Strengths | Provider |
|---|---|---|---|---|
| Amazon Titan Embeddings v2 | 256 / 512 / 1,024 | 8,192 | Good all-rounder, native AWS integration, variable dimensions | AWS Bedrock |
| Cohere Embed v3 | 1,024 | 512 | Excellent multilingual, strong for search | Cohere / Bedrock |
| OpenAI text-embedding-3-large | 3,072 | 8,191 | High quality, adjustable dimensions | OpenAI |
| OpenAI text-embedding-3-small | 1,536 | 8,191 | Good balance of quality and cost | OpenAI |
| BGE-large-en-v1.5 | 1,024 | 512 | Open source, self-hostable, competitive quality | Hugging Face |
| E5-mistral-7b-instruct | 4,096 | 32,768 | Long context, instruction-following embeddings | Hugging Face |
Dimensions matter. More dimensions means the model can capture finer distinctions in meaning, but also means larger storage and slower similarity searches. For most applications, 1,024 dimensions is the sweet spot. Going below 512 noticeably hurts retrieval quality for nuanced queries.
Token limits matter. If your model handles 512 tokens and you feed it a 2,000-word document, it truncates. You’ll lose meaning from the truncated portion. Either chunk your documents to fit within the limit or choose a model with a larger context window.
Multilingual matters if your users write in multiple languages. A good multilingual model will place “opening hours” and “orari di apertura” close together. A monolingual English model won’t.
What Happens at Query Time
When someone searches your system, the process is:
- Their query goes through the same embedding model used to index your data.
- The resulting vector is compared against all stored vectors.
- The most similar vectors are returned, ranked by similarity score.
sequenceDiagram
participant User
participant App
participant Model as Embedding Model
participant DB as Vector Store
User->>App: "family activities for rainy days"
App->>Model: Encode query
Model-->>App: [0.45, 0.72, 0.33, ...]
App->>DB: Find top 5 nearest vectors
Note over DB: Compares against all stored<br/>embeddings using cosine similarity
DB-->>App: 1. Indoor play centre (0.89)<br/>2. Museum family pass (0.85)<br/>3. Cooking class for kids (0.81)<br/>4. Trampoline park (0.78)<br/>5. Cinema listings (0.74)
App-->>User: Display ranked resultsNotice that “rainy days” doesn’t appear in any of the results. The embedding model understood that rainy days implies indoor activities, and surfaced results accordingly. No synonym list. No manual tagging. The understanding is baked into the vectors.
The same model must be used for indexing and querying. Different models produce vectors in different spaces. A vector from Titan Embeddings and a vector from OpenAI’s model can’t be meaningfully compared — they’d be coordinates on different maps.
The Distance Between Meaning: Similarity Metrics
Two vectors are “similar” when they point in roughly the same direction. The standard measure is cosine similarity: the cosine of the angle between the two vectors.
- 1.0 — identical direction (same meaning)
- 0.7 to 0.9 — strongly related (paraphrases, same-topic texts)
- 0.4 to 0.7 — loosely related (same domain, different aspect)
- Below 0.3 — unrelated
In practice, you set a similarity threshold. For a customer support chatbot, you might only return results above 0.75 — better to say “I don’t know” than return irrelevant context. For a discovery search (“show me interesting things”), 0.5 might be fine.
The threshold is something you tune with real queries. There’s no universal correct value.
Where Embeddings Go Wrong
Embedding models aren’t magic. They fail in predictable ways:
Negation is hard. “This product is great” and “This product is not great” produce surprisingly similar embeddings. The model captures the topic (product quality) more strongly than the negation. This is improving with newer models but remains a known weakness.
Specificity is lost at short lengths. The embedding for “Python” alone could mean the programming language, the snake, or the Monty Python comedy troupe. With more context — “Python web framework” — the model disambiguates. Short queries produce less precise embeddings.
Domain jargon needs domain models. A general-purpose embedding model might not distinguish between legal terms, medical terminology, or engineering concepts as well as a model fine-tuned on domain-specific text. If your application is in a specialised domain, test general models before assuming they’ll work.
Outdated training data. An embedding model trained in 2023 might not handle concepts, brands, or terminology that emerged in 2025. Unlike language models that get frequent updates, embedding models are often trained once and used for years.
Embeddings Beyond Text
Everything we’ve discussed so far has been about text. But the same principle — turn data into vectors, compare vectors to find similarity — applies to images, audio, and video too.
Image Embeddings and Alternatives
Image embedding models convert images into vectors, just like text embedding models convert sentences. Models like CLIP create joint embeddings for images and text in the same vector space — meaning you can search for images using text descriptions (“sunset over mountains”) or find images similar to another image.
This is how reverse image search works, how e-commerce platforms find “products that look like this photo,” and how content moderation systems detect similar images at scale.
But embeddings aren’t the only way to search images. Content-Based Image Retrieval (CBIR) is an older approach that analyses visual features directly — colour distributions, shapes, textures, edge patterns — without converting anything into a semantic vector. CBIR systems compare images by their visual characteristics rather than their meaning.
When to use which:
- Use embeddings when you need semantic understanding — matching images to natural language queries, finding conceptually similar images, or clustering images by topic. “Find me photos of happy families outdoors” requires the model to understand what those concepts look like.
- Use CBIR when visual characteristics alone are enough — finding images with similar colour palettes, matching textures in manufacturing quality control, or detecting near-duplicate images. These tasks don’t need semantic understanding; they need pixel-level comparison.
In practice, embeddings are the default choice for most modern applications because they handle both visual and semantic similarity. CBIR is simpler and more efficient when you genuinely only care about low-level visual features.
Video Embeddings: The Harder Problem
Video is where things get more complex. You can embed video content, but not as a single operation like text or images. A video is a sequence of frames with temporal structure, audio, and often spoken dialogue — multiple data streams that all carry meaning.
The most practical approaches break the problem down:
- Key frame extraction. Sample frames at regular intervals — one every few seconds — and embed each frame individually. This captures visual content but loses temporal flow. Good for finding “videos that contain a scene like this.”
- Text transcription. Extract audio, transcribe it to text, and embed the transcript. This captures the spoken content and is often the richest source of meaning. Good for finding “videos that discuss this topic.”
- Hybrid approach. Combine key frame embeddings with transcript embeddings. This gives you both visual and semantic search — the system can find a video because it shows a specific diagram and discusses a specific concept. This is usually the best approach.
flowchart TD
V["Video"] --> F["Extract key frames<br/>(1 per N seconds)"]
V --> T["Transcribe audio<br/>to text"]
F --> FE["Image embedding<br/>model"]
T --> TE["Text embedding<br/>model"]
FE --> VS["Vector store<br/>(visual index)"]
TE --> VS2["Vector store<br/>(semantic index)"]
Q["Search query"] --> VS
Q --> VS2
VS --> MERGE["Merge and rank<br/>results"]
VS2 --> MERGE
MERGE --> R["Matched video<br/>segments"]
style F fill:#fef3c7,stroke:#d97706
style T fill:#fef3c7,stroke:#d97706
style FE fill:#dbeafe,stroke:#2563eb
style TE fill:#dbeafe,stroke:#2563eb
style VS fill:#dcfce7,stroke:#16a34a
style VS2 fill:#dcfce7,stroke:#16a34aThere are also dedicated video retrieval frameworks and models being developed, but the hybrid approach — key frames for visual cues, transcription for semantics — is the most reliable and widely used in production today.
The Common Thread
Whether you’re embedding text, images, audio, or video, the underlying mathematics is identical. You’re converting data into numerical vectors and comparing those vectors to find similarity. The encoder changes — a text model, an image model, a multimodal model — but the search operation is always the same: nearest-neighbour lookup in a vector space.
And in every case, when you’re storing and searching these vectors at scale, you end up needing a vector database. The data type that produced the vectors doesn’t matter to the database — it just sees numbers and measures distances.
What I’d Tell You Before You Start Building
If you’re about to add semantic search or RAG to a product, here’s the decision tree I’d walk through:
First: do you actually need embeddings? If your data is structured (prices, dates, categories) and your queries are precise, a traditional database with good filtering is simpler and more predictable. Embeddings shine for unstructured text where users express intent in natural language.
Second: does your data fit in the LLM’s context window? For small datasets — a few hundred FAQ entries, a product catalogue under 50 pages — you can skip embeddings entirely and inject all the content into the language model’s prompt. We do this for our small business chatbots. No vector database, no retrieval step, no chunking to get wrong. When the data grows beyond the context window, add retrieval.
Third: choose your embedding model based on your constraints. If you’re on AWS, Titan Embeddings v2 avoids cross-provider data transfer. If you need multilingual, Cohere Embed v3 is strong. If you want to self-host and control costs, BGE or E5 on your own infrastructure. Don’t default to the most expensive model — test with your actual data and see where quality plateaus.
For the vector database side, Pinecone’s free tier gives you enough to build and test a real system — up to 5 million vectors on a single index, with no credit card required. That’s more than enough for a prototype or a small production workload. I’ve seen teams spend weeks evaluating vector databases when they could have had a working system on Pinecone’s free tier in an afternoon. Start there, prove the concept works with your data, and migrate to a managed solution (or self-hosted) only when you have a concrete reason to.
Fourth: invest in evaluation before you invest in infrastructure. Build a test set of 50 to 100 queries with known good answers. Run your embedding + retrieval pipeline against it. Measure how often the correct document appears in the top 3 results. This number — retrieval recall — is the single most important metric for a RAG system. If retrieval is bad, no amount of prompt engineering on the generation side will fix it.
Fifth: keep the embedding model and the indexed data in sync. If you re-embed your data with a new model version, you must re-embed all of it. You cannot mix vectors from different model versions in the same collection. This is easy to forget and painful to debug when search quality silently degrades.
Embedding models are one of those technologies that sound abstract until you use them, and then they feel obvious. They’re the reason a search box can understand what you mean, not just what you typed. They’re the reason a chatbot can find the right paragraph in a thousand-page manual. And they’re the reason AI systems can work with your specific data without retraining a billion-parameter model.
The maths is deep. The practical application is straightforward. Turn text into numbers. Compare numbers. Return the closest match. Everything else is engineering around that core operation.