Embeddings Explained

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meaning get vectors that are close to each other. This is the idea that makes RAG (Retrieval-Augmented Generation) work: instead of matching exact words, your app can find passages that mean the same thing as the user’s question.

Overview: How Embeddings Work

Computers cannot compare meaning directly, but they can compare numbers. An embedding model is a neural network trained so that it turns text into a fixed-length vector, for example 768 or 1536 numbers. The model is trained so that sentences used in similar contexts end up pointing in similar directions in this high-dimensional space.

Think of a map. On a map, two cities that are near each other have close coordinates. An embedding is like coordinates, but with hundreds or thousands of axes instead of two. The sentence “How do I reset my password?” and the sentence “I forgot my login credentials” share almost no words, yet a good model places them close together.

Measuring closeness

To decide how close two vectors are, we use a similarity metric. The most common one is cosine similarity. It measures the angle between two vectors:

• 1.0 means the vectors point in exactly the same direction (very similar meaning).
• 0.0 means they are perpendicular (unrelated).
• -1.0 means they point in opposite directions (rare in practice for text).

The formula is: dot product of the two vectors, divided by the product of their lengths. Many embedding models return vectors already normalised to length 1. For those, cosine similarity equals the plain dot product, which is why vector databases often offer “dot product” as a fast option. Another metric is Euclidean (L2) distance, where a smaller number means closer.

Where embeddings fit in RAG

A RAG pipeline uses embeddings twice:

1. Indexing time: every chunk of your documents is embedded and the vector is stored in a vector store.
2. Query time: the user’s question is embedded with the same model, and the vector store returns the chunks whose vectors are nearest.

Those retrieved chunks are then placed into the prompt for the LLM. If the embeddings are poor, retrieval is poor, and the LLM never sees the right facts. So embedding quality directly limits answer quality.

Key properties to remember

• Fixed dimension: a given model always returns vectors of the same length. You cannot compare vectors from models with different dimensions.
• Model-specific space: even two models with the same dimension have different, incompatible spaces. Always use one model for both documents and queries.
• Context limit: each model accepts only a maximum number of tokens per input. This is one reason we split documents into chunks first.
• Cost and speed: hosted models charge per token and add network latency. Local models cost nothing per call but need your own hardware. Check your provider’s current pricing and limits.

Syntax and Key Concepts: The Embeddings Interface

LangChain defines one common interface, Embeddings, in langchain_core.embeddings. Every provider integration (OpenAI, Hugging Face, Ollama, Google and others) implements it, so you can swap providers by changing one line.

from langchain_core.embeddings import Embeddings

class Embeddings:
    def embed_documents(self, texts: list[str]) -> list[list[float]]: ...
    def embed_query(self, text: str) -> list[float]: ...

Why two methods? Some models treat queries and documents differently, for example by adding a different prefix to each. Keeping two methods lets the integration handle this correctly. Async versions, aembed_documents and aembed_query, also exist.

Examples

Example 1: Offline fake embeddings

You do not need an API key to learn the interface. DeterministicFakeEmbedding returns pseudo-random but repeatable vectors. They carry no real meaning, but they are perfect for testing your pipeline code.

from langchain_core.embeddings import DeterministicFakeEmbedding

embeddings = DeterministicFakeEmbedding(size=8)

v1 = embeddings.embed_query('hello world')
v2 = embeddings.embed_query('hello world')
v3 = embeddings.embed_query('goodbye')

print(len(v1))
print(v1 == v2)
print(v1 == v3)

Output:

8
True
False

The vector has 8 numbers because we set size=8. The same text always gives the same vector, and different text gives a different one. Use this class in unit tests so they run fast, free and without network access. Do not expect it to rank results by meaning.

Example 2: Cosine similarity from scratch

Understanding the maths removes the mystery. Here is cosine similarity in plain Python, applied to three tiny hand-made vectors.

import math

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(y * y for y in b))
    return dot / (norm_a * norm_b)

a = [1, 0, 0]
b = [1, 1, 0]
c = [0, 1, 0]

print(round(cosine_similarity(a, b), 4))
print(round(cosine_similarity(a, c), 4))
print(round(cosine_similarity(b, c), 4))

Output:

0.7071
0.0
0.7071

Vector a and c are perpendicular, so their similarity is 0. Vector b sits at 45 degrees between them, so it scores 1 divided by the square root of 2, which is about 0.7071, against both.

Example 3: A custom Embeddings class with meaningful ranking

To see retrieval work offline, we can write our own tiny model. It counts how often each word from a fixed vocabulary appears. It is far weaker than a neural model, but it follows the exact same interface, so the ranking logic is identical to a real app.

import math
from langchain_core.embeddings import Embeddings

class BagOfWordsEmbeddings(Embeddings):
    def __init__(self, vocab):
        self.vocab = vocab

    def _embed(self, text):
        words = text.lower().split()
        return [float(words.count(w)) for w in self.vocab]

    def embed_documents(self, texts):
        return [self._embed(t) for t in texts]

    def embed_query(self, text):
        return self._embed(text)

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(y * y for y in b))
    return dot / (na * nb) if na and nb else 0.0

vocab = ['cat', 'dog', 'pet', 'car', 'engine', 'fuel']
model = BagOfWordsEmbeddings(vocab)

docs = [
    'the cat is a pet',
    'my dog is a pet',
    'the car engine needs fuel',
]
doc_vectors = model.embed_documents(docs)
query_vector = model.embed_query('cat pet')

scored = [(cosine(query_vector, v), d) for v, d in zip(doc_vectors, docs)]
for score, text in sorted(scored, reverse=True):
    print(f'{score:.2f} {text}')

Output:

1.00 the cat is a pet
0.50 my dog is a pet
0.00 the car engine needs fuel

The query shares both words with the first document, one word (“pet”) with the second, and none with the third. This is exactly what a vector store does: embed the query, score every stored vector, and return the top results. A vector store simply does it with fast indexes instead of a loop.

Example 4: A real provider

With a hosted model, only the class changes. The key is read from an environment variable, never written in code. Here we use the langchain-openai package.

import os
from langchain_openai import OpenAIEmbeddings

assert 'OPENAI_API_KEY' in os.environ, 'Set OPENAI_API_KEY first'

embeddings = OpenAIEmbeddings(model='text-embedding-3-small')

docs = [
    'Our refund window is 30 days.',
    'Shipping takes 3 to 5 business days.',
]
vectors = embeddings.embed_documents(docs)
query = embeddings.embed_query('How long do I have to return an item?')

print(len(vectors), len(vectors[0]), len(query))

Output (illustrative; it needs a valid key and network, and the dimension depends on the model):

2 1536 1536

Two documents give two vectors, and each vector has the same length as the query vector. Model names and prices change often, so check your provider’s documentation for the current list. Other providers follow the same pattern, for example langchain-huggingface or langchain-ollama for local models.

Under the Hood: Step by Step

When you call embed_documents on a hosted model, this is what happens:

1. LangChain batches your list of strings into one or more requests (large lists are split to respect provider limits).
2. The provider tokenises each text (splits it into sub-word pieces) and truncates or rejects inputs that are too long.
3. The neural network processes the tokens and pools the result into one fixed-size vector per text.
4. The vectors come back as lists of floats, in the same order as your input.

Later, a vector store such as Chroma or FAISS stores the vectors with their text. For a query it uses an index (often approximate nearest neighbour search) to find close vectors without comparing against every single one. That approximation is what keeps search fast on millions of chunks.

Common Mistakes

Mistake 1: Passing a single string to embed_documents

vectors = embeddings.embed_documents('Our refund window is 30 days.')

This is wrong because embed_documents expects a list of strings. A plain string is a sequence of characters, so you may embed each letter separately or get an error. Fix it by wrapping the text in a list, or by using embed_query for one string:

vectors = embeddings.embed_documents(['Our refund window is 30 days.'])
vector = embeddings.embed_query('Our refund window is 30 days.')

Mistake 2: Mixing embedding models

index_embeddings = OpenAIEmbeddings(model='text-embedding-3-small')
query_embeddings = OpenAIEmbeddings(model='text-embedding-3-large')

Documents were indexed with one model and queries use another. The dimensions differ (an error) or, with equal dimensions, the spaces are incompatible, so similarity scores are meaningless and results look random. Always store the model name with your index, and use the identical model for queries. If you change the model, re-embed every document.

Mistake 3: Embedding huge, unsplit documents

Sending a whole 50-page PDF as one text either fails on the token limit or produces one blurry vector that matches nothing well. Split documents into focused chunks first (see the text splitting lesson).

Best Practices

• Use the same embedding model for documents and queries, and record its name and version alongside your index.
• Read API keys from environment variables; never hard-code them or commit them.
• Use DeterministicFakeEmbedding or a small custom class in unit tests, so tests are fast, free and repeatable.
• Batch with embed_documents when indexing; do not call embed_query in a loop for thousands of chunks.
• Cache embeddings of unchanged documents so you do not pay to recompute them on every run.
• Evaluate on your own questions. A model that tops a public leaderboard may not be best for your language or domain, including Indian languages and technical text.
• Consider a local open-source model if data is sensitive or cost matters, and check the dimension and token limit of any model before choosing.
• Keep chunks a sensible size so each vector represents one clear idea.

Practice Exercises

1. Using DeterministicFakeEmbedding(size=16), embed three different sentences with embed_documents. Print the number of vectors and the length of each vector. Hint: use a list comprehension with len.
2. Extend the BagOfWordsEmbeddings example with a new document about “dog” and “fuel”, then query with 'dog pet'. Predict the ranking on paper before running it.
3. Write a function top_k(query_vector, doc_vectors, texts, k) that returns the k most similar texts using the cosine function from Example 3. Hint: sort the scored pairs in reverse order and slice.

Summary

• An embedding is a fixed-length vector of numbers that captures the meaning of text.
• Cosine similarity compares the direction of two vectors; closer to 1 means more similar.
• LangChain’s Embeddings interface has embed_documents (list in, list of vectors out) and embed_query (one string in, one vector out).
• Indexing and querying must use the same model; changing models means re-embedding everything.
• Fake or custom embeddings let you build and test RAG pipelines offline without an API key.
• Chunk your documents first, batch your calls, cache results, and evaluate models on your own data.