What is an Embedding? Vectors for Meaning, Explained (2026)

Prerequisites & Setup

You need basic Python (functions, virtual environments, pip), the ability to make an HTTP request, and an API key for OpenAI or Azure OpenAI. NumPy for the math.

python -m venv .venv
source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install openai sentence-transformers numpy

Set OPENAI_API_KEY as an environment variable. Get one at platform.openai.com/api-keys, or use Azure OpenAI in EU-region for production.

Why Embeddings Exist (The Story)

Before 2013, computers did not understand meaning. They understood exact strings. "The cat sat on the mat" and "A feline rested on the rug" looked completely unrelated, even though they mean the same thing. Search engines worked on keyword matching: type "refund" and you only found documents containing the literal word "refund."

• 2013 — word2vec (Mikolov et al., Google): showed that words could be mapped to vectors where similar words cluster together. The famous demo: king − man + woman ≈ queen.
• 2018 — BERT (Devlin et al., Google): contextual embeddings. The meaning of "bank" now depended on whether you were talking about a river or a financial institution.
• 2019 — Sentence-BERT (Reimers & Gurevych): made it efficient to embed full sentences, not just words.
• 2024–2026: OpenAI text-embedding-3, Voyage AI, Cohere Embed v3, BGE-M3 — instruction-tuned, multilingual, multimodal embeddings became the standard.

By 2026, embeddings are the foundation of every serious LLM application. Every RAG system computes them. Every recommendation engine uses them. Every semantic search builds on them.

The Mental Model

Five terms you must internalise:

Anatomy: Follow One Sentence

Let us trace one sentence through an embedding model. Input: "What is our refund policy?"

Hands-On Level 1 — Toy Example

The smallest useful embedding example. Twelve lines. In-memory. Embed three sentences, compute cosine similarity, see which two are most alike.

import os, numpy as np
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def embed(text):
    return np.array(client.embeddings.create(input=text, model="text-embedding-3-small").data[0].embedding)

a = embed("Our refund policy allows returns within 30 days.")
b = embed("Customers may return items in the first month.")
c = embed("Shipping is free for orders above 50 euros.")

# Cosine similarity (vectors are already normalised by OpenAI, so dot product = cosine)
print(f"a vs b: {a @ b:.3f}")
print(f"a vs c: {a @ c:.3f}")

Hands-On Level 2 — Realistic

Level 1 has three problems for real use: (a) every restart re-pays the embedding bill, (b) one-by-one calls are slow, (c) no error handling. Level 2 fixes all three with batching, caching, and Sentence Transformers as an offline-capable alternative.

import os, json, hashlib, numpy as np
from pathlib import Path
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
CACHE = Path("./embeddings_cache.jsonl")
MODEL = "text-embedding-3-small"

def _key(text: str) -> str:
    return hashlib.sha256(f"{MODEL}::{text}".encode()).hexdigest()

# Load cache once
cache: dict[str, list[float]] = {}
if CACHE.exists():
    for line in CACHE.read_text().splitlines():
        row = json.loads(line)
        cache[row["key"]] = row["vec"]

def embed_many(texts: list[str]) -> np.ndarray:
    """Embed a batch. Returns (n, 1536) array. Caches by text+model hash."""
    missing = [t for t in texts if _key(t) not in cache]
    if missing:
        # Batch up to 2,048 inputs per call (OpenAI limit)
        for i in range(0, len(missing), 2048):
            batch = missing[i:i+2048]
            resp = client.embeddings.create(input=batch, model=MODEL)
            with CACHE.open("a") as f:
                for text, item in zip(batch, resp.data):
                    cache[_key(text)] = item.embedding
                    f.write(json.dumps({"key": _key(text), "vec": item.embedding}) + "\n")
    return np.array([cache[_key(t)] for t in texts])

# Use it
docs = [
    "Our refund policy allows returns within 30 days.",
    "Customers may return items in the first month.",
    "Shipping is free for orders above 50 euros.",
]
vecs = embed_many(docs)
print(vecs.shape)           # (3, 1536)
print(vecs @ vecs.T)        # 3x3 similarity matrix

Hands-On Level 3 — Production Pattern

In production, embeddings go through a dedicated service with retries, observability, and a real cache backend (Redis or pgvector, not JSON files). Sketch:

# embeddings/service.py
from dataclasses import dataclass
from typing import Sequence
import hashlib, logging, time
from tenacity import retry, stop_after_attempt, wait_exponential

@dataclass
class EmbeddingResult:
    vector: list[float]
    model: str
    cached: bool

class EmbeddingService:
    def __init__(self, client, cache, model: str = "text-embedding-3-small", batch_size: int = 1024):
        self.client = client
        self.cache = cache  # e.g. Redis or pgvector
        self.model = model
        self.batch_size = batch_size
        self.log = logging.getLogger("embeddings")

    def _key(self, text: str) -> str:
        return hashlib.sha256(f"{self.model}::{text}".encode()).hexdigest()

    @retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=8))
    def _call_api(self, batch: list[str]) -> list[list[float]]:
        resp = self.client.embeddings.create(input=batch, model=self.model)
        return [d.embedding for d in resp.data]

    def embed(self, texts: Sequence[str], *, trace_id: str = "") -> list[EmbeddingResult]:
        t0 = time.perf_counter()
        keys = [self._key(t) for t in texts]
        cached = self.cache.mget(keys)
        results: list[EmbeddingResult | None] = [
            EmbeddingResult(vector=v, model=self.model, cached=True) if v else None
            for v in cached
        ]
        missing_idx = [i for i, r in enumerate(results) if r is None]
        if missing_idx:
            # Batch the API calls
            for batch_start in range(0, len(missing_idx), self.batch_size):
                batch_idx = missing_idx[batch_start:batch_start + self.batch_size]
                batch_texts = [texts[i] for i in batch_idx]
                vectors = self._call_api(batch_texts)
                # Write through cache + populate result
                self.cache.mset({self._key(t): v for t, v in zip(batch_texts, vectors)})
                for i, v in zip(batch_idx, vectors):
                    results[i] = EmbeddingResult(vector=v, model=self.model, cached=False)
        # Observability
        n_cached = sum(1 for r in results if r and r.cached)
        self.log.info(
            "embeddings.batch",
            extra={
                "trace_id": trace_id,
                "total": len(texts),
                "cached": n_cached,
                "fresh": len(texts) - n_cached,
                "latency_ms": int((time.perf_counter() - t0) * 1000),
                "model": self.model,
            },
        )
        return results  # type: ignore[return-value]

Eight Pitfalls (Lessons from Production)

Embedding Model Comparison

Cost & Performance Reality

Embeddings are the cheapest part of any RAG system — until they are not. The cost is dominated by how many tokens you embed, not by which model (unless you switch to text-embedding-3-large or run a big self-hosted model).

Latency: a single OpenAI embedding call takes 50–100ms; a batch of 1,000 takes ~500ms (10× faster per item). Self-hosted BGE-M3 on a modern GPU does ~20,000 sentences/second. CPU-only sentence-transformers do ~100 sentences/second.

Going Further

Curated Resources

• Mikolov et al. (2013) — Efficient Estimation of Word Representations in Vector Space (the word2vec paper)
• Reimers & Gurevych (2019) — Sentence-BERT (foundational for sentence embeddings)
• OpenAI embeddings documentation
• MTEB leaderboard (the only fair way to compare embedding models)
• sentence-transformers documentation
• Cohere — What are embeddings? (best plain-English explainer)

Self-Check: Have You Learned It?

def search(query, docs):
    q_vec = embed(query)
    scores = [np.dot(q_vec, embed(d)) for d in docs]
    return sorted(zip(docs, scores), key=lambda x: -x[1])[:5]

Frequently Asked Questions