Add Embeddings and Search — Building with LLMs, Part 4
Updated: 5 days ago
Updated, 4 October 2026. Embedding prices checked against OpenAI's pricing page and dated.
Building with LLMs · Part 4
Embeddings convert text into vectors. Similar text → similar vectors. This enables semantic search: finding by meaning, not just keywords. This tutorial builds it.
What You'll Build
A working semantic search over a corpus of your documents.
Step 1: Pick the Embedding Model (5 min)
Common choices:
OpenAI text-embedding-3-small: $0.02 per million tokens; 1536 dimensions
OpenAI text-embedding-3-large: $0.13 per million tokens; 3072 dimensions
Cohere Embed: a hosted alternative; compare current pricing and retrieval benchmarks before you choose
Local (sentence-transformers): free to run; you pay for the hosting
OpenAI prices as of October 2026, from OpenAI's pricing page. Check it before you budget.
For starting, OpenAI small is cheap and good enough.
Step 2: Generate Embeddings (15 min)
from openai import OpenAI
client = OpenAI()
def embed(text: str) -> list[float]:
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
# Single query
vector = embed("How do I reset my password?")
print(len(vector)) # 1536
Each call: text → vector of 1536 floats.
Step 3: Embed Your Corpus (30 min)
For your documents:
documents = [
{"id": "doc1", "text": "To reset your password, click 'Forgot password' on the sign-in page..."},
{"id": "doc2", "text": "Update your billing information in account settings..."},
# ... more docs
]
# Embed each
for doc in documents:
doc["embedding"] = embed(doc["text"])
# Store somewhere
For larger docs, you'll need chunking (next tutorials). For now, assume each doc fits.
Step 4: Pick a Vector Store (10 min)
Options:
Postgres + pgvector: simple, in your existing DB
Pinecone: managed; easy
Weaviate: open source; feature-rich
Chroma: lightweight; local development
Qdrant: open source; production-ready
For most projects starting, Postgres + pgvector is the right answer (one less service).
Step 5: Postgres + pgvector Setup (15 min)
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE documents (
id TEXT PRIMARY KEY,
text TEXT,
embedding vector(1536)
);
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);
The index makes similarity search fast.
Step 6: Store Embeddings (10 min)
import psycopg2
from pgvector.psycopg2 import register_vector
conn = psycopg2.connect(...)
register_vector(conn)
with conn.cursor() as cur:
for doc in documents:
cur.execute(
"INSERT INTO documents (id, text, embedding) VALUES (%s, %s, %s)",
(doc["id"], doc["text"], doc["embedding"])
)
conn.commit()
Step 7: Query (15 min)
def search(query: str, top_k: int = 5):
query_embedding = embed(query)
with conn.cursor() as cur:
cur.execute("""
SELECT id, text, 1 - (embedding <=> %s) as similarity
FROM documents
ORDER BY embedding <=> %s
LIMIT %s
""", (query_embedding, query_embedding, top_k))
return cur.fetchall()
results = search("How do I change my password?")
for id, text, similarity in results:
print(f"{similarity:.3f}: {text[:100]}...")
<=> is the cosine distance operator. Lower distance = more similar.
Step 8: Handle Updates (15 min)
When documents change, re-embed:
def upsert_document(doc):
embedding = embed(doc["text"])
with conn.cursor() as cur:
cur.execute("""
INSERT INTO documents (id, text, embedding)
VALUES (%s, %s, %s)
ON CONFLICT (id) DO UPDATE SET
text = EXCLUDED.text,
embedding = EXCLUDED.embedding
""", (doc["id"], doc["text"], embedding))
conn.commit()
Documents stay current.
Step 9: Evaluate Quality (30 min)
Build an eval set:
TEST_QUERIES = [
{"query": "reset password", "expected_id": "doc1"},
{"query": "update billing", "expected_id": "doc2"},
# ...
]
correct = 0
for case in TEST_QUERIES:
results = search(case["query"], top_k=3)
if case["expected_id"] in [r[0] for r in results]:
correct += 1
print(f"Recall@3: {correct/len(TEST_QUERIES):.2%}")
Measure recall (did the right doc appear in top results). Use to compare embedding models or improvements.
Step 10: Hybrid Search (advanced, varies)
For best quality, combine semantic + keyword:
def hybrid_search(query, top_k=5):
# Semantic
semantic_results = search(query, top_k=20)
# Keyword (BM25 or simple LIKE)
keyword_results = keyword_search(query, top_k=20)
# Combine scores
combined = {}
for id, text, score in semantic_results:
combined[id] = {"semantic_score": score, "keyword_score": 0, "text": text}
for id, text, score in keyword_results:
if id in combined:
combined[id]["keyword_score"] = score
else:
combined[id] = {"semantic_score": 0, "keyword_score": score, "text": text}
# Weighted combination
for id in combined:
combined[id]["score"] = (
combined[id]["semantic_score"] * 0.7 +
combined[id]["keyword_score"] * 0.3
)
return sorted(combined.items(), key=lambda x: -x[1]["score"])[:top_k]
Often better than either alone.
What You Just Did
You built semantic search over a corpus. Foundation for RAG (next tutorial).
Common Failure Modes
Wrong embedding model. Different models, different vector spaces. Don't mix.
No re-embedding on updates. Docs change; embeddings stale.
No evaluation. Quality drift invisible.
Vector dimension mismatch. Schema vs. actual; cryptic errors.
Single tool obsession. Pure semantic misses exact-match cases. Hybrid usually wins.
Continue the Building with LLMs path
Previous — Part 3: Stream Responses to a UI
For the reasoning behind this step-by-step guide, read Embedding Strategies That Make or Break Retrieval.
Part of the Building with LLMs learning path.
