Tutorial 4: Add Embeddings and Search

Contributor
4 days ago
3 min read

Embeddings convert text into vectors. Similar text → similar vectors. This enables semantic search: finding by meaning, not just keywords. This tutorial builds it.

What You'll Build

A working semantic search over a corpus of your documents.

Step 1: Pick the Embedding Model (5 min)

Common choices:

OpenAI text-embedding-3-small: $0.02/M tokens; 1536 dims
OpenAI text-embedding-3-large: $0.13/M tokens; 3072 dims
Cohere Embed v3: competitive
Local (sentence-transformers): free; requires hosting

For starting, OpenAI small is cheap and good enough.

Step 2: Generate Embeddings (15 min)

from openai import OpenAI
client = OpenAI()

def embed(text: str) -> list[float]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    return response.data[0].embedding

# Single query
vector = embed("How do I reset my password?")
print(len(vector))  # 1536

Each call: text → vector of 1536 floats.

Step 3: Embed Your Corpus (30 min)

For your documents:

documents = [
    {"id": "doc1", "text": "To reset your password, click 'Forgot password' on the sign-in page..."},
    {"id": "doc2", "text": "Update your billing information in account settings..."},
    # ... more docs
]

# Embed each
for doc in documents:
    doc["embedding"] = embed(doc["text"])

# Store somewhere

For larger docs, you'll need chunking (next tutorials). For now, assume each doc fits.

Step 4: Pick a Vector Store (10 min)

Options:

Postgres + pgvector: simple, in your existing DB
Pinecone: managed; easy
Weaviate: open source; feature-rich
Chroma: lightweight; local development
Qdrant: open source; production-ready

For most projects starting, Postgres + pgvector is the right answer (one less service).

Step 5: Postgres + pgvector Setup (15 min)

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id TEXT PRIMARY KEY,
    text TEXT,
    embedding vector(1536)
);

CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

The index makes similarity search fast.

Step 6: Store Embeddings (10 min)

import psycopg2
from pgvector.psycopg2 import register_vector

conn = psycopg2.connect(...)
register_vector(conn)

with conn.cursor() as cur:
    for doc in documents:
        cur.execute(
            "INSERT INTO documents (id, text, embedding) VALUES (%s, %s, %s)",
            (doc["id"], doc["text"], doc["embedding"])
        )
conn.commit()

Step 7: Query (15 min)

def search(query: str, top_k: int = 5):
    query_embedding = embed(query)
    
    with conn.cursor() as cur:
        cur.execute("""
            SELECT id, text, 1 - (embedding <=> %s) as similarity
            FROM documents
            ORDER BY embedding <=> %s
            LIMIT %s
        """, (query_embedding, query_embedding, top_k))
        return cur.fetchall()

results = search("How do I change my password?")
for id, text, similarity in results:
    print(f"{similarity:.3f}: {text[:100]}...")

<=> is the cosine distance operator. Lower distance = more similar.

Step 8: Handle Updates (15 min)

When documents change, re-embed:

def upsert_document(doc):
    embedding = embed(doc["text"])
    with conn.cursor() as cur:
        cur.execute("""
            INSERT INTO documents (id, text, embedding) 
            VALUES (%s, %s, %s)
            ON CONFLICT (id) DO UPDATE SET
                text = EXCLUDED.text,
                embedding = EXCLUDED.embedding
        """, (doc["id"], doc["text"], embedding))
    conn.commit()

Documents stay current.

Step 9: Evaluate Quality (30 min)

Build an eval set:

TEST_QUERIES = [
    {"query": "reset password", "expected_id": "doc1"},
    {"query": "update billing", "expected_id": "doc2"},
    # ...
]

correct = 0
for case in TEST_QUERIES:
    results = search(case["query"], top_k=3)
    if case["expected_id"] in [r[0] for r in results]:
        correct += 1

print(f"Recall@3: {correct/len(TEST_QUERIES):.2%}")

Measure recall (did the right doc appear in top results). Use to compare embedding models or improvements.

Step 10: Hybrid Search (advanced, varies)

For best quality, combine semantic + keyword:

def hybrid_search(query, top_k=5):
    # Semantic
    semantic_results = search(query, top_k=20)
    
    # Keyword (BM25 or simple LIKE)
    keyword_results = keyword_search(query, top_k=20)
    
    # Combine scores
    combined = {}
    for id, text, score in semantic_results:
        combined[id] = {"semantic_score": score, "keyword_score": 0, "text": text}
    for id, text, score in keyword_results:
        if id in combined:
            combined[id]["keyword_score"] = score
        else:
            combined[id] = {"semantic_score": 0, "keyword_score": score, "text": text}
    
    # Weighted combination
    for id in combined:
        combined[id]["score"] = (
            combined[id]["semantic_score"] * 0.7 +
            combined[id]["keyword_score"] * 0.3
        )
    
    return sorted(combined.items(), key=lambda x: -x[1]["score"])[:top_k]

Often better than either alone.

What You Just Did

You built semantic search over a corpus. Foundation for RAG (next tutorial).

Common Failure Modes

Wrong embedding model. Different models, different vector spaces. Don't mix.

No re-embedding on updates. Docs change; embeddings stale.

No evaluation. Quality drift invisible.

Vector dimension mismatch. Schema vs. actual; cryptic errors.

Single tool obsession. Pure semantic misses exact-match cases. Hybrid usually wins.

ShiftQuality