Tutorial 6: Add Conversation Memory

Each LLM call is stateless. Conversation memory makes the agent remember. This tutorial walks through the patterns.

What You'll Build

A system that persists conversation history and selectively includes relevant past context in new turns.

Step 1: Store Full Conversation (15 min)

The simplest pattern:

# Schema
CREATE TABLE conversations (
    id UUID PRIMARY KEY,
    user_id TEXT,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE TABLE messages (
    id UUID PRIMARY KEY,
    conversation_id UUID REFERENCES conversations(id),
    role TEXT,
    content TEXT,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    INDEX (conversation_id, created_at)
);

Store every turn.

Step 2: Send Recent History (10 min)

For active conversations:

def chat(conversation_id, user_message):
    # Get last 20 turns
    history = db.query("""
        SELECT role, content FROM messages
        WHERE conversation_id = %s
        ORDER BY created_at DESC
        LIMIT 20
    """, [conversation_id])[::-1]  # Reverse to chronological
    
    messages = [
        {"role": m["role"], "content": m["content"]} 
        for m in history
    ]
    messages.append({"role": "user", "content": user_message})
    
    response = client.messages.create(
        model="claude-sonnet-4-6",
        messages=messages,
        system=SYSTEM_PROMPT
    )
    
    # Save the new turn
    save_message(conversation_id, "user", user_message)
    save_message(conversation_id, "assistant", response.content[0].text)
    
    return response.content[0].text

Last 20 turns is workable for many use cases.

Step 3: Summarize Older Turns (varies)

When history gets long:

def summarize_old_turns(messages):
    if len(messages) < 30:
        return messages
    
    old = messages[:-20]  # Everything except last 20
    recent = messages[-20:]
    
    summary_prompt = f"""
    Summarize this conversation in 2-3 paragraphs, focusing on:
    - User's goals and interests
    - Key facts shared
    - Important context
    
    Conversation:
    {format_messages(old)}
    """
    
    summary = call_llm(summary_prompt)
    
    return [
        {"role": "system", "content": f"Earlier in this conversation: {summary}"},
        *recent
    ]

Preserves the gist; saves tokens.

Step 4: Extract Persistent Facts (varies)

Some things should persist across sessions:

def extract_facts(message_text):
    """Extract durable facts from a message."""
    
    prompt = f"""
    Extract any durable facts about the user from this message.
    Return JSON list. Examples: name, role, preferences, important context.
    
    Message: {message_text}
    """
    
    facts = json.loads(call_llm(prompt))
    return facts

# Store
for fact in extract_facts(user_message):
    db.execute("""
        INSERT INTO user_facts (user_id, fact, source_message_id, created_at)
        VALUES (%s, %s, %s, NOW())
    """, [user_id, fact, message_id])

User facts live across all conversations.

Step 5: Inject Facts Into Prompts (10 min)

For new sessions:

def chat_with_user_context(user_id, conversation_id, user_message):
    # Pull stable user facts
    facts = db.query("""
        SELECT fact FROM user_facts WHERE user_id = %s
    """, [user_id])
    
    facts_text = "\n".join(f["fact"] for f in facts)
    
    system_with_context = f"""
    {BASE_SYSTEM_PROMPT}
    
    Known about this user:
    {facts_text}
    """
    
    # ... rest of chat logic

Each conversation starts with relevant user context.

Step 6: Episodic Memory (advanced, varies)

For complex agents, separate types of memory:

• Working memory: the current conversation (last N turns)

• Episodic memory: what happened in past conversations

• Semantic memory: facts and learned patterns

class AgentMemory:
    def __init__(self, user_id):
        self.user_id = user_id
        self.working = []  # Current conversation
        self.facts = self.load_facts()  # Long-term
        self.episodes = self.load_episodes()  # Past conversations
    
    def relevant_episodes(self, current_topic, top_k=3):
        # Embed-similarity search over past conversation summaries
        return search_episodes(self.user_id, current_topic, top_k)

Step 7: Forget (15 min)

Privacy and noise reduction:

def forget(user_id, scope="all"):
    if scope == "all":
        db.execute("DELETE FROM messages WHERE user_id = %s", [user_id])
        db.execute("DELETE FROM user_facts WHERE user_id = %s", [user_id])
    elif scope == "conversation":
        # Forget specific conversation
        pass
    elif scope == "after_date":
        # GDPR-style; forget after a date
        pass

Users may request deletion. Plan for it.

Step 8: Memory Refresh (10 min)

User says "actually, my name is Pat, not Sam":

def update_fact(user_id, old_fact, new_fact):
    db.execute("""
        UPDATE user_facts 
        SET fact = %s, updated_at = NOW()
        WHERE user_id = %s AND fact = %s
    """, [new_fact, user_id, old_fact])

Detect updates from conversation; correct stored facts.

Step 9: Test the Memory (varies)

# Test scenarios
def test_remembers_name():
    chat(conv_id, "My name is Pat")
    response = chat(conv_id, "What's my name?")
    assert "Pat" in response

def test_remembers_across_sessions():
    chat(conv_1, "I work in healthcare")
    response = chat(conv_2, "What field do I work in?")  # New conversation
    assert "healthcare" in response.lower()

Memory that doesn't work is theater. Test it.

Step 10: Monitor Memory Quality (ongoing)

In production:

• Random sample: does memory work?

• User feedback: "you forgot what I said"

• Conversation length: does the model lose the thread at length?

Pattern problems suggest memory architecture changes.

What You Just Did

You added memory across turns and sessions. The agent feels persistent. Users get value from the relationship over time.

Common Failure Modes

Unbounded history. Token cost explodes; context overflows.

Lossy summarization. Important context lost in summary.

Stale facts. User updated something; old fact still in prompt.

No forget mechanism. Privacy issue; data accumulates.

Memory bugs invisible. Users complain; no monitoring catches it.