Tutorial 7: Multi-Turn Conversations

Single-turn prompts are easy. Multi-turn conversations introduce context management — how much history to keep, what to summarize, when to reset. This tutorial walks through it.

What You'll Build

A multi-turn conversation handler with sensible context management.

Step 1: Understand Context Windows (5 min)

Each model has a maximum context (tokens it can attend to):

• Claude Sonnet: 200k tokens (or 1M with extended context)

• GPT-4: 128k tokens

• Older models: 4k-32k

Long conversations approach the limit. Plan for it.

Step 2: The Basic Pattern (10 min)

def chat(history, user_message):
    messages = history + [{"role": "user", "content": user_message}]
    
    response = client.messages.create(
        model="claude-sonnet-4-6",
        system=system_prompt,
        messages=messages
    )
    
    assistant_message = response.content[0].text
    new_history = messages + [{"role": "assistant", "content": assistant_message}]
    
    return assistant_message, new_history

Each turn appends to history. History grows.

Step 3: Token Counting (15 min)

Track token usage:

import tiktoken  # or anthropic's tokenizer

def count_tokens(messages, model="gpt-4"):
    encoding = tiktoken.encoding_for_model(model)
    total = 0
    for msg in messages:
        total += len(encoding.encode(msg["content"]))
    return total

# Before each call
tokens = count_tokens(history)
if tokens > 0.8 * MAX_TOKENS:
    # Need to manage context
    history = manage_context(history)

Know how close to the limit you are.

Step 4: Truncation Strategies (20 min)

When approaching the limit, drop oldest turns:

def truncate_history(history, target_tokens):
    # Keep recent turns; drop oldest
    while count_tokens(history) > target_tokens and len(history) > 2:
        # Drop the oldest user/assistant pair
        history = history[2:]
    return history

Simple but loses early context. May confuse the model about earlier parts.

Step 5: Summarization Strategies (varies)

Instead of dropping, summarize older turns:

def summarize_history(history, model="claude-sonnet-4-6"):
    if len(history) < 10:
        return history
    
    # Take the oldest half; summarize
    old_messages = history[:len(history)//2]
    recent = history[len(history)//2:]
    
    summary_prompt = f"""
    Summarize this conversation in 2-3 sentences, focusing on:
    - User's goals
    - Key information shared
    - Decisions made
    
    Conversation:
    {format_messages(old_messages)}
    """
    
    summary = call_llm(summary_prompt)
    
    return [
        {"role": "system", "content": f"Earlier in conversation: {summary}"},
        *recent
    ]

Preserves the gist; saves tokens.

Step 6: Hybrid Approach (15 min)

Often best:

• Keep the system prompt

• Keep the last N turns verbatim

• Summarize the middle

• Drop the very oldest

def manage_context(history, n_recent=10):
    if len(history) <= n_recent:
        return history
    
    middle = history[2:-n_recent]  # everything except first 2 and last N
    recent = history[-n_recent:]
    
    summary = summarize(middle)
    
    return [
        history[0],  # opening turn
        history[1],
        {"role": "system", "content": f"[Previous conversation summarized]: {summary}"},
        *recent
    ]

Step 7: Stateful Context Across Sessions (varies)

For ongoing relationships (e.g., a returning user), persist context:

class Conversation:
    def __init__(self, user_id):
        self.user_id = user_id
        self.history = self.load_history(user_id)
        self.user_facts = self.load_facts(user_id)
    
    def send(self, message):
        # Include known facts in system prompt
        system = f"""
        {base_system_prompt}
        
        Known facts about this user:
        {self.user_facts}
        """
        
        response = call_llm(system, self.history + [message])
        
        # Update history
        self.history.append({"role": "user", "content": message})
        self.history.append({"role": "assistant", "content": response})
        self.save_history()
        
        # Extract any new facts
        new_facts = self.extract_facts(message, response)
        self.user_facts.update(new_facts)
        self.save_facts()
        
        return response

The model has context across sessions without needing the full history every time.

Step 8: Reset Strategies (10 min)

Sometimes context becomes confused. When to reset:

• User explicitly says "let's start over"

• Topic changes dramatically

• Previous context is producing wrong outputs

• Conversation exceeds a sensible length

Reset is just: new conversation, possibly with a brief "previous context" summary.

Step 9: Test the Limits (15 min)

Try edge cases:

• 100-turn conversation about one topic

• Conversation that switches topics mid-way

• Conversation that exceeds the context window

Where does the model lose the thread? That's information about your context management.

Step 10: Monitor in Production (ongoing)

Track:

• Average conversation length

• Conversations exceeding context limit

• User-reported issues with model "forgetting"

Each is a signal about whether context management is working.

What You Just Did

You handled the realities of multi-turn conversation. Context stays manageable; the model remembers what matters; the conversation doesn't fall over at length.

Common Failure Modes

Unbounded history. Token cost explodes; context runs out.

Aggressive truncation. Drop too much; model forgets key info.

Bad summaries. Summary misses key info; "summarized" conversation worse than fragments.

No state across sessions. Each session starts cold.

Never reset. Confused context persists; model produces increasingly wrong outputs.