Tutorial 2: Build a Single-Tool Agent

Contributor
Jun 8
3 min read

A useful agent often starts with one well-chosen tool. This tutorial builds one that does real work.

What You'll Build

An agent with a single tool — search — that can answer questions by searching, refining queries, and synthesizing.

Step 1: Pick the Tool (5 min)

A search tool over your knowledge base, or any single capability that's useful when called multiple times:

def search(query: str, top_k: int = 5) -> list[dict]:
    """Search knowledge base; return relevant chunks."""
    return semantic_search(query, top_k)

Step 2: The Tool Definition (10 min)

TOOLS = [{
    "name": "search",
    "description": (
        "Search the knowledge base. Use this when you need information "
        "to answer a question. The query should be specific and short. "
        "You can call this multiple times with different queries to "
        "find different pieces of information."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "Search query (specific terms work better than questions)"
            },
        },
        "required": ["query"],
    }
}]

The description matters — it teaches the model when to call.

Step 3: System Prompt (10 min)

SYSTEM_PROMPT = """
You are a knowledge base assistant. You answer questions using the 
search tool to find relevant information.

Process:
1. Read the user's question carefully
2. Search the knowledge base for relevant info
3. If needed, search again with different terms to fill gaps
4. Synthesize the answer using what you found
5. Cite the source

If the answer isn't in the knowledge base, say so. Don't make things up.

Each search query should be specific and focused. Multiple targeted 
searches are better than one broad search.
"""

The behavior instructions matter as much as the tool definition.

Step 4: The Agent Loop (15 min)

def kb_agent(question: str, max_steps: int = 10):
    messages = [{"role": "user", "content": question}]
    
    for step in range(max_steps):
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=1024,
            tools=TOOLS,
            system=SYSTEM_PROMPT,
            messages=messages,
        )
        
        if response.stop_reason == "tool_use":
            tool_use = next(c for c in response.content if c.type == "tool_use")
            
            results = search(tool_use.input["query"])
            results_text = "\n\n".join([r["text"] for r in results])
            
            print(f"[Search: {tool_use.input['query']}]")
            
            messages.append({"role": "assistant", "content": response.content})
            messages.append({
                "role": "user",
                "content": [{
                    "type": "tool_result",
                    "tool_use_id": tool_use.id,
                    "content": results_text,
                }]
            })
        else:
            return response.content[0].text
    
    return "Max steps reached without final answer."

Step 5: Try Real Questions (15 min)

answer = kb_agent("How do I reset my password?")
print(answer)

Watch:

[Search: password reset]
[Search: forgot password steps]

To reset your password: 1) Go to the sign-in page, 2) Click 
"Forgot password", 3) Enter your email, 4) Check your inbox 
for a reset link.

Source: Password Reset Documentation

The agent searched, refined, synthesized. More useful than a single retrieval.

Step 6: Try Hard Questions (15 min)

answer = kb_agent("Compare our refund policy to our exchange policy")

The agent likely searches for both, then compares. Watch the trace.

If it fails (only searches once; only finds one), tune the system prompt or add an example.

Step 7: Add Query Logging (10 min)

def search(query, top_k=5):
    results = semantic_search(query, top_k)
    
    log_query({
        "query": query,
        "results_count": len(results),
        "top_score": results[0]["score"] if results else 0,
        "timestamp": datetime.now(),
    })
    
    return results

Track what queries the agent generates. They tell you what users actually need.

Step 8: Handle No Results (10 min)

def search(query, top_k=5):
    results = semantic_search(query, top_k)
    
    if not results:
        return "No relevant information found in the knowledge base."
    
    if results[0]["score"] < 0.3:
        return f"Low-confidence results for '{query}'. May not be relevant."
    
    return format_results(results)

The agent sees the no-result signal; can refine query or give up gracefully.

Step 9: Limit Iterations (5 min)

def kb_agent(question, max_steps=8):
    # ...
    
    if step == max_steps - 1:
        # Last step; force a final answer
        messages.append({
            "role": "user",
            "content": "Based on what you've found so far, give your best answer."
        })

Prevents the agent from searching forever.

Step 10: Compare to Plain RAG (varies)

Run the same question through plain RAG (one search → answer) and through the agent. Compare:

Answer quality
Cost (agent costs more tokens)
Latency (agent is slower)

For complex questions, agent often wins. For simple ones, RAG wins. Use the right tool.

What You Just Did

You built an agent that uses one tool to do useful work. The single tool with iteration often beats one-shot approaches for non-trivial questions.

Common Failure Modes

Single search only. Tool defined but agent never iterates.

Bad queries. Agent searches with the question verbatim; misses relevant chunks.

No "I don't know". Agent confabulates when search is unfruitful.

Cost explosion. No iteration limit; agent searches 50 times.

Trace not logged. Can't debug why a specific answer was wrong.

Next Tutorial

Add more tools: Tutorial 3: Add Multiple Tools to an Agent.

ShiftQuality