Tutorial 5: Tool Use and Agents

Contributor
4 days ago
2 min read

LLM decides what to do, then does it. Powerful pattern; needs constraints.

Step 1: Tool Use Basics (15 min)

tools = [{
  'type': 'function',
  'function': {
    'name': 'get_weather',
    'description': 'Get current weather for location',
    'parameters': {
      'type': 'object',
      'properties': {
        'location': { 'type': 'string', 'description': 'City name' }
      },
      'required': ['location']
    }
  }
}]

response = openai.chat.completions.create(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Weather in Paris?'}],
  tools=tools
)

# If response.choices[0].message.tool_calls:
#   call the function; pass result back

LLM decides whether + which tool to call.

Step 2: Tool Loop (15 min)

def agent_loop(question, tools):
    messages = [{'role': 'user', 'content': question}]
    while True:
        response = llm.chat(messages, tools=tools)
        msg = response.choices[0].message
        if not msg.tool_calls:
            return msg.content
        
        messages.append(msg)
        for call in msg.tool_calls:
            result = execute_tool(call.function.name, json.loads(call.function.arguments))
            messages.append({
              'role': 'tool',
              'tool_call_id': call.id,
              'content': json.dumps(result),
            })

LLM calls tools until done.

Step 3: Tool Design (15 min)

Good tools:

Specific purpose
Clear input schema
Predictable output schema
Idempotent (when possible)
Documented (LLM reads description)

Bad:

Catch-all "do_action"
Loose schemas
Side effects undocumented

Step 4: Iteration Limits (10 min)

Always cap loops:

for _ in range(MAX_ITERATIONS):
    ...

Without: agent loops on confusion, wastes tokens, occasionally infinite.

10-20 typical cap.

Step 5: Cost / Token Tracking (15 min)

Each tool call:

LLM input tokens (history + tools)
LLM output tokens
Tool execution time

For complex tasks: $1-10 per run.

Track:

total_tokens += response.usage.total_tokens

Alert / cap if exceeds.

Step 6: Frameworks (15 min)

LangChain: established; lots of integrations
LlamaIndex: RAG-focused; also agents
AutoGen: multi-agent
CrewAI: agent crews
OpenAI Assistants API: managed

Frameworks help but can hide important details.

For production: often roll your own loop for control.

Step 7: Safety: Authorization (15 min)

Tools can be dangerous:

"delete_database" tool
"send_email" with attacker-controlled content

Always:

Authorize before execute
Confirm destructive actions
Limit scope per session
Audit logs

Agents shouldn't have powers users don't.

Step 8: Reasoning + Action (15 min)

ReAct pattern:

Thought: I need to find the population of Paris.
Action: search("population of Paris")
Observation: Paris has 2.1 million inhabitants in 2024.
Thought: User asked about Paris in the context of France.
Action: respond with "Paris has 2.1M people, capital of France"

Sometimes built-in to frameworks. Sometimes you scaffold.

Helps LLM plan vs. just react.

Step 9: Multi-Agent (15 min)

Multiple agents collaborate:

Researcher: gathers info
Writer: drafts content
Editor: reviews / improves

Each specialized.

Frameworks: AutoGen, CrewAI.

Promising but: cost; coordination complexity; debug nightmare.

For most apps: single agent + good tools sufficient.

Step 10: When NOT Agents (15 min)

Skip agents for:

Simple Q&A (use RAG)
Single-step extraction (use structured output)
Deterministic workflows (use code; not LLM)

Agents shine for:

Open-ended tasks
Multi-step research
Adaptive workflows

For most production apps: simpler shapes win. Agents are flashy; sometimes overkill.

What You Just Did

Tool use + agents: basics, loop, tool design, iteration limits, cost tracking, frameworks, safety, ReAct, multi-agent, when not.

Common Failure Modes

Unbounded loops. Token explosion.

No safety on destructive tools. Disaster.

Agent for trivial task. Cost + unpredictable.

Over-reliance on framework. Hidden bugs.

No cost cap. Surprise bill.

ShiftQuality