Tutorial 1: Set Up Your Prompting Workspace

Contributor
4 days ago
3 min read

Before writing useful prompts, you need a workspace where you can iterate quickly. This tutorial sets one up.

What You'll Build

A working environment where you can write prompts, test them, compare outputs, and track what works.

Step 1: Pick a Model and Get Access (15 min)

For starting:

Anthropic Claude: good general-purpose; current Sonnet model
OpenAI GPT: widely used; GPT-4 class
Local models: Ollama, llama.cpp for self-hosted

For learning, the cloud APIs are easiest. Get an API key.

Step 2: Install the SDK (10 min)

Python (Anthropic):

pip install anthropic

Python (OpenAI):

pip install openai

Node:

npm install @anthropic-ai/sdk
# or
npm install openai

Step 3: Make Your First Call (5 min)

import anthropic

client = anthropic.Anthropic(api_key="...")

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, what's 2+2?"}
    ]
)

print(response.content[0].text)

Expected: a response. If it works, you have a baseline.

Step 4: Build a Comparison Script (30 min)

Iteration is faster when you can compare prompts side-by-side:

def compare_prompts(prompts, model="claude-sonnet-4-6"):
    for i, prompt in enumerate(prompts):
        response = client.messages.create(
            model=model,
            max_tokens=1024,
            messages=[{"role": "user", "content": prompt}]
        )
        print(f"--- Prompt {i+1} ---")
        print(f"Input: {prompt[:100]}...")
        print(f"Output: {response.content[0].text}")
        print()

compare_prompts([
    "Summarize the French Revolution in one paragraph.",
    "Summarize the French Revolution in one paragraph. Focus on causes.",
    "Summarize the French Revolution. Use 3 bullet points.",
])

Three prompts; three outputs visible together.

Step 5: Add Cost Tracking (15 min)

LLM calls cost money. Track:

def call_with_tracking(prompt, model="claude-sonnet-4-6"):
    response = client.messages.create(
        model=model,
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}]
    )
    
    # Usage
    input_tokens = response.usage.input_tokens
    output_tokens = response.usage.output_tokens
    
    # Approximate cost (check current pricing)
    cost = input_tokens * 0.003 / 1000 + output_tokens * 0.015 / 1000
    
    print(f"Tokens: {input_tokens} in, {output_tokens} out. Cost: ${cost:.4f}")
    return response.content[0].text

Awareness prevents surprises.

Step 6: Use a Playground UI (10 min)

For exploration, the vendor playgrounds are useful:

Anthropic Console at console.anthropic.com
OpenAI Playground at platform.openai.com/playground
Local UIs: Open WebUI for Ollama

The playground is faster than code for trying many prompts. Use for exploration; move to code for production.

Step 7: Save Prompts as Files (15 min)

For prompts you're iterating on:

prompts/
  summarize_v1.txt
  summarize_v2.txt
  classify_v1.txt
  ...

Version controlled. Easy to compare. Easy to share.

Step 8: Build an Eval Set Early (20 min)

A handful of test cases:

EVAL_CASES = [
    {
        "input": "Customer is asking about refund policy",
        "expected_topic": "refund",
        "expected_tone": "helpful",
    },
    {
        "input": "User wants help understanding their bill",
        "expected_topic": "billing",
        "expected_tone": "helpful",
    },
    # ... 5-10 cases
]

These become your test cases as you iterate. Without them, you can't tell if a prompt change improved things.

Step 9: Set Up Logging (15 min)

For every prompt run, log:

Timestamp
Prompt version
Input
Output
Tokens used
Latency

import json
from datetime import datetime

def log_call(prompt, input, output, usage):
    with open("prompt_log.jsonl", "a") as f:
        f.write(json.dumps({
            "timestamp": datetime.now().isoformat(),
            "prompt": prompt[:200],
            "input": input,
            "output": output,
            "tokens_in": usage.input_tokens,
            "tokens_out": usage.output_tokens,
        }) + "\n")

The log is your record of what you tried.

Step 10: Test the Whole Setup (5 min)

Run a quick end-to-end:

prompt = "Summarize this: {input}"
input_text = "The French Revolution began in 1789..."
output = call_with_tracking(prompt.format(input=input_text))
log_call(prompt, input_text, output, response.usage)
print(output)

Output appears, cost is tracked, log is written. The workspace works.

What You Just Did

You have a working prompt engineering environment. You can write prompts, test them, compare, track costs, and log results. Iteration cycle: under a minute per attempt.

Common Failure Modes

No iteration tooling. Each prompt change involves re-typing in a chat UI. Slow.

No eval cases. Can't tell if changes are improvements.

No cost tracking. Surprised by the bill.

No logs. Don't remember what you tried last week.

Production from start. Skip experimentation; go straight to building. Misses learnings.

ShiftQuality