top of page

LLM App Shapes — LLM Application Engineering, Part 1

  • Shawn West
  • Jul 8
  • 3 min read

Updated: Aug 6

LLM Application Engineering · Part 1

Most LLM projects that stall do it at the very first fork: the team picks the wrong shape for the app and then fights that choice for months. A question-answering tool built as freeform chat when it should have been RAG; an agent loop where a single structured completion would have done the job. The shape sets your cost, your latency, and your failure modes before you write a line of prompt. This walks through the handful of shapes LLM apps actually take — completion, chat, RAG, tool use, agents — and how to pick the right one.

LLM apps have recognizable patterns. Pick the right shape; build deliberately.

Step 1: Simple Completion (10 min)

response = openai.chat.completions.create(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Summarize this article: ...'}]
)

User submits text → LLM responds.

For: summarization; transformation; basic helpers.

Simplest LLM app. Lots of value already.

Step 2: Chat (15 min)

Stateful conversations:

messages = [
  {'role': 'system', 'content': 'You are a helpful assistant.'},
  {'role': 'user', 'content': 'What's the capital of France?'},
  {'role': 'assistant', 'content': 'Paris.'},
  {'role': 'user', 'content': 'And of Germany?'},
]

History accumulates. Context grows.

For: chatbots; customer support.

Watch for context window limits.

Step 3: RAG (Retrieval-Augmented Generation) (15 min)

1. User question
2. Search docs (vector / keyword / hybrid)
3. Pass relevant docs as context to LLM
4. LLM answers based on context

For: knowledge-base Q&A; docs search.

Most LLM apps with company data: RAG.

(Part 4 deep.)

Step 4: Tool Use / Function Calling (15 min)

LLM decides to call tools:

tools = [
  { 'type': 'function', 'function': {
    'name': 'get_weather',
    'parameters': { 'location': {'type': 'string'} },
  }},
]

response = openai.chat.completions.create(
  model='gpt-4o',
  messages=[...],
  tools=tools,
)

if response.choices[0].message.tool_calls:
    # Execute tool; pass result back

For: agents; structured actions; multi-step tasks.

(Part 5 deep.)

Step 5: Agents (Loops) (15 min)

Loop:
  1. LLM plans action
  2. App executes tool
  3. LLM evaluates; decides next
  4. Until task done or iteration limit

Examples:

  • Browse web; summarize

  • Code generation + testing

  • Research assistants

Capable but expensive + unpredictable. Guard rails essential.

Step 6: Copilots (10 min)

LLM augments human in-line:

  • GitHub Copilot

  • Notion AI

  • Linear's AI

Suggest next action; user accepts / edits.

For: workflows where human stays in loop.

Step 7: Structured Output Apps (15 min)

LLM as data extractor:

class Invoice(BaseModel):
    invoice_number: str
    total: float
    line_items: list[LineItem]

response = openai.chat.completions.create(
  model='gpt-4o-mini',
  messages=[...],
  response_format=Invoice,  # Pydantic
)

For: data extraction; classification; routing.

Fastest ROI. Old NLP tasks now cheap.

Step 8: Embedding-Based Apps (10 min)

Without generation:

  • Search (Part 7 of Path 78)

  • Clustering

  • Classification

  • Recommendation

LLMs used to embed, not generate.

Cheap; deterministic; effective.

Step 9: Multimodal (10 min)

GPT-4 Vision; Claude; Gemini:

  • Images in, text out

  • Receipts → structured data

  • Diagrams → descriptions

  • Screenshots → UI suggestions

Many tasks previously requiring CV models.

Step 10: Shape Decision (15 min)

For your problem:

  • One-shot task: completion / structured output

  • Conversation: chat

  • Knowledge from your docs: RAG

  • Actions on systems: tools / agent

  • In-line assistance: copilot

  • Just embeddings: skip generation

Smallest shape that works > complex agent for simple need.

What You Just Did

LLM app shapes: simple completion, chat, RAG, tool use, agents, copilots, structured output, embeddings, multimodal, shape decision.

Common Failure Modes

Agent for simple task. Cost; flakiness.

RAG without retrieval quality. Hallucination.

Chat without context management. Token waste.

Structured output without schema validation. Garbage.

Skip evals. Drift unnoticed.

Continue the LLM Application Engineering path

Part of the LLM Application Engineering learning path.

bottom of page