How to Actually Read a Flame Graph

Performance Engineering · Part 6

A flame graph looks like abstract art until someone shows you how to read it — and then it becomes the single fastest way to see where a program spends its time. Width is time; the widest bars are your problem. This walks through reading one: the anatomy, finding the hotspots, and going from "that bar is wide" to an actual fix.

A flame graph visualizes where CPU time goes. Once you know how to read them, bottlenecks jump out.

Step 1: The Anatomy (5 min)

A flame graph:

• X-axis: sorted alphabetically (NOT time). Width = total time in that function.

• Y-axis: stack depth. Top of stack at the top.

• Box width: time spent (or samples).

• Color: usually just for differentiation; sometimes semantic.

Look for wide boxes near the top — those are the hot spots.

Step 2: Generate One (10 min)

Python with py-spy:

pip install py-spy
py-spy record -o profile.svg -- python my_script.py

Or attach to running process:

py-spy record -o profile.svg --pid 12345

Node.js:

node --prof my_script.js
node --prof-process isolate-0xNNNN-v8.log > processed.txt
# Use d3-flame-graph to visualize

Java: async-profiler.

Go: built-in pprof.

Step 3: Open It (3 min)

Open profile.svg in a browser. Interactive — click boxes to zoom, search to highlight.

Step 4: Read the Tree (10 min)

Start from bottom-up:

[main]
  [process_request]
    [validate]
    [fetch_data]   ← wide here = lots of time
      [db_query]    ← even wider = the actual slow spot

The widest box at the top of a chain is doing the work taking the time.

Step 5: Find the Hotspots (10 min)

Scroll through. Look for:

• Wide boxes at the top: time spent in this function

• Wide boxes with lots of children: function calls many things; each child shows where time goes within it

• Tall thin stacks: deep call chains; usually not bottlenecks

Click into wide regions to zoom.

Step 6: Inverted Flame Graph (5 min)

Sometimes useful: invert (icicle graph) — top of stack at bottom.

Lets you see which leaves consume most time across the whole graph. Bottom-up perspective.

Step 7: Differential Flame Graphs (10 min)

Compare two profiles:

Profile A: before optimization
Profile B: after
Diff:      shows what changed

Green = faster; red = slower. Spot regressions or verify improvements.

Tools: differential flame graphs (Brendan Gregg's scripts).

Step 8: Common Patterns (10 min)

Most common bottlenecks:

• Wide DB function: slow query

• Wide HTTP client: external service slow

• Wide JSON parsing: large payloads

• Wide regex: catastrophic backtracking

• Wide GC functions: memory pressure

• Wide string concatenation: O(n²) string building

Each has known fixes.

Step 9: Production Profiling (10 min)

Continuous profiling captures over time:

• Datadog Continuous Profiler

• Pyroscope (open source)

• Parca (open source)

Always-on; sample-based; low overhead. View any time period.

Useful for catching intermittent issues that don't reproduce locally.

Step 10: From Flame Graph to Fix (10 min)

The graph identifies. The fix happens in code.

Process:

1. Find the widest top box (the bottleneck)

2. Open the source code for that function

3. Understand why it's slow

4. Optimize:
   

   • Cache results

   • Better algorithm

   • Move work elsewhere

5. Re-profile; verify

6. Move to next biggest

Don't try to fix everything. Fix the biggest; measure; repeat.

What You Just Did

You can read flame graphs and find CPU bottlenecks. The visualization is the diagnostic; the fix is in code.

Common Failure Modes

Trying to read raw profile data. Use flame graphs.

Looking only at the top. Sometimes the issue is the wide box below, calling the narrower one many times.

Optimizing thin boxes. Don't matter.

No baseline. Can't tell if optimization helped.

Profiling the wrong scenario. Profile what the user actually does.

---

Continue the Performance Engineering path

• Previous — Part 5: Load Test with k6

• Next — Part 7: Optimize a Slow Database Query

Part of the Performance Engineering learning path.