top of page

Profile a Slow Endpoint

S. West
Jul 30
3 min read

Updated: 4 days ago

Corrected, 4 October 2026. We re-checked this post's figures. A percentage presented as a general fact is now labelled as a rule of thumb, with a way to measure it on your own team.

Performance Engineering · Part 1

The first rule of performance work is the one everyone breaks: don't guess. Almost every developer's intuition about why code is slow is wrong, so hours get burned optimizing the fast part while the real culprit sits untouched. A profiler tells you the truth. This walks through profiling a slow endpoint to find where the time actually goes — spans, flame graphs, and the usual suspects: a database call, an N+1, a blocking HTTP request.


"My endpoint is slow." First step: find out where the time goes. Guessing wastes hours.


Step 1: Measure (5 min)


Add timing:


import time

@app.route('/slow-endpoint')
def slow_endpoint():
    start = time.time()
    
    # Your endpoint logic
    result = do_work()
    
    elapsed = time.time() - start
    print(f"Total: {elapsed:.3f}s")
    return result

Establish baseline. Is it 500ms or 5s?


Step 2: Add Spans (10 min)


Wrap sections:


@app.route('/slow-endpoint')
def slow_endpoint():
    timings = {}
    
    t = time.time()
    user = db.get_user(request.user_id)
    timings["get_user"] = time.time() - t
    
    t = time.time()
    items = db.get_items(user.id)
    timings["get_items"] = time.time() - t
    
    t = time.time()
    enriched = enrich(items)
    timings["enrich"] = time.time() - t
    
    print(timings)
    return jsonify(enriched)

Reveals which section is slow. 4 seconds in enrich(); now you know where to look.


Step 3: Use a Profiler (15 min)


Manual timing has limits. Use a profiler.


Python (cProfile):


import cProfile

cProfile.run('slow_endpoint()', sort='cumulative')

Outputs every function with time spent.


py-spy for production:


py-spy record -o profile.svg --pid PID

Flame graph; no code changes; works in production.


Step 4: Read a Flame Graph (10 min)


Flame graph:


  • X-axis: time

  • Y-axis: stack depth

  • Width: time spent


Wide boxes = bottlenecks. Find them at the top of stacks.


Common patterns:


  • Wide DB call boxes → slow query

  • Wide HTTP client boxes → external service slow

  • Many tiny boxes → CPU-bound work or N+1


Step 5: Database Profiling (10 min)


For DB-heavy endpoints:


EXPLAIN ANALYZE SELECT ...;

Shows the plan and timing. Look for:


  • Sequential scans on big tables

  • Many rows returned then filtered

  • Nested loops with high iteration


In your code:


db.execute("EXPLAIN ANALYZE " + query, params)

Capture in development; tune the query.


Step 6: HTTP Client Profiling (5 min)


External API calls:


import requests
from time import time

t = time()
response = requests.get('https://external.com/api/data')
elapsed = time() - t
log.info(f"External call took {elapsed:.3f}s")

If external is 500ms and you're calling 10 times, that's 5s right there. Parallelize or cache.


Step 7: Memory Profiling (10 min)


Slow can be memory pressure (paging, GC). Check:


import tracemalloc

tracemalloc.start()
result = slow_function()
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics('lineno')[:10]:
    print(stat)

Shows where memory is allocated.


Memory leaks accumulate → slow GC pauses → endpoint slowness.


Step 8: Distributed Tracing (covered in observability) (5 min)


For microservices:


with tracer.start_as_current_span("db_query"):
    user = db.get_user(...)

with tracer.start_as_current_span("call_other_service"):
    data = external.get(...)

Each span recorded; visible in tracing UI.


Most useful when the slow path crosses service boundaries.


Step 9: Common Findings (10 min)


Patterns you'll see:


  1. N+1 queries: 1 query + N more (tutorial 2)

  2. Missing index: seq scan on big table (tutorial 3)

  3. External API blocking: sync call to slow vendor

  4. Cache miss: every request hits DB (tutorial 4)

  5. GC pause: Java/Python pause for memory cleanup

  6. Lock contention: many threads waiting on a lock


Most issues fall into these categories.


Step 10: Iterate (10 min)


Profile → fix biggest → measure → repeat.


Don't fix things that don't matter. Fixing the biggest hotspot first often delivers most of the gain (a rule of thumb, so re-measure after each change); later fixes tend to get marginal.


Stop when you hit the latency target. Premature optimization is real.


What You Just Did


You can find where time goes in a slow endpoint. Profilers; flame graphs; database EXPLAIN. The foundation for everything else.


Common Failure Modes


Guessing instead of measuring. "I think it's the DB." Wrong half the time.


Optimizing what's already fast. 0.1% time on inner loop; 99% on one DB query.


Profiler overhead. Sampling profilers are cheap; tracing profilers can be heavy.


Profile in dev only. Production traffic differs; profile production (carefully).


Fixing one bottleneck; another appears. Iterate.



Continue the Performance Engineering path



Part of the Performance Engineering learning path.

bottom of page