Cache Hot Reads Without Breaking Data

Performance Engineering · Part 4

Caching is the fastest way to speed up a read-heavy system — and the fastest way to serve stale or wrong data if you get it slightly off. The right cache turns a repeated expensive lookup into a near-instant one; the wrong one creates bugs that only appear under load. This walks through caching hot reads safely: picking a candidate, choosing a TTL, and handling the two hard parts — invalidation and cache stampedes.

When the same data is fetched repeatedly, caching it eliminates the work. This tutorial walks through adding cache to a real endpoint.

Step 1: Find a Cache Candidate (5 min)

Good candidates:

• Same query, same parameters, called repeatedly

• Data doesn't change every second

• Stale data is acceptable for some seconds

Bad candidates:

• Per-user, per-request data (low repeat)

• Data that must be exact (account balance)

• Things that change every read

Step 2: Measure Without Cache (5 min)

Baseline:

@app.route('/api/product/<int:product_id>')
def get_product(product_id):
    start = time.time()
    product = db.get_product(product_id)
    print(f"Query: {time.time() - start:.3f}s")
    return jsonify(product)

Say it's 50ms. With cache: should be sub-ms.

Step 3: Add a Simple Cache (10 min)

import redis
cache = redis.Redis()

@app.route('/api/product/<int:product_id>')
def get_product(product_id):
    key = f"product:{product_id}"
    
    cached = cache.get(key)
    if cached:
        return cached  # Pre-serialized JSON
    
    product = db.get_product(product_id)
    json_str = json.dumps(product)
    
    cache.setex(key, 300, json_str)  # TTL 5 minutes
    return json_str

5-minute TTL. Cache for hot products; stale for 5 min max.

Step 4: Measure With Cache (3 min)

# Load test
for i in range(1000):
    requests.get(f'http://localhost/api/product/{i % 10}')  # 10 products, repeated

First fetch per product: 50ms. After: < 1ms. Massive win.

Step 5: Pick the Right TTL (5 min)

Too short: cache useless. Too long: stale data.

Match the use case:

• Real-time-ish (prices, status): 30-60 seconds

• Profile data: 5-15 minutes

• Catalog data: 1+ hour

• Reference data (countries, currencies): days

When in doubt: 5 minutes.

Step 6: Handle Invalidation (10 min)

When data changes:

def update_product(product_id, data):
    db.update_product(product_id, data)
    cache.delete(f"product:{product_id}")

Cache cleared; next request fetches fresh.

For data with many keys (e.g., search results), use patterns:

def update_category(category_id, data):
    db.update_category(category_id, data)
    
    # Invalidate everything related
    for key in cache.scan_iter(f"category:{category_id}:*"):
        cache.delete(key)

Step 7: Stampede Protection (10 min)

When TTL expires under load, many requests hit DB simultaneously:

def get_product(product_id):
    key = f"product:{product_id}"
    cached = cache.get(key)
    if cached:
        return cached
    
    # Get a lock
    lock_key = f"lock:{key}"
    if cache.set(lock_key, "1", nx=True, ex=10):
        try:
            product = db.get_product(product_id)
            cache.setex(key, 300, json.dumps(product))
            return product
        finally:
            cache.delete(lock_key)
    else:
        # Wait briefly; retry
        time.sleep(0.05)
        return get_product(product_id)

Only one request rebuilds; others wait.

Step 8: Cache Negative Lookups (5 min)

If the DB returns "not found," cache that too:

product = db.get_product(product_id)
if not product:
    cache.setex(key, 60, "null")  # Short TTL
    return None

cache.setex(key, 300, json.dumps(product))

Prevents repeated DB hits for non-existent items.

Step 9: Multi-Level Cache (5 min)

For very hot data, cache in memory too:

from functools import lru_cache

@lru_cache(maxsize=1000)
def get_product_local(product_id):
    return get_product(product_id)

L1 (in-process) — microseconds. L2 (Redis) — milliseconds. L3 (DB) — milliseconds+. Each layer adds complexity but reduces tail latency.

Step 10: Monitor Cache Health (5 min)

Track:

• Hit rate per cache type

• TTL adequacy (how often expire vs invalidate)

• Memory usage (Redis eviction?)

• Errors (cache unavailable?)

Falls back to DB if cache fails:

try:
    cached = cache.get(key)
    if cached:
        return cached
except RedisError:
    # Cache unavailable; fall through to DB
    pass

product = db.get_product(product_id)

Cache outage shouldn't break the app.

What You Just Did

You added caching with proper TTL, invalidation, stampede protection, negative caching, fallback. Hot reads now near-instant.

Common Failure Modes

Caching everything. Memory blows up.

Forever TTL. Stale forever.

No invalidation. Stale data; user complaints.

Cache stampede. TTL expiry pile-up; DB hammered.

Hard dependency on cache. Cache down = app down.

---

Continue the Performance Engineering path

• Previous — Part 3: Add an Index That Works

• Next — Part 5: Load Test with k6

Part of the Performance Engineering learning path.