Cut Memory Usage in Production

Performance Engineering · Part 8

High memory usage costs money on every instance you run and eventually crashes the process outright — and the fix is usually not "add more RAM" but "stop holding onto things you don't need." This walks through reducing memory: finding what's using it, cutting allocations, streaming instead of loading everything at once, and right-sizing limits so you pay for what you actually use.

Memory issues cause OOM crashes, slow GC pauses, and surprise bills. This tutorial walks through reducing usage.

Step 1: Measure (5 min)

Add memory metrics:

import psutil
process = psutil.Process()
print(f"Memory: {process.memory_info().rss / 1024 / 1024:.1f} MB")

Or use container metrics; Prometheus exposes memory metrics in K8s.

Baseline. After load. Over time.

Step 2: Detect Leaks (10 min)

Memory grows unbounded → leak.

# Snapshot over time
import tracemalloc

tracemalloc.start()

# Do work
work_iteration_1()
snap1 = tracemalloc.take_snapshot()

work_iteration_2()
snap2 = tracemalloc.take_snapshot()

# Diff
top_stats = snap2.compare_to(snap1, 'lineno')
for stat in top_stats[:10]:
    print(stat)

Lines that consistently grow = leak source.

Step 3: Common Leak Patterns (10 min)

Global state growing:

all_users = []   # Module-level list

def process_user(user):
    all_users.append(user)   # Never cleared

Caches without bounds:

cache = {}
def get(key):
    cache[key] = expensive_compute(key)
    return cache[key]

Use functools.lru_cache(maxsize=1000) for bounded.

Closures holding refs:

def make_handler(big_data):
    def handler(event):
        process(big_data, event)  # big_data captured
    return handler

# big_data lives as long as handler does

Callbacks/listeners not unregistered:

emitter.on('event', callback)
# Object goes out of scope; callback still registered

Each prevents GC.

Step 4: Reduce Allocations (10 min)

Frequent allocations stress GC:

# Bad: new dict per loop
for item in items:
    result = {"a": item.a, "b": item.b}
    process(result)

# Better: reuse
result = {}
for item in items:
    result["a"] = item.a
    result["b"] = item.b
    process(result)

In hot loops, reuse buffers.

Step 5: Stream Instead of Load (10 min)

For large data:

# Bad: loads everything
def process_file(path):
    with open(path) as f:
        data = f.readlines()  # All lines in memory
    
    for line in data:
        process(line)

# Better: streams
def process_file(path):
    with open(path) as f:
        for line in f:  # One at a time
            process(line)

For DB:

# Bad
rows = db.fetchall()  # All in memory

# Better
for row in db.cursor():  # Stream
    process(row)

Step 6: Use Right Data Structures (5 min)

# Set instead of list for membership tests
known_ids = set(all_ids)  # O(1) lookup vs O(n)

# Deque for queue/stack
from collections import deque
queue = deque()  # O(1) pop from either end

# array module for homogeneous numeric data
import array
nums = array.array('i', [1, 2, 3])  # Less memory than list

Right structure = less memory + faster.

Step 7: GC Tuning (10 min)

For long-running processes:

import gc

# Reduce GC frequency for less overhead but more memory
gc.set_threshold(50000, 100, 100)

# Or force GC at known low-traffic times
gc.collect()

Less GC = less pause time. But memory peaks higher.

Java/Node similar tunings exist.

Step 8: Object Pooling (10 min)

For expensive-to-create objects:

class ObjectPool:
    def __init__(self, factory, size=10):
        self.factory = factory
        self.pool = [factory() for _ in range(size)]
    
    def acquire(self):
        return self.pool.pop() if self.pool else self.factory()
    
    def release(self, obj):
        obj.reset()
        self.pool.append(obj)

Connection pools work this way.

Step 9: Monitor in Production (10 min)

Track:

• RSS (resident set size)

• Heap size and growth

• GC frequency and duration

• OOM kill events

Alert on:

• Memory > 80% of limit

• Sustained growth (leak indicator)

• OOM kills

Catch issues before crashes.

Step 10: Right-Size Limits (5 min)

Don't over-provision blindly:

• Set memory limits per container (K8s, ECS)

• Limit too low: OOM kill

• Limit too high: wasted capacity

Profile actual usage; set limit ~1.5x peak. Monitor over time.

What You Just Did

You can find leaks, reduce allocations, stream large data, tune GC, and monitor memory in production. Your apps run on less.

Common Failure Modes

Caching without bounds. Eventually OOM.

Closures holding big data. Even after "done."

Loading whole files into memory. Crashes on bigger files.

No memory monitoring. First sign of leak = production crash.

Over-tuning GC. Counterproductive; default tuning is usually OK.

---

Continue the Performance Engineering path

• Previous — Part 7: Optimize a Slow Database Query

• Next — Part 9: Reduce Cold Start Time

Part of the Performance Engineering learning path.