Kill Cold-Start Latency

Performance Engineering · Part 9

In serverless and autoscaled systems, the first request to a fresh instance pays a tax nobody else does: the cold start, while the runtime boots and your code initializes. When it's slow, real users feel it as a random slow page. This walks through cutting cold-start time — trimming what runs at startup, deferring heavy initialization, and the trade-offs of provisioned capacity.

Cold start = first request after a fresh process. For serverless and autoscaling, slow cold starts hurt UX.

Step 1: Measure (5 min)

import time
START = time.time()

# All imports
import django  # 200ms?
import numpy  # 300ms?
# ...

# When ready
print(f"Boot: {time.time() - START:.3f}s")

Or use built-in profiling — Python's -X importtime:

python -X importtime my_app.py 2>&1 | sort -k 2 -n -r | head

Shows slowest imports.

Step 2: Reduce Imports at Startup (10 min)

# Bad: imports at top
import pandas  # Loads on every cold start, even if rarely used

def rare_endpoint():
    return pandas.read_csv(...)

# Better: lazy import
def rare_endpoint():
    import pandas
    return pandas.read_csv(...)

Only loads when called. Cold start drops.

Step 3: Defer Heavy Initialization (10 min)

# Bad: eager
class MyApp:
    def __init__(self):
        self.model = load_model("big_model.pkl")  # 5 seconds

# Better: lazy
class MyApp:
    def __init__(self):
        self._model = None
    
    @property
    def model(self):
        if self._model is None:
            self._model = load_model("big_model.pkl")
        return self._model

First call to model pays the cost. Other endpoints don't.

Step 4: Connection Pre-Warming (5 min)

# At startup, after imports
db.connect()  # Establish connection so first request doesn't wait
cache.ping()  # Verify cache reachable

Pays the cost during startup; first request is fast.

For serverless (Lambda), use provisioned concurrency or warmup pings.

Step 5: Strip the Image (5 min)

For containers:

• Multi-stage build (Tutorial in Docker path)

• Use slim base images

• Remove dev dependencies

• No build tools in runtime image

Smaller image = faster pull = faster cold start.

# Bad
FROM python:3.11
COPY . .
RUN pip install -r requirements.txt

# Better
FROM python:3.11 AS builder
COPY requirements.txt .
RUN pip wheel -r requirements.txt --wheel-dir=/wheels

FROM python:3.11-slim
COPY --from=builder /wheels /wheels
RUN pip install /wheels/*.whl
COPY . .

Step 6: Compile Where Possible (10 min)

Some languages compile to faster startup:

Python: consider Cython for hot paths.

JavaScript: ESBuild or SWC for fast builds; smaller bundles.

Go/Rust: already compiled; cold start usually fast.

Java: GraalVM native image — converts JVM apps to native binaries. Sub-second startup vs 10s+ traditional.

Step 7: Reduce Bundle Size (Web) (10 min)

For frontend:

# Analyze
npm install --save-dev source-map-explorer
source-map-explorer 'build/static/js/*.js'

Identifies bloat. Common fixes:

• Tree-shake unused exports

• Dynamic imports for routes

• Remove dependencies (each one costs)

• Use smaller alternatives (date-fns vs moment)

Cold start = parse + execute. Less code = faster.

Step 8: Cache Compiled Code (5 min)

Python: __pycache__ (automatic). Persist across cold starts (in image; mounted volume).

Java: AOT compilation; class data sharing.

Node: V8 compile cache.

Step 9: Provisioned Capacity (10 min)

For serverless that needs to be fast:

• Lambda Provisioned Concurrency: keep N instances warm

• Cloud Run min instances: never scale below N

• Cloudflare Workers: isolates start in ms; no cold start

Costs more. Eliminates cold starts.

Step 10: Measure End-to-End (5 min)

Before: 3.5s cold start
After deferred imports: 1.2s
After lazy model loading: 0.8s
After image optimization: 0.6s
With provisioned concurrency: ~50ms

Each technique adds up. Pick what matches your latency target and cost budget.

What You Just Did

You can reduce cold start dramatically. Defer work, strip images, compile where possible, warm-up strategies.

Common Failure Modes

Heavy imports at top. Unavoidable boot cost.

No measurement. Don't know what's slow.

Cold start "fixed" for one path. Other paths still slow.

Premature provisioned concurrency. Pay for warm instances when you don't need them.

No image strip. Cold start dominated by image pull, not code.

---

Continue the Performance Engineering path

• Previous — Part 8: Reduce Memory Usage

• Next — Part 10: Establish Performance Budgets

Part of the Performance Engineering learning path.