Lambda Cold Starts, VPC ENIs, and Concurrency: Fixing Bimodal p99 Latency

The problem: p99 is 3 seconds, p50 is 40 ms

Your Lambda averages look fine in the dashboard. Customers still complain. You dig into traces and find a bimodal distribution: warm invokes are fast; cold starts (especially in a VPC) dominate p99. “Just add more memory” helps a little. The real levers are init path, concurrency, and networking.

What a cold start actually includes

1. Download your deployment package / image layers.
2. Start the execution environment (MicroVM).
3. Runtime init + your static/global code.
4. For VPC: attach Hyperplane ENIs (historically painful; improved but not free).

Global/module-level code runs on cold start. Opening DB connections, loading huge ORMs, or parsing megabyte JSON configs in the initializer is how you donate seconds to every scale-from-zero burst.

Provisioned concurrency vs reserved concurrency

• Reserved concurrency caps how many executes your function can run — protects downstream and prevents account-level throttling from one noisy function. Set it or a runaway SQS trigger will take the account’s burst concurrency.
• Provisioned concurrency keeps environments warm. You pay for readiness. Great for latency-critical APIs; wasteful for spiky batch if you over-provision 24/7.

Combine with Application Auto Scaling on provisioned concurrency when traffic is diurnal.

VPC Lambdas: the ENI story in 2026

AWS moved to Hyperplane-backed ENIs so every function in a subnet no longer needs a unique ENI per environment the way early VPC Lambda did. You still pay for:

• Cold starts that need network setup path
• NAT Gateway data processing if private subnets need internet (package repos, third-party APIs)
• Security group + subnet capacity planning

Prefer not putting Lambda in a VPC unless it must reach RFC1918 resources. Reach AWS APIs via public endpoints / VPC endpoints; reach the internet from non-VPC functions without NAT charges.

When you must use VPC: put interfaces in multiple AZs, use VPC endpoints for S3/Secrets Manager/STS to avoid NAT, and keep security groups tight.

Concurrency and downstream collapse

Lambda scales to thousands of concurrent executes faster than RDS or a third-party API. Classic failure:

1. SQS queue depth grows overnight.
2. Lambda scales to 500 concurrent consumers.
3. Each opens DB connections → database refuses connections.
4. Messages return to queue → more invokes → death spiral.

Controls: reserved concurrency on the consumer, RDS Proxy, batch size / maximum concurrency on the event source mapping, and idempotent handlers so retries are safe.

Package size and runtime choices

• Strip unused deps; prefer AWS SDK v3 modular packages.
• Container images help large ML deps but can worsen cold start if layers are fat — measure.
• SnapStart (Java) dramatically improves cold starts for compatible managed runtimes — validate serialization constraints.
• Arm64 (Graviton) often wins on price/perf; benchmark your workload.

Observability that finds the bimodal latency

• Emit a custom metric ColdStart=1 from init (or use reported Init Duration in CloudWatch Logs Insights).
• Trace with X-Ray / OpenTelemetry — separate downstream HTTP from init.
• Alarm on concurrent executions, throttles, and p99 duration — not average duration.

Design patterns that work

• Keep handlers thin; initialize clients once in global scope carefully (reuse, but avoid secret work at import time that is not needed for all paths).
• Use SQS/SNS/EventBridge for async; API Gateway + Lambda for request/response with provisioned concurrency only where SLOs demand it.
• For VPC data access, consider moving the hot path to a service already in the VPC (ECS) if cold start SLOs cannot be met economically.

Cold starts are not a moral failing — they are a distributed systems tradeoff. Measure p99, budget concurrency, and only then spend money on provisioned capacity.