Stage 4 · Provision
Resilience Patterns
Timeout Budgets
Connection, request, and overall deadline budgets across Envoy, gRPC, and HTTP clients.
Why Timeouts Matter
Without timeouts, a slow dependency can hold resources indefinitely. Threads block, connections pile up, and the entire system cascades into failure. Timeouts are the first line of defense against cascading failures. Every outbound call must have a timeout.
Types of Timeouts
| Timeout | Purpose | Typical Value |
|---|---|---|
| Connection timeout | Time to establish TCP connection | 1-5 seconds |
| Request timeout | Time to complete a single request | 5-30 seconds |
| Overall deadline | Maximum time for entire operation | 30-120 seconds |
| Idle timeout | Time before closing idle connections | 60-300 seconds |
| Stream timeout | Time between stream chunks (gRPC) | 30-60 seconds |
Deadline Propagation
When Service A calls Service B which calls Service C, the deadline should propagate through the chain. If A sets a 5-second deadline, B should not set its own 5-second deadline — it should pass the remaining time to C. This prevents cascading timeouts.
Without deadline propagation:
A sets 5s deadline → B sets 5s deadline → C sets 5s deadline
Total possible time: 15s (exceeds A's deadline)
With deadline propagation:
A sets 5s deadline → B propagates remaining 4s → C propagates remaining 3s
Total possible time: 5s (matches A's deadline)gRPC propagates deadlines automatically via the grpc-timeout header. For HTTP services, implement deadline propagation manually using request context.
Timeout Budget Pattern
import time
class TimeoutBudget:
def __init__(self, deadline_seconds: float):
self.start = time.monotonic()
self.deadline = deadline_seconds
@property
def remaining(self) -> float:
elapsed = time.monotonic() - self.start
return max(0, self.deadline - elapsed)
@property
def expired(self) -> bool:
return self.remaining <= 0
def child_budget(self, fraction: float = 0.5) -> 'TimeoutBudget':
"""Create a child budget with a fraction of remaining time"""
return TimeoutBudget(self.remaining * fraction)
# Usage
budget = TimeoutBudget(deadline_seconds=10)
# Call downstream with 50% of remaining time
child = budget.child_budget(0.5)
response = call_service_b(timeout=child.remaining)A timeout budget tracks remaining time across a chain of calls. Each call gets a fraction of the remaining time, ensuring the total never exceeds the original deadline.
Envoy Timeout Configuration
routes:
- match:
prefix: /api/v1/orders
route:
cluster: order-service
timeout: 30s
retry_policy:
retry_on: "5xx,reset,connect-failure"
num_retries: 3
per_try_timeout: 10s
request_headers_to_add:
- header:
key: x-timeout-budget
value: "%REQUEST_DEADLINE%"Envoy supports per-route and per-try timeouts. The per_try_timeout limits each individual attempt, while the overall timeout limits the entire request including retries.
Graceful Shutdown Timeouts
- Stop accepting new connections immediately on SIGTERM.
- Wait for in-flight requests to complete (drain timeout: 30s).
- Close connections after drain timeout expires.
- Kubernetes preStop hook: sleep 5 before SIGTERM to allow LB removal.
An unbounded timeout is a time bomb. If a downstream service hangs, your service hangs with it. Every outbound HTTP call, database query, and gRPC call must have a timeout. No exceptions.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.