Stage 6 · Operate
Logs & Traces
Structured Logging
Emitting JSON logs with trace_id, span_id, request_id, severity, and stable field names.
Why Structured Logging?
Structured logs use a consistent format (typically JSON) with predefined fields. They are machine-parseable, queryable, and consistent across services. Unstructured text logs are difficult to search, correlate, and analyze at scale.
| Approach | Pros | Cons |
|---|---|---|
| Unstructured | Easy to read manually | Hard to parse, search, correlate |
| Structured (JSON) | Queryable, consistent, correlatable | Slightly harder to read raw |
JSON Log Format
Every application should emit logs as single-line JSON objects. Each JSON object represents one log entry. The JSON format enables promtail, Fluent Bit, or Vector to parse and route logs efficiently.
{
"timestamp": "2024-01-15T10:30:00.123Z",
"level": "info",
"service": "api-gateway",
"trace_id": "abc123def456",
"span_id": "789xyz",
"request_id": "req-001",
"method": "GET",
"path": "/api/v1/users",
"status": 200,
"duration_ms": 45,
"message": "Request processed successfully"
}Standard Fields
Define a consistent schema across all services. Standard fields make logs queryable and correlatable. Every service should include at minimum: timestamp, level, service, message, and a correlation identifier.
timestamp - ISO 8601 timestamp (required)
level - Log severity (required)
service - Service name (required)
message - Human-readable description (required)
trace_id - Distributed trace ID (recommended)
span_id - Span ID for correlation (recommended)
request_id - Unique request identifier (recommended)
error_class - Error type for grouping (optional)
user_id - User identifier (optional, avoid as label)Trace Context Fields
Including trace_id and span_id in every log entry enables correlation between logs and traces. This is the foundation of the three pillars of observability: metrics, logs, and traces linked together.
{
"timestamp": "2024-01-15T10:30:00.123Z",
"level": "error",
"service": "payment-service",
"trace_id": "abc123def456789",
"span_id": "a1b2c3d4e5f6",
"message": "Payment processing failed",
"error_class": "PaymentDeclined",
"amount": 99.99,
"currency": "USD"
}Severity Levels
| Level | When to Use | Example |
|---|---|---|
| error | Operation failed, needs attention | Database connection failed |
| warn | Operation degraded but succeeded | Retried request succeeded after timeout |
| info | Normal operation milestones | Server started, request processed |
| debug | Detailed diagnostic information | SQL query executed, cache hit |
Implementation Examples
import structlog
structlog.configure(
processors=[
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.JSONRenderer()
]
)
logger = structlog.get_logger()
logger.info("Request processed", method="GET", path="/api/v1/users", status=200)Most modern frameworks have built-in structured logging support. Use Go's slog, Python's structlog, or Node.js's pino. Do not manually format JSON strings.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.