Course outline

01 · Kubernetes Architecture Deep Dive

02 · Workloads & Scheduling

03 · Networking, Storage & Service Mesh

04 · Platform Engineering & Multi-Tenancy

05 · Networking & Service Mesh

06 · Observability & Autoscaling

07 · Configuration & Secrets Management

08 · Stateful Storage Operations

09 · Security, Identity & Admission

10 · Release Packaging with Helm & Kustomize

11 · Operators, CRDs & Controllers

12 · Production Troubleshooting & Operations

Stage 5 · Platform

Security, Identity & Admission

Audit & Compliance

Audit policies, kube-apiserver logs, Kubernetes events, and compliance evidence collection.

6 min readKubernetes & Cloud NativePlatform

Audit Overview

Audit logging records who did what and when in the Kubernetes API server. It captures every API request: authentication, authorization, and the request/response body. Audit logs are essential for security investigations and compliance.

Level	What's Logged	Use Case
None	Nothing	Non-sensitive endpoints
Metadata	Who, when, what	General auditing
Request	Metadata + request body	Debugging
RequestResponse	Metadata + request + response	Full audit trail

Audit Policy

Audit policy defines what gets logged and at what level. Policies are evaluated in order — the first matching rule determines the audit level. Always log secret access and resource modifications at RequestResponse level.

Comprehensive audit policy


apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# Don't log read-only requests to kube-system
- level: None
  users: ["system:kube-proxy"]
  verbs: ["watch"]
  resources:
  - group: ""
    resources: ["endpoints", "services", "services/status"]

# Log secrets at RequestResponse level
- level: RequestResponse
  resources:
  - group: ""
    resources: ["secrets"]

# Log pod creation/deletion
- level: Metadata
  resources:
  - group: ""
    resources: ["pods"]
  verbs: ["create", "delete", "deletecollection"]

# Log everything else at Metadata level
- level: Metadata
  omitStages:
  - RequestReceived

The first matching rule determines the audit level. Secrets are always logged at RequestResponse. Pods are logged at Metadata. kube-system watch requests are not logged. omitStages: RequestReceived skips the initial request stage.

Audit Backends

Audit logs go to backends: log (file), webhook (HTTP endpoint), or streaming (continuous push). For production, use webhook or streaming to send logs to a centralized system (Elasticsearch, Splunk, Azure Monitor).

Audit webhook backend


# kube-apiserver audit flags
--audit-policy-file=/etc/kubernetes/audit-policy.yaml
--audit-log-path=/var/log/kubernetes/audit.log
--audit-log-maxage=30
--audit-log-maxbackup=10
--audit-log-maxsize=100
--audit-webhook-config-file=/etc/kubernetes/audit-webhook.yaml
--audit-webhook-batch-max-size=1000
--audit-webhook-batch-max-wait=30s

log backend writes to a local file. webhook backend sends to a remote endpoint. batch-max-size and batch-max-wait control batching for webhook. maxage, maxbackup, and maxsize control log rotation.

Kubernetes Events

Kubernetes Events record state changes: pod scheduling, container starts, probe failures, and scaling. Events are ephemeral (1 hour by default). Use event exporters to persist events for analysis.

Event monitoring


# Watch events in real-time
kubectl get events -w --all-namespaces

# Filter by resource
kubectl get events --field-selector involvedObject.name=my-pod

# Filter by type
kubectl get events --field-selector type=Warning

# Export events for analysis
kubectl get events -o json | jq '.items[] | {
  reason: .reason,
  message: .message,
  namespace: .metadata.namespace,
  timestamp: .lastTimestamp
}' > events.json

Events provide real-time visibility into cluster state. Warning events indicate problems (FailedScheduling, OOMKilled, Unhealthy). Export events to external systems for long-term retention and analysis.

Compliance Evidence

Compliance requires evidence: who accessed what, when, and with what permissions. Collect: audit logs, RBAC bindings, Pod Security Admission labels, NetworkPolicies, and encryption-at-rest configuration.

Compliance evidence collection


# Export RBAC configuration
kubectl get clusterrolebindings -o yaml > clusterrolebindings.yaml
kubectl get rolebindings --all-namespaces -o yaml > rolebindings.yaml

# Export Pod Security labels
kubectl get namespaces -o json | jq '.items[] | {
  name: .metadata.name,
  labels: .metadata.labels | to_entries | map(select(.key | startswith("pod-security")))
}'

# Export NetworkPolicies
kubectl get networkpolicies --all-namespaces -o yaml > networkpolicies.yaml

# Export encryption configuration
kubectl get apiserver -o json | jq '.spec.encryptionConfiguration'

Collect evidence periodically and store it securely. This data proves compliance with security policies. Automate collection with CronJobs that export configurations to a secure storage location.

Log Analysis

Audit log analysis


# Find who accessed secrets
grep '"resource":"secrets"' audit.log | jq '.user.username'

# Find failed authorization attempts
grep '"responseStatus.code":403' audit.log | jq '.user.username'

# Find API requests from a specific user
grep '"username":"admin@example.com"' audit.log | jq '.verb, .objectRef.resource'

# Count requests by user
cat audit.log | jq -r '.user.username' | sort | uniq -c | sort -rn

Audit logs are JSON — use jq for analysis. Find unauthorized access attempts, track who accessed secrets, and monitor API usage patterns. Export to Elasticsearch for dashboards and alerting.

Audit log retention

Retain audit logs for at least 90 days for compliance. Use log rotation (maxage, maxbackup) to prevent disk exhaustion. Export to centralized logging for long-term retention and cross-cluster analysis.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.