Course outline

01 · Kubernetes Architecture Deep Dive

02 · Workloads & Scheduling

03 · Networking, Storage & Service Mesh

04 · Platform Engineering & Multi-Tenancy

05 · Networking & Service Mesh

06 · Observability & Autoscaling

07 · Configuration & Secrets Management

08 · Stateful Storage Operations

09 · Security, Identity & Admission

10 · Release Packaging with Helm & Kustomize

11 · Operators, CRDs & Controllers

12 · Production Troubleshooting & Operations

Stage 5 · Platform

Configuration & Secrets Management

External Secrets

External Secrets Operator, Secret Store CSI Driver, Azure Key Vault, and rotation workflows.

8 min readKubernetes & Cloud NativePlatform

External Secrets Overview

External Secrets tools sync secrets from external vaults (Azure Key Vault, AWS Secrets Manager, HashiCorp Vault) into Kubernetes. This eliminates storing secrets in etcd. Secrets are fetched from the vault and injected into pods at runtime.

Tool	Approach	Secret Location
External Secrets Operator	Syncs to K8s Secrets	etcd (synced from vault)
Secret Store CSI Driver	Mounts directly from vault	Never in etcd
Vault Agent Injector	Sidecar from Vault	Never in etcd

External Secrets Operator

External Secrets Operator (ESO) syncs secrets from external vaults into Kubernetes Secrets. It creates Secret resources that look like normal K8s Secrets but are populated from external sources. Secrets are re-synced on a configurable interval.

ExternalSecret with Azure Key Vault


apiVersion: external-secrets.io/v1beta1
kind: SecretStoreRef
metadata:
  name: azure-keyvault
  namespace: production
spec:
  provider:
    azurekv:
      tenantId: "00000000-0000-0000-0000-000000000000"
      vaultUrl: "https://my-vault.vault.azure.net"
      authType: WorkloadIdentity
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: azure-keyvault
    kind: SecretStore
  target:
    name: db-credentials-synced
    creationPolicy: Owner
  data:
  - secretKey: password
    remoteRef:
      key: db-password
      version: latest
  - secretKey: username
    remoteRef:
      key: db-username
      version: latest

SecretStoreRef points to the Azure Key Vault provider. ExternalSecret maps remote vault keys to Kubernetes Secret keys. refreshInterval: 1h re-syncs every hour. The synced Secret (db-credentials-synced) is a normal K8s Secret.

Secret Store CSI Driver

Secret Store CSI Driver mounts secrets directly from the vault into pods as volumes. Secrets never touch etcd — they are fetched at pod startup and updated on a schedule. This provides stronger security than ESO.

CSI Secret Store Driver


apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-keyvault
  namespace: production
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "00000000-0000-0000-0000-000000000000"
    keyvaultName: "my-vault"
    objects: |
      array:
        - |
          objectName: db-password
          objectType: secret
          objectVersion: ""
        - |
          objectName: tls-cert
          objectType: secret
          objectVersion: ""
  secretObjects:
  - secretName: db-credentials-csi
    type: KubernetesSecret
    data:
    - objectName: db-password
      key: password

SecretProviderClass defines which secrets to fetch from Key Vault. The CSI driver mounts them at /mnt/secrets-store/db-password. secretObjects optionally creates a K8s Secret from the mounted files.

CSI vs ESO tradeoffs

CSI Driver is more secure (secrets never in etcd) but requires CSI driver installation on every node. ESO is simpler (creates K8s Secrets) but secrets exist in etcd temporarily. Use CSI for high-security workloads, ESO for simplicity.

Azure Key Vault Integration

Azure Key Vault stores secrets, certificates, and keys. Kubernetes integrates with Key Vault through workload identity, managed identity, or service principal. Workload identity is recommended — it federates K8s service accounts with Azure AD.

Workload Identity for Key Vault access


apiVersion: v1
kind: ServiceAccount
metadata:
  name: keyvault-reader
  namespace: production
  annotations:
    azure.workload.identity/client-id: "00000000-0000-0000-0000-000000000000"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
  namespace: production
spec:
  template:
    metadata:
      labels:
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: keyvault-reader
      containers:
      - name: app
        image: my-app:1.0

Workload Identity federates the K8s service account with Azure AD. The pod gets an Azure AD token using the service account identity. This token authenticates with Key Vault without storing credentials.

Secret Rotation

Secret rotation ensures secrets are updated before they expire. ESO re-syncs on refreshInterval. CSI Driver rotates on a configurable schedule. For applications, use Stakater Reloader to restart pods when secrets change.

Secret rotation with ESO


apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: rotating-secret
spec:
  refreshInterval: 5m  # Check for changes every 5 minutes
  secretStoreRef:
    name: azure-keyvault
    kind: SecretStore
  target:
    name: rotating-secret
    creationPolicy: Owner
    template:
      type: Opaque
      data:
        password: "{{ .password }}"
        last-rotated: "{{ now | date `2006-01-02T15:04:05Z` }}"
  data:
  - secretKey: password
    remoteRef:
      key: rotating-password
      version: latest

refreshInterval: 5m checks Key Vault every 5 minutes. If the secret version changes, ESO updates the K8s Secret. template allows formatting the synced secret with additional metadata.

Multi-Source Secrets

Combine secrets from multiple vaults or sources into a single Secret. This is useful when different teams manage different secrets, or when you need secrets from multiple cloud providers.

Multiple SecretStores


apiVersion: external-secrets.io/v1beta1
kind: SecretStoreRef
metadata:
  name: aws-secrets-manager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        secretRef:
          accessKeyIDSecretRef:
            name: aws-credentials
            key: access-key
          secretAccessKeySecretRef:
            name: aws-credentials
            key: secret-key
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: combined-secret
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: azure-keyvault
    kind: SecretStore
  target:
    name: combined-secret
  dataFrom:
  - secretStoreRef:
      name: azure-keyvault
      kind: SecretStore
    extract:
      key: db-*
  - secretStoreRef:
      name: aws-secrets-manager
      kind: SecretStore
    extract:
      key: api-*

dataFrom extracts multiple keys from different SecretStores. db-* from Azure Key Vault and api-* from AWS Secrets Manager are combined into one Secret. This provides a unified view of secrets from multiple sources.

Test secret access

Test that pods can access secrets: kubectl exec -it test-pod -- env | grep PASSWORD. For CSI Driver, check the mounted volume. For ESO, check the synced Secret with kubectl get secret db-credentials-synced -o yaml.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.