Stage 4 · Provision

Continuous Delivery & GitOps

Encrypted Secrets

Sealed Secrets, Mozilla SOPS, age keys, and External Secrets Operator workflows — secrets management for GitOps.

7 min readInfrastructure as CodeProvision

Secrets in GitOps

GitOps requires all desired state in Git. But Kubernetes secrets are base64-encoded, not encrypted. Storing them in plaintext exposes credentials. The solution is to encrypt secrets before committing, store references instead of values, or use a dedicated secrets manager.

Each approach has tradeoffs. SealedSecrets encrypt in Git. SOPS encrypts in Git with better key management. External Secrets Operator stores secrets externally and syncs at runtime. Vault provides dynamic secrets with short-lived credentials.

Sealed Secrets Deep Dive

SealedSecret with template


apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: production
  labels:
    app: my-app
spec:
  encryptedData:
    username: AgBy3i4OJSWK+PiTySYZZA9rO...
    password: AgC4P8b2ZqYv7k3x9m1N...
    connection-string: AgD8n9x2QwRt4k3m1p5S...
  template:
    metadata:
      name: db-credentials
      namespace: production
      labels:
        app: my-app
    type: Opaque

The template section controls the metadata of the decrypted Secret. Labels and annotations are preserved. The encryptedData section contains the encrypted values. Each value is encrypted independently with the cluster key.

SealedSecrets CLI operations


$ kubeseal --format yaml < secret.yaml > sealed-secret.yaml
$ kubeseal --format yaml --re-encrypt < old-sealed.yaml > new-sealed.yaml
$ kubeseal --fetch-cert > pub-sealed.pem

# Rotate keys
$ kubectl delete secret -n kube-system -l app=sealed-secrets
$ kubectl rollout restart deployment sealed-secrets -n kube-system

re-encrypt re-encrypts with the current key. fetch-cert exports the public key for offline encryption. Rotating keys deletes the old key and generates a new one. Existing SealedSecrets must be re-encrypted after rotation.

SOPS with age Keys

SOPS setup with age


$ age-keygen -o keys.txt
# Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

$ sops --encrypt --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p secrets.yaml
$ sops --decrypt secrets.yaml
$ sops --encrypt --in-place --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p secrets.yaml

age is a modern encryption tool that is simpler than GPG. The keygen command generates a key pair. SOPS encrypts values using the public key. Only holders of the private key can decrypt.

.sops.yaml configuration


creation_rules:
  - path_regex: prod/.*
    age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
  - path_regex: dev/.*
    age: age1devkeyplaceholder...

The .sops.yaml file maps file paths to encryption keys. Files in the prod directory use the production key. Files in dev use the dev key. This ensures environment isolation.

External Secrets Operator Deep Dive

ExternalSecret with dataFrom


apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  dataFrom:
    - extract:
        key: prod/database/credentials

dataFrom extracts all key-value pairs from a secret. The extract key specifies the secret in the external manager. All values become keys in the Kubernetes Secret. This is simpler than mapping each value individually.

ClusterSecretStore for cross-namespace


apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: vault
spec:
  provider:
    vault:
      server: "https://vault.example.com"
      path: "secret"
      version: "v2"
      auth:
        kubernetes:
          mountPath: "kubernetes"
          role: "external-secrets"
          serviceAccountRef:
            name: external-secrets
            namespace: external-secrets

ClusterSecretStore is cluster-scoped and available to all namespaces. This allows different namespaces to use the same Vault server without duplicating the configuration. The kubernetes auth method uses pod service accounts.

Secret Rotation

Automatic rotation with External Secrets


apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
spec:
  refreshInterval: 5m  # Check for changes every 5 minutes
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
    - secretKey: password
      remoteRef:
        key: prod/database/credentials
        property: password
        version: AWSCURRENT

The refreshInterval controls how often External Secrets checks for updates. When the secret changes in AWS Secrets Manager, External Secrets syncs it to Kubernetes. The version AWSCURRENT always points to the latest version.

Best Practices

Use External Secrets Operator for automatic rotation.
Use SOPS with age for simple encrypt-in-Git workflows.
Use SealedSecrets only when External Secrets is not available.
Rotate encryption keys periodically.
Restrict Git repository access to authorized personnel.
Audit secret access through the secrets manager.
Use namespace-scoped secrets to limit exposure.

Combine SOPS with Flux for best DX

Flux natively decrypts SOPS-encrypted files during reconciliation. Developers encrypt with SOPS, commit to Git, and Flux handles decryption automatically. No controllers or CRDs needed inside the cluster.

Never commit plaintext secrets

Even in private repositories, plaintext secrets are a security risk. Repository access is often broader than expected. Always encrypt secrets before committing to Git.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.