DevSecOps

Kubernetes Secrets Management Best Practices

Advanced
14 minutes
Updated October 2026

Secure Kubernetes secrets using external secret managers, encryption at rest, and access controls — because base64 is NOT encryption.

Table of Contents

The Problem with Default Secrets

Kubernetes Secrets are base64-encoded, not encrypted by default:

kubectl get secret db-password -o jsonpath='{.data.password}' | base64 -d
# Anyone with read access can decode this instantly

Additional risks:

  • Secrets stored in etcd in plaintext (unless encryption configured)
  • Visible in kubectl describe events if misused
  • Often committed to Git by accident
  • Accessible to anyone with namespace read access

Encryption at Rest

Enable encryption for secrets in etcd with an EncryptionConfiguration:

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: <base64-encoded-32-byte-key>
      - identity: {}

Managed Clusters

AKS — use Azure Key Vault KMS provider:

az aks update \
  --resource-group my-rg \
  --name my-cluster \
  --enable-azure-keyvault-kms \
  --azure-keyvault-kms-key-id <key-id>

GKE/EKS — use their native KMS integration. Managed clusters encrypt etcd by default at the infrastructure level, but KMS envelope encryption adds a second layer you control.

Rotate Encryption Keys

# Re-encrypt all existing secrets after enabling encryption
kubectl get secrets --all-namespaces -o json | \
  kubectl replace -f -

RBAC for Secrets

Secrets respect namespace boundaries — restrict access tightly:

# role.yaml — read-only access to specific secrets
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["db-credentials"]     # only this secret
  verbs: ["get"]                        # no list — prevents enumeration
# Check who can read secrets
kubectl auth can-i get secrets --as=system:serviceaccount:production:my-sa -n production

# Audit access
kubectl get clusterrolebindings -o wide | grep secret

Key rules:

  • Use resourceNames to limit to specific secrets
  • Avoid granting list/watch on secrets (exposes all secrets in the namespace)
  • get on a secret = full decode access — treat it as "read the password"

Choosing Your Approach

The decision tree:

  • "We have a cloud provider's secret manager" → External Secrets Operator. Syncs AWS SM/Azure KV/GCP SM into K8s Secrets — apps see normal Secrets, rotation happens in the vault.
  • "We want secrets in git but safe" → Sealed Secrets. Encrypt once, commit the encrypted blob, the cluster controller decrypts it. No external infrastructure needed.
  • "We want zero secrets in etcd" → CSI Secrets Store driver. Mounts directly from the vault into the pod — the Secret object never exists in Kubernetes at all.
  • "We're just getting started" → Enable etcd encryption + strict RBAC + never commit Secrets to git. The baseline hygiene that every approach builds on.

Most teams land on External Secrets — it balances operational simplicity (apps don't change) with real security (source of truth lives in a managed vault, not etcd).

External Secrets Operator

Store secrets in a real vault, sync to Kubernetes:

helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets -n external-secrets --create-namespace

Sync from Azure Key Vault

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: azure-kv
  namespace: production
spec:
  provider:
    azurekv:
      vaultUrl: "https://my-keyvault.vault.azure.net"
      authType: WorkloadIdentity
      serviceAccountRef:
        name: external-secrets-sa
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: azure-kv
    kind: SecretStore
  target:
    name: db-credentials        # K8s secret created automatically
  data:
    - secretKey: password
      remoteRef:
        key: db-password        # key name in Key Vault

Benefits: automatic rotation, centralized audit, secrets never in Git.

Azure Key Vault Integration (CSI Driver)

Mount secrets directly into pods — no Kubernetes Secret object needed:

# Enable the add-on on AKS
az aks enable-addons \
  --addons azure-keyvault-secrets-provider \
  --resource-group my-rg \
  --name my-cluster
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: app-secrets
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "<identity-client-id>"
    keyvaultName: "my-keyvault"
    objects: |
      array:
        - |
          objectName: db-password
          objectType: secret
    tenantId: "<tenant-id>"
# Pod mounts secrets as a volume
volumes:
  - name: secrets
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: app-secrets

Sealed Secrets (GitOps)

If you want secrets in Git safely — encrypt them with Sealed Secrets (Bitnami):

# Install controller
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/latest/download/controller.yaml

# Install kubeseal CLI
brew install kubeseal

# Create an encrypted secret safe for Git
kubectl create secret generic db-creds \
  --from-literal=password=s3cret \
  --dry-run=client -o yaml | \
kubeseal -o yaml > sealed-secret.yaml

Commit sealed-secret.yaml to Git — only the cluster's private key can decrypt it.

Common Mistakes

1. Secrets in environment dumps

# Anyone with exec access sees all secrets
kubectl exec -it pod -- env

Mount as files when possible — they don't appear in env output or crash dumps.

2. Secrets in ConfigMaps

# NEVER do this
kind: ConfigMap
data:
  password: "s3cret"

ConfigMaps have no encryption or access restrictions.

3. Secrets in image layers

# WRONG — persists in image history even if deleted
RUN echo $API_KEY > /tmp/key && rm /tmp/key

4. Overly broad RBAC

# DANGEROUS — reads ALL secrets in ALL namespaces
resources: ["secrets"]
verbs: ["get", "list", "watch"]

5. Forgetting audit logging

# audit-policy.yaml
rules:
  - level: Metadata
    resources:
      - group: ""
        resources: ["secrets"]

Secret Rotation Checklist

  1. Update the secret in your vault / secret store
  2. ESO syncs automatically (respecting refreshInterval)
  3. Restart pods or use a reloader to pick up new values:
# stakater/reloader annotation
annotations:
  reloader.stakater.com/auto: "true"

Quick Reference

ApproachSecrets in Git?EncryptionRotation
Plain K8s SecretNeveretcd encryption optionalManual
External SecretsNoIn vaultAutomatic sync
CSI DriverNoIn vaultOn pod restart
Sealed SecretsYes (encrypted)Cluster keyRe-seal
Written & Reviewed by Yasir Masood, CloudOpsGuide Editor

Our tutorials and troubleshooting runbooks are written by a DevOps practitioner with hands-on experience operating Kubernetes, Terraform, CI/CD pipelines, and Linux infrastructure in production. Commands, manifests, and configurations are verified against current tooling releases. Learn more about our editorial process or report a correction or outdated flag.