OpenShift 4 - 用 External Secret 集成 Hashicorp Vault

原创已于 2025-04-08 08:11:59 修改 · 988 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#openshift #kubernetes #docker

于 2022-10-12 15:06:22 首次发布

OpenShift 4 同时被 3 个专栏收录

272 篇文章

订阅专栏

安全

76 篇文章

订阅专栏

secret

7 篇文章

订阅专栏

《OpenShift / RHEL / DevSecOps 汇总目录》
文本已在 OpenShift 4.10 环境中进行验证。

文章目录

什么是 External Secret
安装配置
参考

什么是 External Secret

External Secret 用来集成外部 Secret 管理系统，如 AWS Secrets Manager、HashiCorp Vault、Google Secrets Manager、Azure Key Vault。External Secret 从外部 Secret 管理系统中读取所需的信息，并自动将这些值注入到指定的 Secret 中。
在这里插入图片描述

安装配置

安装 Helm

curl -OL https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/helm/latest/helm-linux-amd64
chmod +x helm-linux-amd64
sudo mv helm-linux-amd64 /usr/local/bin/helm

安装 Vault 环境

安装 hashicorp 的 Helm Repo。

helm repo add hashicorp https://helm.releases.hashicorp.com

下载 values.openshift.yaml 文件。

curl -OL https://raw.githubusercontent.com/hashicorp/vault-helm/main/values.openshift.yaml

查看 values.openshift.yaml 文件内容。

global:
  openshift: true
injector:
  image:
    repository: "registry.connect.redhat.com/hashicorp/vault-k8s"
    tag: "1.0.0-ubi"
  agentImage:
    repository: "registry.connect.redhat.com/hashicorp/vault"
    tag: "1.11.3-ubi"
server:
  image:
    repository: "registry.connect.redhat.com/hashicorp/vault"
    tag: "1.11.3-ubi"

执行命令在 vault 项目中安装 Hashicorp Vault。

oc new-project vault
helm install vault-server hashicorp/vault -f values.openshift.yaml

在 vault-server-0 的 Pod 状态为 Running 后初始化 Vault 服务。

$ oc get pod -n vault
NAME                                           READY   STATUS    RESTARTS   AGE
vault-server-0                                 0/1     Running   0          44s
vault-server-agent-injector-57dddc846b-fvxxk   1/1     Running   0          47s
 
$ oc exec -n vault -ti vault-server-0 -- vault operator init
Unseal Key 1: zRJqpv4wQWRMj0mkkSTHu9tpeaNu+x7xcnLf/B8gFvOD
Unseal Key 2: FPhr0NRHzhY21Jk3Qx2WsMmqBj+4xW2ZlHauw4DUHbnP
Unseal Key 3: x+bK1TWlrm11tL4a/jTOAyz89nVC58SBJf0e1ZHKEhYK
Unseal Key 4: clVpIBgJPaZoKteysjuBYULonEe5hOZ/Fs4Ky/ZUe38R
Unseal Key 5: DhDr8Y01Vq3kGrdeJBzIeKyIxf4zrmxuBTalZhxsIsJt
 
Initial Root Token: hvs.BLLImnznmIM1rf2ppHfyQIAP

使用上面的任意 3 个 Unseal Key 解封 Vault 环境，然后用 Root Token 登录 Vault。

oc exec -n vault -ti vault-server-0 -- vault operator unseal zRJqpv4wQWRMj0mkkSTHu9tpeaNu+x7xcnLf/B8gFvOD
oc exec -n vault -ti vault-server-0 -- vault operator unseal x+bK1TWlrm11tL4a/jTOAyz89nVC58SBJf0e1ZHKEhYK
oc exec -n vault -ti vault-server-0 -- vault operator unseal DhDr8Y01Vq3kGrdeJBzIeKyIxf4zrmxuBTalZhxsIsJt
oc exec -n vault -it vault-server-0 -- vault login hvs.BLLImnznmIM1rf2ppHfyQIAP

在 Vault 中的 secret/ 路径中增加名为 openshiftpullsecret 的 kv 类 Secret 存储，然后向其中添加数据 dockerconfigjson。

$ oc exec -n vault -it vault-server-0 -- vault secrets enable -path=secret/ kv
Success! Enabled the kv secrets engine at: secret/
 
$ oc exec -n vault -it vault-server-0 -- vault kv put secret/openshiftpullsecret dockerconfigjson='{"auths":{"cloud.openshift.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"quay.io":{"auth":"ZZMVhJRUJUR1I3WUwxN05VMQ==","email":"example@redhat.com"},"registry.connect.redhat.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"registry.redhat.io":{"auth":"==","email":"example@redhat.com"}}}'
Success! Data written to: secret/openshiftpullsecret
 
$ oc exec -n vault -it vault-server-0 -- vault kv get secret/openshiftpullsecret
========== Data ==========
Key                 Value
---                 -----
dockerconfigjson    {"auths":{"cloud.openshift.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"quay.io":{"auth":"ZZMVhJRUJUR1I3WUwxN05VMQ==","email":"example@redhat.com"},"registry.connect.redhat.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"registry.redhat.io":{"auth":"==","email":"example@redhat.com"}}}

安装 External Secret

执行命令，通过 Helm 在 external-secrets 项目中安装 external-secrets 运行环境。

$ helm repo add external-secrets https://charts.external-secrets.io
$ helm install external-secrets \
  external-secrets/external-secrets \
  -n external-secrets --create-namespace

查看运行的Pod 环境。

$ oc get pod -n external-secrets
NAME                                                READY   STATUS    RESTARTS   AGE
external-secrets-7886b578b4-dqplk                   1/1     Running   0          14s
external-secrets-cert-controller-5578bf565f-j2c44   1/1     Running   0          13s
external-secrets-webhook-69c58546c6-262bj           1/1     Running   0          13s

在应用项目中通过 External Secret 获取 Vault 中的 Secret

使用 Value 的 Root Token 创建一个 Secret。

$ ROOT_TOKEN=$(echo -n "ROOT-TOKEN" | base64)
$ cat << EOF | oc apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: vault-token
  namespace: vault
data:
  token: ${ROOT_TOKEN}
EOF

使用 vault 服务地址和上一步生成的 vault-token 作为认证信息，创建一个 ClusterSecretStore 对象。

$ cat << EOF | oc apply -f -
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: vault-backend
spec:
  provider:
    vault:
      server: "http://vault-server-internal.vault:8200"
      path: "secret"
      version: "v1"
      auth:
        tokenSecretRef:
          name: "vault-token"
          key: "token"
          namespace: vault
EOF

创建应用项目 my-app。

$ oc new-project my-app

创建一个 ExternalSecret 对象，它会通过 ClusterSecretStore 访问 Vault 后端 secret/openshiftpullsecret 中的 dockerconfigjson 主键，并将其保存到名为 pullsecret 的 Secret 中。

$ cat << EOF | oc apply -f -
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: pullsecret-eso
  namespace: my-app
spec:
  refreshInterval: "15s"
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: pullsecret
  data:
  - secretKey: dockerconfigjson
    remoteRef:
      key: secret/openshiftpullsecret
      property: dockerconfigjson
EOF

查看创建的 externalsecret 对象，其状态为 SecretSynced 说明已经从 Vault 后端完成 Secret 的数据同步。

$ oc get externalsecret pullsecret-eso -n my-app
NAME             STORE           REFRESH INTERVAL   STATUS         READY
pullsecret-eso   vault-backend   15s                SecretSynced   True
 
$ oc get externalsecret pullsecret-eso -n my-app -oyaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"external-secrets.io/v1alpha1","kind":"ExternalSecret","metadata":{"annotations":{},"name":"pullsecret-eso","namespace":"my-app"},"spec":{"data":[{"remoteRef":{"key":"secret/openshiftpullsecret","property":"dockerconfigjson"},"secretKey":"dockerconfigjson"}],"refreshInterval":"15s","secretStoreRef":{"kind":"ClusterSecretStore","name":"vault-backend"},"target":{"name":"pullsecret"}}}
  creationTimestamp: "2022-09-25T13:58:37Z"
  generation: 1
  name: pullsecret-eso
  namespace: my-app
  resourceVersion: "454324"
  uid: b04ff758-dae9-4b3a-a4e2-14c5f94b3c1e
spec:
  data:
  - remoteRef:
      conversionStrategy: Default
      decodingStrategy: None
      key: secret/openshiftpullsecret
      property: dockerconfigjson
    secretKey: .dockerconfigjson
  refreshInterval: 15s
  secretStoreRef:
    kind: ClusterSecretStore
    name: vault-backend
  target:
    creationPolicy: Owner
    deletionPolicy: Retain
    name: pullsecret
status:
  conditions:
  - lastTransitionTime: "2022-09-25T13:58:38Z"
    message: could not get secret data from provider
    reason: SecretSyncedError
    status: "False"
    type: Ready
  refreshTime: null

查看名为 pullsecret 的 Secret 对象，它是 External Secret 根据 pullsecret-eso 的要求自动生成的。

$ oc get secrets pullsecret -n my-app
NAME            TYPE     DATA   AGE
pullsecret      Opaque   1      116s

$ oc get secrets pullsecret -n my-app -o yaml
apiVersion: v1
data:
  dockerconfigjson: eyJhdXRocyI6eyJjbG91ZC5vcGVuc2hpZnQuY29tIjp7ImF1dGgiOiIzQmxibk5vYVdaMExYSmw9PSIsImVtYWlsIjoiZXhhbXBsZUByZWRoYXQuY29tIn0sInF1YXkuaW8iOnsiYXV0aCI6IlpaTVZoSlJVSlVSMUkzV1V3eE4wNVZNUT09IiwiZW1haWwiOiJleGFtcGxlQHJlZGhhdC5jb20ifSwicmVnaXN0cnkuY29ubmVjdC5yZWRoYXQuY29tIjp7ImF1dGgiOiIzQmxibk5vYVdaMExYSmw9PSIsImVtYWlsIjoiZXhhbXBsZUByZWRoYXQuY29tIn0sInJlZ2lzdHJ5LnJlZGhhdC5pbyI6eyJhdXRoIjoiPT0iLCJlbWFpbCI6ImV4YW1wbGVAcmVkaGF0LmNvbSJ9fX0=
immutable: false
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"external-secrets.io/v1alpha1","kind":"ExternalSecret","metadata":{"annotations":{},"name":"pullsecret-eso","namespace":"my-app"},"spec":{"data":[{"remoteRef":{"key":"secret/openshiftpullsecret","property":"dockerconfigjson"},"secretKey":".dockerconfigjson"}],"refreshInterval":"15s","secretStoreRef":{"kind":"ClusterSecretStore","name":"vault-backend"},"target":{"name":"pullsecret"}}}
    reconcile.external-secrets.io/data-hash: 2694caf8571c330a93cdbe61167ce782
  creationTimestamp: "2022-09-25T14:06:38Z"
  name: pullsecret
  namespace: my-app
  ownerReferences:
  - apiVersion: external-secrets.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: ExternalSecret
    name: pullsecret-eso
    uid: b04ff758-dae9-4b3a-a4e2-14c5f94b3c1e
  resourceVersion: "456916"
  uid: 6e02accd-cbf2-4885-b62d-4ce667806ce9
type: Opaque

确认 pullsecret 中的数据是经过 base64 后的受保护信息。

$ echo "$(oc get secrets pullsecret -n my-app -o jsonpath='{.data.dockerconfigjson}')" | base64 -d
{"auths":{"cloud.openshift.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"quay.io":{"auth":"ZZMVhJRUJUR1I3WUwxN05VMQ==","email":"example@redhat.com"},"registry.connect.redhat.com":{"auth":"3BlbnNoaWZ0LXJl==","email":"example@redhat.com"},"registry.redhat.io":{"auth":"==","email":"example@redhat.com"}}}

参考

https://www.goglides.dev/bkpandey/how-to-manage-secrets-in-openshiftkubernetes-using-vault-and-external-secrets-1n6k
https://github.com/hashicorp/vault-helm/blob/main/values.openshift.yaml
https://faun.pub/setup-a-production-ready-vault-cluster-with-kubernetes-and-azure-30943ff91bf3
https://medium.com/hashicorp-engineering/consuming-secrets-in-your-openshift-applications-using-hashicorp-vault-injector-4fbf84156e7
https://picluster.ricsanfre.com/docs/vault/