Dev./Kubernetes & Helm

Kubernetes: 안전하게 업로드하기 by using Kubeseal

Ivan'show 2023. 10. 14.

728x90

Sealing

helm-chart 로 만든 데이터를 argocd 에서 관리하려면 모든 정보가 담긴 파일들이 git 에 올라가 있어야 한다. 이때 민감한 정보를 어떻게 핸들링 해야하는지 고민이 많았는데, kubeseal 이라는 오픈소스를 사용해보면 어떨까 한다.

<https://github.com/bitnami-labs/sealed-secrets>

Sealed Secrets 는 쿠버네티스에서 비밀 정보를 안전하기 관리하기 위한 도구라고 한다. 클러스터 내 컨트롤러와 클라이언트 측 유틸리티인 kubeseal 로 구성되며 kubeseal 이 SealedSecret 이라는 리소스를 생성하여 클러스터 내에서 일반 Secret 으로 사용된다.

스코핑 설정도 가능하고 argocd 와 함께 사용하여 helm chart 를 안전하게 배포할 수 있다고 한다.

Installation

brew install kubeseal

helm repo add sealed-secrets <https://bitnami-labs.github.io/sealed-secrets>

helm install sealed-secrets -n kube-system --set-string fullnameOverride=sealed-secrets-controller sealed-secrets/sealed-secrets

# result
NAME: sealed-secrets
LAST DEPLOYED: Tue Sep 19 21:23:47 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
....
....
3. Apply the sealed secret

    kubectl create -f mysealedsecret.[json|yaml]

Running 'kubectl get secret secret-name -o [json|yaml]' will show the decrypted secret that was generated from the sealed secret.

Both the SealedSecret and generated Secret must have the same name and namespace.

k get po -A

# result
(venv) kimminhyeok@Ivans-Mac helm-chart % k get po -A
NAMESPACE     NAME                                                READY   STATUS    RESTARTS   AGE
argocd        argocd-application-controller-0                     1/1     Running   0          52m
argocd        argocd-applicationset-controller-745cd84657-vwh62   1/1     Running   0          52m
argocd        argocd-dex-server-684c58b4b5-ghxb4                  1/1     Running   0          52m
argocd        argocd-notifications-controller-f5877f4fb-w5bdb     1/1     Running   0          52m
argocd        argocd-redis-685866888c-xvcbm                       1/1     Running   0          52m
argocd        argocd-repo-server-76bc8c68b9-vfm2d                 1/1     Running   0          52m
argocd        argocd-server-b456cd7d5-tc8w4                       1/1     Running   0          52m
kube-system   cilium-5nb6t                                        1/1     Running   0          27h
....
kube-system   nodelocaldns-t78gj                                  1/1     Running   0          27h
kube-system   sealed-secrets-controller-7667686698-vz4v6          1/1     Running   0          7m1s
kube-system   snapshot-controller-0                               1/1     Running   0          27h
kube-system   startup-script-6vzwn                                1/1     Running   0          27h
kube-system   startup-script-prlcm                                1/1     Running   0          27h

Usage

사용방법은 생각보다 간단한데, 아래와 같은 secret 파일을 seal 해준다는 느낌으로 해주면 된다.

# db-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: db-sns
type: Opaque
data:
    POSTGRES_DB: cG9z    
		POSTGRES_USER: GdyZXM=
    POSTGRES_PASSWORD: c9zX=
    POSTGRES_PORT: NQM==

kubeseal --format=yaml < db-secret.yaml > db-secret-sealed.yaml

그럼 지정한 새로운 파일이 생기는 데,

# db-secret-sealed.yaml

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: db-sns
  namespace: default
spec:
  encryptedData:
    POSTGRES_DB: AgA1Eh11dZx7Rcr8dTnMSUOuZ4QtyDO8cuVa8e8oN1cK77OjTnBQULIOM0Kl88OaPO2ApgyTIbB2qSJWIaGfCQdIkwI/jZde1zqltOof+mXptsXTxs6sV5CyOL3xWZZP68nZn8z6an0R68hgpi/oKQOA6kIG97mEwqq+ihA6KcsfENXXrekdhHawHUhrZ/hBQLZ/K0SnQNuXCxZlrl2rzcT64ubuaP+zW4ohDtoN1DFhWijDZVnlHD2/MCMpAwZ3FFICkvbn9lNrheShODzLSHeawfcIMJPPKXNzyqRJHjV4EsAu3yVcf1POPUeK8K+8f41jQz7wFFHKsVFEVwhs72a4VByIkuYhBMmxLEl6Yj62jCXwAs+f8p+T5xWOgsdGjK+ZRFndCjFtrnxULTR71pTnXC+B5J+hH34LpNmXUsLAEirg8cLHRa9mR5Zzhio6WxwMd0AF0IXf9QyBbgnP6bjqhBQI994x8gqiiFYCurml+JkAYZZG9tEX4sx800K5t+NKdqPe5LZ6f9weQfQ5h76oDiW3WQqsVohsUl/uxwo9T3Uxt5qlrqhfMkUTFGOiSCY2V9u61Jg6lXVSxIs6pYdCM3gl3CqY2tnuMajyHKIDA8QRNSbUbIq9KdExPU3iT0D+bW4eIf5ONp3J/fitIFW7Eu6ryd7ZzA==
    POSTGRES_PASSWORD: AgB51ai9Tby1UgyiLSQH5yxNw/9sNhCUM6+peQQW2H/XnWsN+SAyhd4q7eBhhERKk8hRJoLfVmN7O3B1w3Z7DW6Q26hDvBox+VEdQRyYjEJluHqtetjhe7tU4Tvy1UXEgl0jat0fLezMJadFgQeKyPO9uzg8IfVvnW2c7YmUbSzowVAFT6iJo55QOiUwcTc1J9p3vRmF8U7pZvsfLjBB2esv2KHe4P/xAWhFUqf4Xl/m1y0CyQ3IvlYNS5kSuhB5LXTkhNyLee49X7NZJxnyY2paLU3V+r+Lk2P+TUJ0rfBd+knBJwGTkFMeVfE6DX+4T8ajG+7xPurjGmvCAEuZRWCvp4J5R/k47RZ3yaZB5x1HsiwvmoYF6NW/ha/jUyKUIIooQggNRM5II+0yJw6HXrCJhD1Frp4nrIFEc6lDfQzhjocgtJMVCYlADgIWewxPmkw8w7xOzRi1CLlHCSvn89ZPf3USluKHUITGpa6ZOEb/xRZYjlIH8B6ceeLyqgexOTuMFMEU/dtZwzX51kX7vow3OSe+3Zg9cZKaBuGvyastDnIQf7uwzOZU+aJeWXWxWis9vjtGoXNw71meyira2315DANG7s5C4TTzwqRPhoUtbr9bu6f9VR3wndZwK7Xl7ShnyT04l2mCrlHH10IH7A+dBPkbAKrdhVUfupSh60m+sEQ==
    POSTGRES_PORT: AgCi6o7x97ORLdbZ7PZ8h9sSUOWDFjk3srBBRO8SoPx+luEDssFUlGvYavP5XSSITKjb2PjXEhfHRpX8fQH729xROia2aEKN66s0+Is+zBZG7RZP0jbcvPMeeUalWUwy6SBLtRdw7qMTSjhmR2NvrEc8sfyEZp+zJzQI5xGFXfr4jYTv66NV3OGvU6fjffBaYaoLHEwhPyr+Xp7SteUo8GN+loUNNoYpo6SzmIzVGWZd0fiA1VGLE/4OnK0uJ2WH3Jpx7t/OCa9m7Qn8GBUxfGYFtNkP5XsMLEgZ/hUDh/NkDcaxWhR/DnEUGtU5MpQaMtcO4BYetmGO50mqGST9tTUGsRh5mg8zr8Pq2DwWGkh5C825Rcw4D9docIgrhO1GvJ6LxuA1MzrVoLMUnSJ79pwGe111zg1/mpZ6ZH1C2zfdJV/l4EBGpRn9GYp5yAImB+OUE6xk1iv22ysuo84bZV1p7fIzX6oc76UaWJEhIDp/HJsMKF5mIvMbjdSr/Fz2/Fg8eZm9L83mQENq6upIoy0ha6/nxLDfiUL7giJDP7JncIOfWSQmMHIQo/A++ogmMDrN8f/i4vNcJ+5Yz7dOteeJJvWxGelVOHk1I+XEgS5o7fhUnM2PXsaWrV1S6OjZpvoGts5pvHFT9EA2Fo9T4V1dRuhbMrbxMfMZbK
    POSTGRES_USER: AgBf9EzqA/aXW6JQVAGi21cAXsklst0WGJhht4ODjpEYTYtXUZXBctcIHmEScb/k8Pi7//2avbbTzTp7rREgj3+dBe2v1qO+G0JJ9XTBCDLEpETbjJ6dk+xJCD1cYFDFJ5krCjoyM1F+BqdEvEwVpT3s8Yu1oXMl3O11cq8dqQu+BiJYaR9/jz9eJTIpt4K8O+Y2SesUzx7NSsnRH8J4DAHVcoel3I8nH5hcGSjgZHTOD7GVhpArBKqiruqUuomiGqI4crljU6MsV+bAtMeVzrmHZxAROm+0ZLGkvLMXSr/HnD77nS6nBzHLtm6bkwYpaGVIYjWp8N/XYHyDV2ck+0xxZEEdcQd1/OH5dN6THv+scEfcSHq7JtrlpknSUw9D5HRQAvZDGRC3eYabGLAWLnEqtK9t9XOGiIALWtIe9lJGeAgpDk1B146Qk++/TTWgZkADh4K7m9nncjbEftMlHqNNHR5J2ZSSW6pynV0YHWeaqmA8Y282pcNWDxVMyblWMqadcE6u884Axm12oQpZjn6L2UwzwZiXq4Y3a4gszBq3WzX0hhwBfk4IBq9TLHjBze5Y9LE1tBoGG2L2lKPpnMi5X8gxX5s35BtIGga59bt+ZVGga/NNCtaJbtwJ9n2e+Wyuw3UVxAsSUq5BqdWIRc0wnMtkJw5aUcSv4/F36Pxn19WnMIb6A==
  template:
    metadata:
      creationTimestamp: null
      name: db-sns
      namespace: default
    type: Opaque
---

이런식으로 암호화를 한번 더 해준 형태로 된다.

기존에 있던 secret 파일은 gitignore 에 등록해서 업로드 되게 하지 않고, 새로 생성한 파일을 업로드하면 클러스터와 클라이언트 내에서 동작하는 키값으로 해당 파일을 디코딩하여 기존에 있던 파일을 만들어서 사용하는 것 같다.