카테고리 보관물: Container

K8S

CKAD Secrets 문제 후기 – 20250618

댓글 남기기

출제 문제

※ 기억에 의존해 복기하는 문제라 오류가 있을 수 있습니다.
참고만 부탁 드립니다.

  • postgres deployment의 env에 하드코딩 된 argument를 참고하여 secret 생성
    secret name : postgres
    namespace : devops
    key name : username, database, password
  • 생성된 postgres secret을 postgres deployment의 env에 하드코딩 된 argument를 수정 반영
    ※ postgres deployment YAML파일을 제공하고 수정해서 재배포하라고 나왔을 수 있음

# Kubernetes 메뉴얼
https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/
https://kubernetes.io/docs/tasks/inject-data-application/distribute-credentials-secure/

# postgres deployment의 env 확인
$ kubectl describe deployment -n devops postgres | grep -A3 Environment
    Environment:
      PG_USERNAME:       pgadmin
      PG_DATABASE:       pgdatabase
      PG_PASSWORD:       pgpwd1234
# Secret 생성
$ kubectl create secret generic postgres --from-literal=username=pgadmin --from-literal=database=pgdatabase --from-literal=password=pgpwd1234 -n devops

$ kubectl get secrets -n devops
# 제공된 postgres deployment의 yaml 수정 및 저장
$ vi postgres-deployment.yaml
...
spec:
  containers:
  - name: postgres
    image: postgres:stable
    env:
    - name: PG_USERNAME
      valueFrom:
        secretKeyRef:
          name: postgres
          key: username
    - name: PG_DATABASE
      valueFrom:
        secretKeyRef:
          name: postgres
          key: database
    - name: PG_PASSWORD
      valueFrom:
        secretKeyRef:
          name: postgres
          key: password
...
# deployment 재배포
$ kubectl apply -f postgres-deployment.yaml

# Environment 값 확인
$ kubectl describe deployment -n devops postgres
K8S

CKAD Cronjob 문제 후기 – 20250618

댓글 남기기

출제 문제

※ 기억에 의존해 복기하는 문제라 오류가 있을 수 있습니다.
참고만 부탁 드립니다.

  • 30분 마다 job을 스케쥴 처리할 수 있는 cronjob 생성
  • cronjob 이름은 grep, namespace는 devops 사용할 것
  • 성공 history는 64개, 실패 history는 160개 보관
  • job이 실행되고 8초 이내에 완료 되지 못하면 중단할 것
  • pod가 중단 되더라도 재실행 되지 않을 것
  • container 이름: busybox
    image: busybox:stable,
    command: [“grep”, “-i”, “NAMESERVER”, “/etc/resolv.conf”]
  • cronjob 테스트를 위해 job을 별도로 생성해볼 것
    job 이름: grep-test
    namespace: devops

# Kubernetes 메뉴얼
https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs
https://kubernetes.io/docs/concepts/workloads/controllers/job/

# cronjob-grep.yaml 생성

apiVersion: batch/v1
kind: CronJob
metadata:
  name: grep
  namespace: devops
spec:
  schedule: "*/30 * * * *"
  successfulJobsHistoryLimit: 64
  failedJobsHistoryLimit: 160
  jobTemplate:
    spec:
      activeDeadlineSeconds: 8
      template:
        spec:
          containers:
          - name: busybox
            image: busybox:stable
            imagePullPolicy: IfNotPresent
            command: ["grep", "-i", "NAMESERVER", "/etc/resolv.conf"]
          restartPolicy: Never
# job-grep-test.yaml 생성

apiVersion: batch/v1
kind: Job
metadata:
  name: grep-test
  namespace: devops
spec:
  activeDeadlineSeconds: 8
  template:
    spec:
      containers:
      - name: busybox
        image: busybox:stable
        imagePullPolicy: IfNotPresent
        command: ["grep", "-i", "NAMESERVER", "/etc/resolv.conf"]
      restartPolicy: Never
# cronjob 생성 및 상태 확인
$ k apply -f cronjob-grep.yaml
$ k get cronjob -n devops
$ k describe cronjob -n devops grep

# job 생성 및 실행 상태 확인
$ k apply -f job-grep-test.yaml
$ k get job -n devops
$ k describe job -n devops grep-test
$ k get po -n devops # pod의 Completed 상태 확인
K8S

kubectl : certificate has expired or is not yet valid

댓글 남기기

어느날 kubernetes 상태 확인을 위해 kubectl을 치자 발생하는 에러 메시지…

E1218 05:21:48.113070 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.0.200:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.115822 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.0.200:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.118161 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.0.200:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.121162 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.0.200:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.124428 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.0.200:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
Unable to connect to the server: tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z

x509 같은 키워드를 보아 인증서로 추측 되므로 빠른 구글링으로 조치 방법 확인…

kubeadm certs check-expiration 로 인증서 만료 여부를 확인해보겠습니다…

test@test-master-01:~$ sudo kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
apiserver                  Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
apiserver-etcd-client      Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
apiserver-kubelet-client   Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
controller-manager.conf    Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
etcd-healthcheck-client    Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
etcd-peer                  Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
etcd-server                Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
front-proxy-client         Dec 05, 2024 15:09 UTC   <invalid>       front-proxy-ca          no
scheduler.conf             Dec 05, 2024 15:09 UTC   <invalid>       ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 03, 2033 15:09 UTC   8y              no
etcd-ca                 Dec 03, 2033 15:09 UTC   8y              no
front-proxy-ca          Dec 03, 2033 15:09 UTC   8y              no

기존 인증서 백업 처리

sudo cp -pr /etc/kubernetes/ /etc/kubernetes_backup

인증서 갱신 및 확인

test@test-master-01:~$ sudo kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[renew] Error reading configuration from the Cluster. Falling back to default configuration

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.
test@test-master-01:~$ sudo kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 17, 2025 20:29 UTC   364d            ca                      no
apiserver                  Dec 17, 2025 20:29 UTC   364d            ca                      no
apiserver-etcd-client      Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Dec 17, 2025 20:29 UTC   364d            ca                      no
controller-manager.conf    Dec 17, 2025 20:29 UTC   364d            ca                      no
etcd-healthcheck-client    Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
etcd-peer                  Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
etcd-server                Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
front-proxy-client         Dec 17, 2025 20:29 UTC   364d            front-proxy-ca          no
scheduler.conf             Dec 17, 2025 20:29 UTC   364d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 03, 2033 15:09 UTC   8y              no
etcd-ca                 Dec 03, 2033 15:09 UTC   8y              no
front-proxy-ca          Dec 03, 2033 15:09 UTC   8y              no

kubectl을 실행 해도 kube-apiserver, kube-controller-manager, kube-scheduler and etcd 을 재시작 하기 전까지는 계속 오류 발생…

kubectl을 사용하는 계정의 홈디렉토리에도 config에 인증서가 포함되어 있어 해당 파일도 덮어 씌워 줍니다. (root로 실행할 경우 /root/.kube/config)

test@test-master-01:~$ sudo cp /etc/kubernetes/admin.conf /home/test/.kube/config
test@test-master-01:~$ chown test:test /home/test/.kube/config
test@test-master-01:~$ ll /home/test/.kube/config
-rw------- 1 test test 5650 Dec 18 05:42 /home/test/.kube/config

프로세스 중지 및 서비스 재시작

test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-apiserver)
test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-controller-manager)
test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-scheduler)
test@test-master-01:~$ sudo systemctl restart kubelet
test@test-master-01:~$ sudo systemctl daemon-reload

kubectl을 쳐보겠습니다…

test@test-master-01:~$ kubectl get po -A
NAMESPACE          NAME                                                      READY   STATUS      RESTARTS       AGE
ingress-nginx      ingress-nginx-controller-6dfcb8658d-8rhbq                 1/1     Running     1 (72d ago)    172d
kube-system        calico-kube-controllers-7ddc4f45bc-d8259                  1/1     Running     1 (72d ago)    147d
kube-system        calico-node-5mk6f                                         1/1     Running     11 (72d ago)   376d
kube-system        calico-node-b6jxh                                         1/1     Running     16 (72d ago)   376d
kube-system        calico-node-qqmt4                                         1/1     Running     14 (72d ago)   376d
kube-system        calico-node-xjhg4                                         1/1     Running     10 (72d ago)   192d
kube-system        coredns-5dd5756b68-t2pq8                                  1/1     Running     11 (72d ago)   377d
kube-system        coredns-5dd5756b68-tx2xj                                  1/1     Running     11 (72d ago)   377d
kube-system        etcd-k8s-master-01                                        1/1     Running     13 (72d ago)   377d
kube-system        kube-apiserver-k8s-master-01                              1/1     Running     19 (68s ago)   377d
kube-system        kube-controller-manager-k8s-master-01                     1/1     Running     15 (60s ago)   377d
kube-system        kube-proxy-22kqn                                          1/1     Running     7 (72d ago)    173d
kube-system        kube-proxy-b9mbg                                          1/1     Running     2 (72d ago)    173d
kube-system        kube-proxy-n4q6t                                          1/1     Running     7 (72d ago)    173d
kube-system        kube-proxy-x649t                                          1/1     Running     6 (72d ago)    173d
kube-system        kube-scheduler-k8s-master-01                              1/1     Running     15 (54s ago)   377d
kube-system        metrics-server-777dff589b-hmdhl                           1/1     Running     2 (72d ago)    169d
mariadb-system     mariadb-79d8f666bc-jnzlm                                  1/1     Running     1 (72d ago)    170d
metallb-system     controller-686877b9fc-9x9mh                               1/1     Running     2 (72d ago)    172d
metallb-system     speaker-7kn6b                                             1/1     Running     1 (72d ago)    172d
metallb-system     speaker-c7rrx                                             1/1     Running     1 (72d ago)    172d
metallb-system     speaker-hcvlw                                             1/1     Running     1 (72d ago)    172d

잘 됩니다! 완료!