카테고리 보관물:  IT

Certificate

Let’s Encrypt 인증서 생성 실패

댓글 남기기

certbot을 통해 Let’s Encrypt 인증서 생성을 진행할 때 인증서 생성이 실패하는 경우가 있습니다.

다양한 경우의 수가 있겠지만 이 경우는 selinux의 차단으로 인해 발생하는 경우입니다.

[root@test ~]# sudo certbot certonly --cert-name sierracloud.kro.kr -d www.sierracloud.kro.kr

2025-01-01 01:13:22,268:DEBUG:certbot._internal.main:certbot version: 2.11.0
2025-01-01 01:13:22,268:DEBUG:certbot._internal.main:Location of certbot entry point: /bin/certbot
2025-01-01 01:13:22,268:DEBUG:certbot._internal.main:Arguments: ['--cert-name', 'sierracloud.kro.kr', '-d', 'www.sierracloud.kro.kr', '-v']
2025-01-01 01:13:22,268:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2025-01-01 01:13:22,275:DEBUG:certbot._internal.log:Root logging level set at 20
2025-01-01 01:13:22,276:DEBUG:certbot._internal.plugins.selection:Requested authenticator None and installer None
2025-01-01 01:13:22,276:DEBUG:certbot._internal.plugins.selection:Multiple candidate plugins: * standalone
Description: Runs an HTTP server locally which serves the necessary validation files under the /.well-known/acme-challenge/ request path. Suitable if there is no HTTP server already running. HTTP challenge only (wildcards not supported).
Interfaces: Authenticator, Plugin
Entry point: EntryPoint(name='standalone', value='certbot._internal.plugins.standalone:Authenticator', group='certbot.plugins')
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x7f1c2f226fa0>
Prep: True
......
2025-01-01 01:13:43,818:DEBUG:certbot.display.ops:Validator rejected "" when prompting for "Input the webroot for www.sierracloud.kro.kr:"
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot/display/ops.py", line 339, in _get_validated
    validator(raw)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 343, in _validate_webroot
    raise errors.PluginError(webroot_path + " does not exist or is not a directory")
certbot.errors.PluginError:  does not exist or is not a directory
2025-01-01 01:13:43,820:DEBUG:certbot._internal.display.obj:Notifying user:  does not exist or is not a directory
2025-01-01 01:13:52,401:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 88, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 112, in perform
    self._set_webroots(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 129, in _set_webroots
    new_webroot = self._prompt_for_webroot(achall.domain,
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 151, in _prompt_for_webroot
    webroot = self._prompt_for_new_webroot(domain, True)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 178, in _prompt_for_new_webroot
    raise errors.PluginError(
certbot.errors.PluginError: Every requested domain must have a webroot when using the webroot plugin.

2025-01-01 01:13:52,401:DEBUG:certbot._internal.error_handler:Calling registered functions
2025-01-01 01:13:52,401:INFO:certbot._internal.auth_handler:Cleaning up challenges
2025-01-01 01:13:52,402:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2025-01-01 01:13:52,402:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.9/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/auth_handler.py", line 88, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 112, in perform
    self._set_webroots(achalls)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 129, in _set_webroots
    new_webroot = self._prompt_for_webroot(achall.domain,
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 151, in _prompt_for_webroot
    webroot = self._prompt_for_new_webroot(domain, True)
  File "/usr/lib/python3.9/site-packages/certbot/_internal/plugins/webroot.py", line 178, in _prompt_for_new_webroot
    raise errors.PluginError(
certbot.errors.PluginError: Every requested domain must have a webroot when using the webroot plugin.
2025-01-01 01:13:52,404:ERROR:certbot._internal.log:Every requested domain must have a webroot when using the webroot plugin.

사용중인 Apache의 경우 설치시 selinux의 정책에 자동으로 추가 되어 허용 되었지만 certbot이 도메인 인증시 사용되는 80, 443 포트용 프로세스는 selinux에 정책이 등록 되지 않아 차단 되는 것으로 보입니다.
정책 추가보다는 selinux를 통한 제어가 불필요 하기 때문에 비활성화 처리 하도록 하겠습니다.

[root@test ~]# vi /etc/sysconfig/selinux
SELINUX=disabled

[root@test ~]# init 6

# 재기동 이후 selinux 상태 확인
[root@test ~]# sestatus -v
SELinux status:                 disabled

certbot을 통해 Let’s Encrypt 인증서 재생성을 진행해보도록 하겠습니다.

[root@test ~]# sudo certbot certonly --cert-name sierracloud.kro.kr -d www.sierracloud.kro.kr

2025-01-01 01:15:55,925:DEBUG:certbot._internal.main:certbot version: 2.11.0
2025-01-01 01:15:55,926:DEBUG:certbot._internal.main:Location of certbot entry point: /bin/certbot
2025-01-01 01:15:55,926:DEBUG:certbot._internal.main:Arguments: ['--cert-name', 'sierracloud.kro.kr', '-d', 'www.sierracloud.kro.kr', '-v']
2025-01-01 01:15:55,926:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2025-01-01 01:15:55,932:DEBUG:certbot._internal.log:Root logging level set at 20
2025-01-01 01:15:55,933:DEBUG:certbot._internal.plugins.selection:Requested authenticator None and installer None
2025-01-01 01:15:55,933:DEBUG:certbot._internal.plugins.selection:Multiple candidate plugins: * standalone
Description: Runs an HTTP server locally which serves the necessary validation files under the /.well-known/acme-challenge/ request path. Suitable if there is no HTTP server already running. HTTP challenge only (wildcards not supported).
Interfaces: Authenticator, Plugin
Entry point: EntryPoint(name='standalone', value='certbot._internal.plugins.standalone:Authenticator', group='certbot.plugins')
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0x7fbbdcb02fa0>
Prep: True
......
2025-01-01 01:16:06,980:DEBUG:certbot._internal.display.obj:Notifying user:
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/sierracloud.kro.kr/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/sierracloud.kro.kr/privkey.pem
This certificate expires on 2025-03-31.
These files will be updated when the certificate renews.
Certbot has set up a scheduled task to automatically renew this certificate in the background.
2025-01-01 01:16:06,982:DEBUG:certbot._internal.display.obj:Notifying user: If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
2025-01-01 01:16:36,715:DEBUG:certbot._internal.main:certbot version: 2.11.0
2025-01-01 01:16:36,715:DEBUG:certbot._internal.main:Location of certbot entry point: /bin/certbot
2025-01-01 01:16:36,715:DEBUG:certbot._internal.main:Arguments: []
2025-01-01 01:16:36,715:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2025-01-01 01:16:36,722:DEBUG:certbot._internal.log:Root logging level set at 30
2025-01-01 01:16:36,735:DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): e5.o.lencr.org:80
2025-01-01 01:16:36,925:DEBUG:urllib3.connectionpool:http://e5.o.lencr.org:80 "POST / HTTP/1.1" 200 345
2025-01-01 01:16:36,926:DEBUG:certbot.ocsp:OCSP response for certificate /etc/letsencrypt/live/sierracloud.kro.kr/cert.pem is signed by the certificate's issuer.
2025-01-01 01:16:36,927:DEBUG:certbot.ocsp:OCSP certificate status for /etc/letsencrypt/live/sierracloud.kro.kr/cert.pem is: OCSPCertStatus.GOOD
2025-01-01 01:16:36,929:DEBUG:certbot._internal.display.obj:Notifying user: Found the following certs:
  Certificate Name: sierracloud.kro.kr
    Serial Number: 31398fee84f098d5790169b382cc7eb2878
    Key Type: ECDSA
    Domains: www.sierracloud.kro.kr
    Expiry Date: 2025-03-31 15:17:34+00:00 (VALID: 89 days)
    Certificate Path: /etc/letsencrypt/live/sierracloud.kro.kr/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/sierracloud.kro.kr/privkey.pem

실패 없이 완료 되었습니다!

Rocky

Rocky Linux 9.5 NIC IP Setting

댓글 남기기

과거의 RHEL/CentOS 계열 OS에서는 ifcfg-xxx 파일의 Manual한 IP 설정 후 Network Service 또는 ethernet 을 재기동 하여 수동 IP를 반영하였지만 아래와 같이 이제 더이상 network-scripts 하단의 설정 구성 방석은 지원하지 않습니다.

[root@localhost ~]# ll /etc/sysconfig/network-scripts/
total 4
-rw-r--r--. 1 root root 1244 Nov  7 13:30 readme-ifcfg-rh.txt

변경된 방식에 따라 system-connections 하단의 nmconnection 파일을 수정하여 [ipv4] 부분에 수동 IP를 설정하고 nmcli로 반영 하겠습니다.

[root@localhost ~]# ll /etc/NetworkManager/system-connections/
total 4
-rw-------. 1 root root 227 Dec 30 17:19 ens33.nmconnection

[root@localhost ~]# vi /etc/NetworkManager/system-connections/ens33.nmconnection

[ipv4]
method=manual
address1=192.168.0.100/24,192.168.0.1
dns=8.8.8.8

[root@localhost ~]# nmcli connection reload
[root@localhost ~]# nmcli connection up ens33
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)

[root@localhost ~]$ nmcli device show ens33
GENERAL.DEVICE:                         ens33
GENERAL.TYPE:                           ethernet
......
GENERAL.CONNECTION:                     ens33
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/2
WIRED-PROPERTIES.CARRIER:               on
IP4.ADDRESS[1]:                         192.168.0.100/24
IP4.GATEWAY:                            192.168.0.1
IP4.ROUTE[1]:                           dst = 192.168.0.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]:                           dst = 0.0.0.0/0, nh = 192.168.0.1, mt = 100
IP4.DNS[1]:                             8.8.8.8

실제 통신이 가능한지 ping 등을 통해 검증하면 끝입니다. 🙂

K8S

kubectl : certificate has expired or is not yet valid

댓글 남기기

어느날 kubernetes 상태 확인을 위해 kubectl을 치자 발생하는 에러 메시지…

E1218 05:21:48.113070 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.26.101:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.115822 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.26.101:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.118161 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.26.101:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.121162 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.26.101:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
E1218 05:21:48.124428 1685746 memcache.go:265] couldn't get current server API group list: Get "https://192.168.26.101:6443/api?timeout=32s": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z
Unable to connect to the server: tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-12-18T05:21:48+09:00 is after 2024-12-05T15:09:04Z

x509 같은 키워드를 보아 인증서로 추측 되므로 빠른 구글링으로 조치 방법 확인…

kubeadm certs check-expiration 로 인증서 만료 여부를 확인해보겠습니다…

test@test-master-01:~$ sudo kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
apiserver                  Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
apiserver-etcd-client      Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
apiserver-kubelet-client   Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
controller-manager.conf    Dec 05, 2024 15:09 UTC   <invalid>       ca                      no
etcd-healthcheck-client    Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
etcd-peer                  Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
etcd-server                Dec 05, 2024 15:09 UTC   <invalid>       etcd-ca                 no
front-proxy-client         Dec 05, 2024 15:09 UTC   <invalid>       front-proxy-ca          no
scheduler.conf             Dec 05, 2024 15:09 UTC   <invalid>       ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 03, 2033 15:09 UTC   8y              no
etcd-ca                 Dec 03, 2033 15:09 UTC   8y              no
front-proxy-ca          Dec 03, 2033 15:09 UTC   8y              no

기존 인증서 백업 처리

sudo cp -pr /etc/kubernetes/ /etc/kubernetes_backup

인증서 갱신 및 확인

test@test-master-01:~$ sudo kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[renew] Error reading configuration from the Cluster. Falling back to default configuration

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.
test@test-master-01:~$ sudo kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Dec 17, 2025 20:29 UTC   364d            ca                      no
apiserver                  Dec 17, 2025 20:29 UTC   364d            ca                      no
apiserver-etcd-client      Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Dec 17, 2025 20:29 UTC   364d            ca                      no
controller-manager.conf    Dec 17, 2025 20:29 UTC   364d            ca                      no
etcd-healthcheck-client    Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
etcd-peer                  Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
etcd-server                Dec 17, 2025 20:29 UTC   364d            etcd-ca                 no
front-proxy-client         Dec 17, 2025 20:29 UTC   364d            front-proxy-ca          no
scheduler.conf             Dec 17, 2025 20:29 UTC   364d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Dec 03, 2033 15:09 UTC   8y              no
etcd-ca                 Dec 03, 2033 15:09 UTC   8y              no
front-proxy-ca          Dec 03, 2033 15:09 UTC   8y              no

kubectl을 실행 해도 kube-apiserver, kube-controller-manager, kube-scheduler and etcd 을 재시작 하기 전까지는 계속 오류 발생…

kubectl을 사용하는 계정의 홈디렉토리에도 config에 인증서가 포함되어 있어 해당 파일도 덮어 씌워 줍니다. (root로 실행할 경우 /roo/.kube/config)

test@test-master-01:~$ sudo cp /etc/kubernetes/admin.conf /home/test/.kube/config
test@test-master-01:~$ chown test:test /home/test/.kube/config
test@test-master-01:~$ ll /home/test/.kube/config
-rw------- 1 test test 5650 Dec 18 05:42 /home/test/.kube/config

프로세스 중지 및 서비스 재시작

test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-apiserver)
test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-controller-manager)
test@test-master-01:~$ sudo kill -s SIGHUP $(pidof kube-scheduler)
test@test-master-01:~$ sudo systemctl restart kubelet
test@test-master-01:~$ sudo systemctl daemon-reload

kubectl을 쳐보겠습니다…

test@test-master-01:~$ kubectl get po -A
NAMESPACE          NAME                                                      READY   STATUS      RESTARTS       AGE
ingress-nginx      ingress-nginx-controller-6dfcb8658d-8rhbq                 1/1     Running     1 (72d ago)    172d
kube-system        calico-kube-controllers-7ddc4f45bc-d8259                  1/1     Running     1 (72d ago)    147d
kube-system        calico-node-5mk6f                                         1/1     Running     11 (72d ago)   376d
kube-system        calico-node-b6jxh                                         1/1     Running     16 (72d ago)   376d
kube-system        calico-node-qqmt4                                         1/1     Running     14 (72d ago)   376d
kube-system        calico-node-xjhg4                                         1/1     Running     10 (72d ago)   192d
kube-system        coredns-5dd5756b68-t2pq8                                  1/1     Running     11 (72d ago)   377d
kube-system        coredns-5dd5756b68-tx2xj                                  1/1     Running     11 (72d ago)   377d
kube-system        etcd-k8s-master-01                                        1/1     Running     13 (72d ago)   377d
kube-system        kube-apiserver-k8s-master-01                              1/1     Running     19 (68s ago)   377d
kube-system        kube-controller-manager-k8s-master-01                     1/1     Running     15 (60s ago)   377d
kube-system        kube-proxy-22kqn                                          1/1     Running     7 (72d ago)    173d
kube-system        kube-proxy-b9mbg                                          1/1     Running     2 (72d ago)    173d
kube-system        kube-proxy-n4q6t                                          1/1     Running     7 (72d ago)    173d
kube-system        kube-proxy-x649t                                          1/1     Running     6 (72d ago)    173d
kube-system        kube-scheduler-k8s-master-01                              1/1     Running     15 (54s ago)   377d
kube-system        metrics-server-777dff589b-hmdhl                           1/1     Running     2 (72d ago)    169d
mariadb-system     mariadb-79d8f666bc-jnzlm                                  1/1     Running     1 (72d ago)    170d
metallb-system     controller-686877b9fc-9x9mh                               1/1     Running     2 (72d ago)    172d
metallb-system     speaker-7kn6b                                             1/1     Running     1 (72d ago)    172d
metallb-system     speaker-c7rrx                                             1/1     Running     1 (72d ago)    172d
metallb-system     speaker-hcvlw                                             1/1     Running     1 (72d ago)    172d

잘 됩니다! 완료!