Zero To One

ingress-nginx-controller 설치 에러 본문

서비스모니터링

ingress-nginx-controller 설치 에러

Zero_To_One 2022. 5. 9. 19:55

Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: IngressClass "nginx" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "ingress-nginx"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "default"

 

문제발생

Prometheus, Grafana를 통해 모니터링을 해보려는데 nginx-controller가 설치가 안된다.

 

해결시도는 다음과 같다

1. 쉘스크립트로 nginx-controller 삭제

#nano delete.sh

helm repo remove ingress-nginx
kubectl delete namespace ingress-nginx
kubectl delete serviceaccount ingress-nginx
kubectl delete configmap ingress-nginx-controller
kubectl delete clusterrole ingress-nginx
kubectl delete clusterrolebinding ingress-nginx
kubectl delete role ingress-nginx
kubectl delete rolebinding ingress-nginx
kubectl delete service ingress-nginx-controller-admission
kebectl delete service ingress-nginx-controller
kubectl delete deployment ingress-nginx-controller
kubectl delete validatingwebhookconfiguration ingress-nginx-admission
kubectl delete serviceaccount ingress-nginx-admission
kubectl delete clusterrole ingress-nginx-admission
kubectl delete clusterrolebinding ingress-nginx-admission
kubectl delete role ingress-nginx-admission
kubectl delete rolebinding ingress-nginx-admission
kubectl delete job ingress-nginx-admission-create
kubectl delete job ingress-nginx-admission-patch
kubectl delete clusterrolebinding nginx-ingress

참조 : https://jmholly.tistory.com/entry/nginx-ingress-controller-%EB%A7%8C%EB%93%A4%EA%B8%B0%EB%A5%BC-%EC%8B%A4%ED%8C%A8%ED%96%88%EC%9D%84%EB%95%8C-%EA%B9%A8%EB%81%97%ED%9E%88-%EC%82%AD%EC%A0%9C%EB%B0%A9%EB%B2%95

 

nginx ingress controller 만들기를 실패했을때 깨끗히 삭제방법

vim delete.sh kubectl delete namespace ingress-nginx kubectl delete serviceaccount ingress-nginx kubectl delete configmap ingress-nginx-controller kubectl delete clusterrole ingress-nginx kubectl de..

jmholly.tistory.com

처음으로 되돌리기 위해 싹다 삭제하는 명령어를 쉘 스크립트로 블로깅해준 분이 계셨다

 

추가로 스택오버플로우에 있는 삭제 명령어를 포함해서 적었다.

 

2. namespace를 확인후,  kubectl이 "minikube" 클러스터와 "default" 네임스페이스를 기본적으로 사용하도록 구성

> kubectl get ns
> minikube delete

 

3. 다시 재설치

git clone https://github.com/prometheus-operator/kube-prometheus.git
cd kube-prometheus

kubectl create -f manifests/setup
until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done
kubectl create -f manifests/

 

4. 프로메테우스 노출 명령어

kubectl --namespace monitoring port-forward svc/prometheus-k8s 9090

그라파나 노출 명령어

kubectl --namespace monitoring port-forward svc/grafana 3000

 

5. nginx 인그레스 컨트롤러 설치

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
 --set controller.metrics.enabled=true \
 --set controller.metrics.serviceMonitor.enabled=true \
 --set controller.metrics.serviceMonitor.additionalLabels.release="prometheus"

결과

NAME: ingress-nginx
LAST DEPLOYED: Mon May  9 20:53:20 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w ingress-nginx-controller'

An example Ingress that makes use of the controller:
  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: example
    namespace: foo
  spec:
    ingressClassName: nginx
    rules:
      - host: www.example.com
        http:
          paths:
            - pathType: Prefix
              backend:
                service:
                  name: exampleService
                  port:
                    number: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
      - hosts:
        - www.example.com
        secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

설치 완료 후 네임스페이스 확인

 

> kubectl get all -A
NAMESPACE     NAME                                            READY   STATUS              RESTARTS       AGE
default       pod/ingress-nginx-controller-6cf5c8f9b5-tdc75   0/1     ContainerCreating   0              50s
kube-system   pod/coredns-64897985d-cwq4h                     1/1     Running             0              2m16s
kube-system   pod/etcd-minikube                               1/1     Running             0              2m30s
kube-system   pod/kube-apiserver-minikube                     1/1     Running             0              2m30s
kube-system   pod/kube-controller-manager-minikube            1/1     Running             0              2m30s
kube-system   pod/kube-proxy-prc5m                            1/1     Running             0              2m16s
kube-system   pod/kube-scheduler-minikube                     1/1     Running             0              2m31s
kube-system   pod/storage-provisioner                         1/1     Running             1 (106s ago)   2m28s
monitoring    pod/alertmanager-main-0                         0/2     ContainerCreating   0              64s
monitoring    pod/alertmanager-main-1                         0/2     ContainerCreating   0              64s
monitoring    pod/alertmanager-main-2                         0/2     ContainerCreating   0              64s
monitoring    pod/blackbox-exporter-6dfcb55b88-bs9c9          3/3     Running             0              2m5s
monitoring    pod/grafana-6f55cbfbfd-zpwvc                    1/1     Running             0              2m5s
monitoring    pod/kube-state-metrics-f4c97fc87-p2kdp          3/3     Running             0              2m4s
monitoring    pod/node-exporter-p2flx                         2/2     Running             0              2m4s
monitoring    pod/prometheus-adapter-6455646bdc-hrlx7         1/1     Running             0              2m4s
monitoring    pod/prometheus-adapter-6455646bdc-pc6fr         1/1     Running             0              2m4s
monitoring    pod/prometheus-k8s-0                            0/2     PodInitializing     0              64s
monitoring    pod/prometheus-k8s-1                            0/2     PodInitializing     0              64s
monitoring    pod/prometheus-operator-7c475b4689-brxdp        2/2     Running             0              2m4s

NAMESPACE     NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE
default       service/ingress-nginx-controller             LoadBalancer   10.107.215.5     <pending>     80:31733/TCP,443:30756/TCP     50s
default       service/ingress-nginx-controller-admission   ClusterIP      10.111.121.152   <none>        443/TCP                        50s
default       service/ingress-nginx-controller-metrics     ClusterIP      10.104.205.131   <none>        10254/TCP                      50s
default       service/kubernetes                           ClusterIP      10.96.0.1        <none>        443/TCP                        2m31s
kube-system   service/kube-dns                             ClusterIP      10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP         2m30s
kube-system   service/kubelet                              ClusterIP      None             <none>        10250/TCP,10255/TCP,4194/TCP   64s
monitoring    service/alertmanager-main                    ClusterIP      10.108.105.14    <none>        9093/TCP,8080/TCP              2m5s
monitoring    service/alertmanager-operated                ClusterIP      None             <none>        9093/TCP,9094/TCP,9094/UDP     64s
monitoring    service/blackbox-exporter                    ClusterIP      10.104.120.175   <none>        9115/TCP,19115/TCP             2m5s
monitoring    service/grafana                              ClusterIP      10.99.204.70     <none>        3000/TCP                       2m5s
monitoring    service/kube-state-metrics                   ClusterIP      None             <none>        8443/TCP,9443/TCP              2m5s
monitoring    service/node-exporter                        ClusterIP      None             <none>        9100/TCP                       2m4s
monitoring    service/prometheus-adapter                   ClusterIP      10.106.235.127   <none>        443/TCP                        2m4s
monitoring    service/prometheus-k8s                       ClusterIP      10.109.167.246   <none>        9090/TCP,8080/TCP              2m4s
monitoring    service/prometheus-operated                  ClusterIP      None             <none>        9090/TCP                       64s
monitoring    service/prometheus-operator                  ClusterIP      None             <none>        8443/TCP                       2m4s

NAMESPACE     NAME                           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/kube-proxy      1         1         1       1            1           kubernetes.io/os=linux   2m30s
monitoring    daemonset.apps/node-exporter   1         1         1       1            1           kubernetes.io/os=linux   2m4s

NAMESPACE     NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
default       deployment.apps/ingress-nginx-controller   0/1     1            0           50s
kube-system   deployment.apps/coredns                    1/1     1            1           2m30s
monitoring    deployment.apps/blackbox-exporter          1/1     1            1           2m5s
monitoring    deployment.apps/grafana                    1/1     1            1           2m5s
monitoring    deployment.apps/kube-state-metrics         1/1     1            1           2m5s
monitoring    deployment.apps/prometheus-adapter         2/2     2            2           2m4s
monitoring    deployment.apps/prometheus-operator        1/1     1            1           2m4s

NAMESPACE     NAME                                                  DESIRED   CURRENT   READY   AGE
default       replicaset.apps/ingress-nginx-controller-6cf5c8f9b5   1         1         0       50s
kube-system   replicaset.apps/coredns-64897985d                     1         1         1       2m16s
monitoring    replicaset.apps/blackbox-exporter-6dfcb55b88          1         1         1       2m5s
monitoring    replicaset.apps/grafana-6f55cbfbfd                    1         1         1       2m5s
monitoring    replicaset.apps/kube-state-metrics-f4c97fc87          1         1         1       2m5s
monitoring    replicaset.apps/prometheus-adapter-6455646bdc         2         2         2       2m4s
monitoring    replicaset.apps/prometheus-operator-7c475b4689        1         1         1       2m4s

NAMESPACE    NAME                                 READY   AGE
monitoring   statefulset.apps/alertmanager-main   0/3     64s
monitoring   statefulset.apps/prometheus-k8s      0/2     64s

 

'서비스모니터링' 카테고리의 다른 글

성능테스트 정리할것  (0) 2022.05.12
SLO 계산해보기  (0) 2022.05.11
The connection to the server localhost:8080 was refused - did you specify the right host or port? 오류 해결  (0) 2022.05.10
“네가지 황금 시그널(The Four Golden Signals)”로 SRE 모니터링의 주요 측정 항목 4가지  (0) 2022.05.09
Auto Scaling + CloudWatch를 이용한 알림(webhook & terraform)  (0) 2022.05.09
'서비스모니터링' Related Articles more