Revert "Reload existing CA from disk on restart"

[cooktheryan]

Reverts #499

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from cooktheryan after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cooktheryan]

Going to force merge this if passing. We seem to be hitting a leader election issue with service-ca restarting. If this goes back green I wonder if we implement post service-ca IP fix.

[cooktheryan]

I1222 00:34:34.069944       1 base_controller.go:72] Caches are synced for ServiceServingCertController 
I1222 00:34:34.070123       1 base_controller.go:109] Starting #1 worker of ServiceServingCertController controller ...
I1222 00:34:34.070208       1 base_controller.go:109] Starting #2 worker of ServiceServingCertController controller ...
I1222 00:34:34.070286       1 base_controller.go:109] Starting #3 worker of ServiceServingCertController controller ...
I1222 00:34:34.070394       1 base_controller.go:109] Starting #4 worker of ServiceServingCertController controller ...
I1222 00:34:34.070473       1 base_controller.go:109] Starting #5 worker of ServiceServingCertController controller ...
I1222 00:34:34.076115       1 configmap.go:107] updating configmap default/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.076763       1 configmap.go:107] updating configmap openshift-service-ca/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.077155       1 configmap.go:107] updating configmap kube-system/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.079425       1 configmap.go:107] updating configmap openshift-infra/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.079523       1 configmap.go:107] updating configmap openshift/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.083534       1 configmap.go:107] updating configmap openshift-dns/openshift-service-ca.crt with the service signing CA bundle
I1222 00:34:34.086384       1 configmap.go:107] updating configmap openshift-controller-manager/openshift-service-ca.crt with the service signing CA bundle
E1222 00:34:34.516958       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:34.901451       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:35.046371       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:35.579384       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:36.180052       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:36.795663       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:37.374299       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:37.966805       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:38.577065       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9
E1222 00:34:39.165296       1 base_controller.go:264] "ServiceServingCertController" controller failed to sync "openshift-ingress/router-external-default", err: secret openshift-ingress/router-certs-default does not have corresponding service UID 5e1dc19f-64c7-40b1-b352-c2216eee0db9

E1222 00:35:09.433605       1 leaderelection.go:325] error retrieving resource lock openshift-service-ca/service-ca-controller-lock: Get "https://10.43.0.1:443/api/v1/namespaces/openshift-service-ca/configmaps/service-ca-controller-lock?timeout=35s": read tcp 10.85.0.3:54568->10.43.0.1:443: read: connection timed out

I1222 00:35:18.977159       1 leaderelection.go:278] failed to renew lease openshift-service-ca/service-ca-controller-lock: timed out waiting for the condition
E1222 00:35:18.977396       1 leaderelection.go:301] Failed to release lock: resource name may not be empty
W1222 00:35:18.977472       1 leaderelection.go:75] leader election lost
[vagrant@fedora vagrant]$ 
[vagrant@fedora vagrant]$ 
[vagrant@fedora vagrant]$ kubectl logs -f -n openshift-service-ca            service-ca-7bffb6f6bf-g5gzk
W1222 00:36:07.305048       1 cmd.go:204] Using insecure, self-signed certificates
I1222 00:36:07.622886       1 observer_polling.go:159] Starting file observer
W1222 00:36:10.680980       1 builder.go:209] unable to get owner reference (falling back to namespace): Get "https://10.43.0.1:443/api/v1/namespaces/openshift-service-ca/pods": dial tcp 10.43.0.1:443: connect: no route to host
I1222 00:36:10.681888       1 builder.go:240] service-ca-controller version v3.11.0-357-gb66c450-

[cooktheryan]

Letting tests run through. Locally things look better

[vagrant@fedora validate-microshift]$ kubectl get po -wA
NAMESPACE                       NAME                                  READY   STATUS    RESTARTS   AGE
kube-system                     kube-flannel-ds-4w8d5                 1/1     Running   0          2m1s
kubevirt-hostpath-provisioner   kubevirt-hostpath-provisioner-7lcxd   1/1     Running   0          111s
openshift-dns                   dns-default-k77sk                     2/2     Running   0          2m1s
openshift-dns                   node-resolver-2x7b2                   1/1     Running   0          2m1s
openshift-ingress               router-default-6c96f6bc66-89szp       1/1     Running   0          2m6s
openshift-service-ca            service-ca-7bffb6f6bf-nktfb           1/1     Running   0          2m6s

[mangelajo]

This is unrelated to service CA I believe

[mangelajo]

This is just for the main microshift CA when it boots, let's get it back in.