USHIFT-716: add configurable KAS advertise address#1298
USHIFT-716: add configurable KAS advertise address#1298openshift-merge-robot merged 5 commits intoopenshift:mainfrom
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@pacevedom: This pull request references USHIFT-716 which is a valid jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
openshift-ci
bot
added
the
approved
docs/howto_config.md
Outdated
There was a problem hiding this comment.
This is exposed so as to enable CNCF certification tests where we need more than one node. Users shouldnt use it, as it defaults to the api server service IP internally.
There was a problem hiding this comment.
If we put this in the documentation, someone is going to use it. Do we need to include it in this document?
There was a problem hiding this comment.
I included it here just in case. Are we allowed to include undocumented parameters/options in CNCF certification instructions?
I was thinking about making it public here but not in customer docs. Adding a "dev only" tag/disclaimer should also help.
There was a problem hiding this comment.
That's a good question. @jerpeter1 has been looking at the certification rules, maybe he came across something that would help answer it?
|
/hold until enhancement is merged. |
openshift-ci
bot
added
the
do-not-merge/hold
Adds an internal IP to kubernetes service. Pods will use an internal IP to connect to KAS, while external clients will use the node IP. Solves ambiguity problem when selecting which certificate to return to a client, based on the destination IP (KAS). Configures the service IP to the loopback interface so that the endpoint slice for kubernetes service matches the same IP.
|
/retest |
docs/howto_config.md
Outdated
There was a problem hiding this comment.
If we put this in the documentation, someone is going to use it. Do we need to include it in this document?
|
Is there a way to add a test case to verify that this solves the problem and so we do not regress with future changes? |
pacevedom
force-pushed
the
USHIFT-716
branch
3 times, most recently
from
e86b2ba to
68c6554
Compare
|
/retest |
openshift-ci
bot
removed
the
do-not-merge/hold
|
/retest |
To follow service CIDR configuration options, and to allow forcing a different IP, a new configuration parameter is introduced. This will come in handy when setting extra nodes. Also excludes the node IP in the external serving certificate if the KAS advertise address matches the node IP.
Defaults KAS advertise address to always follow the service CIDR from configuration. If this CIDR is changed before starting MicroShift, the KAS advertise address (and therefore the secondary IP to lo interface) will be in that range.
Avoid adding the secondary IP to lo interface when KAS advertise address has been forced to an IP in a different range than service network CIDR.
Use a new service to configure loopback interface in the node if required by the KAS advertise address. Kube apiserver service will depend on this one before starting.
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dhellmann, pacevedom The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/cherry-pick release-4.12 |
|
@dhellmann: once the present PR merges, I will cherry-pick it on top of release-4.12 in a new PR and assign it to you. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
1 similar comment
|
/retest |
|
/retest-required |
|
/override ci/prow/e2e-openshift-conformance-sig-scheduling ci/prow/e2e-openshift-conformance-sig-instrumentation ci/prow/e2e-openshift-conformance-sig-api-machinery |
|
@pacevedom: Overrode contexts on behalf of pacevedom: ci/prow/e2e-openshift-conformance-sig-api-machinery, ci/prow/e2e-openshift-conformance-sig-instrumentation, ci/prow/e2e-openshift-conformance-sig-scheduling DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/hold |
openshift-ci
bot
added
the
do-not-merge/hold
|
/unhold |
openshift-ci
bot
removed
the
do-not-merge/hold
|
@pacevedom: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
@dhellmann: new pull request created: #1360 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Due to the support for IPs in certificates introduced in openshift#1298, the apiserver IP is configured as a secondary address in the lo interface. A VIP is configured by ovnk redirecting 10.43.0.1:443 to 10.43.0.1:6443. 6443 is the port where apiserver listens in the host. 10.43.0.1:443 is used by all pods using client go, as it is computed from the env vars we can find in any pod. If a host network pod or any other tool in the host tries to reach the apiserver by using 10.43.0.1:443 the address is not translated to the endpoint, it tries to contact 10.43.0.1:443 which is not the apiserver but the router. This change computes a new IP endpoint in the next available /32 subnet from the service IP to ensure ovnk does not interfere.
Due to the support for IPs in certificates introduced in openshift#1298, the apiserver IP is configured as a secondary address in the lo interface. A VIP is configured by ovnk redirecting 10.43.0.1:443 to 10.43.0.1:6443. 6443 is the port where apiserver listens in the host. 10.43.0.1:443 is used by all pods using client go, as it is computed from the env vars we can find in any pod. If a host network pod or any other tool in the host tries to reach the apiserver by using 10.43.0.1:443 the address is not translated to the endpoint, it tries to contact 10.43.0.1:443 which is not the apiserver but the router. This change computes a new IP endpoint in the next available /32 subnet from the service IP to ensure ovnk does not interfere.
7 participants
Which issue(s) this PR addresses:
Closes #