NP-756: multinode ovn-kubernetes

[zshi-redhat]

This commit is the first step to introduce multinode capability in microshift, focusing on networking part.

• add a hidden flag --controplane
• split ovnk manifests to common, single-node and multi-node
• apply multi-node manifests for ovnk
• change cluster mtu based on multinode flag

To run microshift in multinode mode:
$ microshift run --controlplane

[dhellmann]

What populates the list of workers? Does the user need to know them in advance?

[zshi-redhat]

I should probably remove it in this commit. I was thinking to use it for generating kubelet kubeconfig for each extra worker node. It could be handled in a follow up PR if needed when adding multi worker nodes support in microshift main binary.

[zshi-redhat]

removed Workers and move the Multinode structure to a separate file pkg/config/multinode.go

[fzdarsky]

Can we call this “controlplane” instead of “master”, a language that Red Hat (and the K8s community) try to deprecate?

Also, how about we revert to the original —roles=controlplane,node approach, which is more flexible than a boolean flag?

[zshi-redhat]

Can we call this “controlplane” instead of “master”, a language that Red Hat (and the K8s community) try to deprecate?

I will use control plane.

Also, how about we revert to the original —roles=controlplane,node approach, which is more flexible than a boolean flag?

sounds good, will update to use --roles

[dhellmann]

Can we call this “controlplane” instead of “master”, a language that Red Hat (and the K8s community) try to deprecate?

I will use control plane.

Also, how about we revert to the original —roles=controlplane,node approach, which is more flexible than a boolean flag?

sounds good, will update to use --roles

We talked about using the standard kubelet binary for the worker. If we do that, do we need MicroShift to have separate roles?

[zshi-redhat]

Can we call this “controlplane” instead of “master”, a language that Red Hat (and the K8s community) try to deprecate?

I will use control plane.

Also, how about we revert to the original —roles=controlplane,node approach, which is more flexible than a boolean flag?

sounds good, will update to use --roles

We talked about using the standard kubelet binary for the worker. If we do that, do we need MicroShift to have separate roles?

Looking at the previous implementation, I think roles is to distinguish the control plane and node in microshift main binary so that:

1. node role only runs kubelet service
2. controlplane role runs service other than kubelet
3. controlplane + node role runs all microshift services

If we use standard kubelet binary for extra worker nodes, roles may not be needed since microshift just needs to support controlplane + node role. controlplane role is not what we wanted to support.

[zshi-redhat]

Still needs to fix the assets structure in scripts/auto-rebase/assets.yaml which causes ci/prow/verify job fail.
@pmtk Do we have script for generating scripts/auto-rebase/assets.yaml or is it manual?

[zshi-redhat]

Still needs to fix the assets structure in scripts/auto-rebase/assets.yaml which causes ci/prow/verify job fail. @pmtk Do we have script for generating scripts/auto-rebase/assets.yaml or is it manual?

updated manually.

[zshi-redhat]

@pacevedom This overrides the advertiseAddress IP of APIServer for multi-node since the service cdir cannot be accessed by extra worker nodes. Do you see any issue with API certificate by doing so?

[pacevedom]

Yes, we do need some changes for this to work. Let me create the PR.

[pacevedom]

I checked and everything is already there, so no additional changes required.

[zshi-redhat]

/jira refresh

[openshift-ci-robot]

@zshi-redhat: This pull request references NP-756 which is a valid jira issue.

Details

In response to this:

This commit is the first step to introduce multinode capability in microshift, focusing on networking part.

• add a hidden flag --controplane
• split ovnk manifests to common, single-node and multi-node
• apply multi-node manifests for ovnk
• change cluster mtu based on multinode flag

To run microshift in multinode mode:
$ microshift run --controlplane

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@zshi-redhat: This pull request references NP-756 which is a valid jira issue.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dhellmann]

A couple of small suggestions, but there's nothing to hold this up.

I'll leave this for @pacevedom to approve.

/assign @pacevedom

[dhellmann]

If we're not doing "modes" this can be "multinode" instead of "controlplane".

[zshi-redhat]

changed to "multinode"

[pacevedom]

/lgtm

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD a45ce75 and 2 for PR HEAD 060a5a07f923355fcefd2d8bca6e27a2f0e63106 in total

[zshi-redhat]

seeing permission denied failures in the CI job:

oc logs -n openshift-ingress pod/router-default-d55c4fb97-9sfnc router
exec container process `/usr/bin/openshift-router`: Permission denied

rebased against master.

[ggiguash]

seeing permission denied failures in the CI job:

oc logs -n openshift-ingress pod/router-default-d55c4fb97-9sfnc router
exec container process `/usr/bin/openshift-router`: Permission denied

rebased against master.

That's a problem in our CI configuration, which manifests itself in all the PRs.

[ggiguash]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ggiguash, pacevedom, zshi-redhat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [ggiguash,pacevedom,zshi-redhat]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@zshi-redhat: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.