using forked version of trilead-ssh2 (from org.jenkins-ci)

[miklosbarabas]

• upgrade to support newer algorithms

Description

CloudStack is using the trilead-ssh2 library for ssh communications, but doesn't have support for the latest MAC algorithms, which can be a blocker of adopting Cloudstack for certain companies with more strict security requirements.

The issue with the current version of trilead-ssh2 is that it is only supporting older MAC algorithms, so if one would like to add a hypervisor host to a cluster and the host runs a sshd with old algos excluded, it will be unable to add the host.

There's a forked version of trilead-ssh2 which is still being maintained.
Upgrade to this fork/version of trilead-ssh2 would provide the possibility to use newer algorithms for SSH without the need to touch actual business logic.

Resolves #3255

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

[miklosbarabas]

@DaanHoogland
Actually the change is done, just wanted to add a test to cover this (that's why the PR was WIP), but didn't find an easy way of doing it. Any thoughts?

[DaanHoogland]

@miklosbarabas, I suppose you could call protected static boolean canEndTheSshConnection(int waitResultTimeoutInMs, com.trilead.ssh2.Session sess, int conditions) throws SshException, InterruptedException in a test that should throw an InterruptedException. I am not entirely sure why you need this change but I'm sure either a unit test or an integration test must be possible for it.
converting it back to draft for now.

[yadvr]

@blueorangutan package

[blueorangutan]

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔centos7 ✔debian. JID-1245

[yadvr]

@blueorangutan test

[blueorangutan]

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-1570)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 46677 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4099-t1570-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 83 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[DaanHoogland]

I see no problem with this code and tests passed, but

• this is a fork that has abandoned its original governance
• we may need to take ownership and maintain our selves if the new governance/maintenance is abandoned
• not a real worry atm, but how do we guarantee compatibility?

none of this should should stop this merge as for now it is easily reversible. Just points of attention.

[yadvr]

One issue it seems Jenkins forked the build-217 than latest build-222

[DaanHoogland]

@rhtyd do you consider that a blocker? (or..?)

[shwstppr]

@blueorangutan package

[weizhouapache]

code lgtm

[yadvr]

@blueorangutan package

[blueorangutan]

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[yadvr]

@miklosbarabas I've updated the jar dependency to latest version, can you check and test if it works for your env?

[yadvr]

Let's merge if it passes a round of tests

[yadvr]

@blueorangutan package

[blueorangutan]

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 334

[yadvr]

@blueorangutan test matrix

[blueorangutan]

@rhtyd a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-361)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 36771 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4099-t361-vmware-65u2.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 83 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_10_traceroute_in_vr	Failure	61.26	test_diagnostics.py
test_01_deploy_kubernetes_cluster	Failure	107.15	test_kubernetes_clusters.py
test_02_invalid_upgrade_kubernetes_cluster	Failure	61.93	test_kubernetes_clusters.py
test_03_deploy_and_upgrade_kubernetes_cluster	Failure	61.51	test_kubernetes_clusters.py
test_04_deploy_and_scale_kubernetes_cluster	Failure	98.35	test_kubernetes_clusters.py
test_05_delete_kubernetes_cluster	Failure	64.91	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	Failure	87.08	test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster	Failure	87.75	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	Failure	80.03	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	Error	123.87	test_kubernetes_clusters.py
ContextSuite context=TestVAppsVM>:setup	Error	43.85	test_vm_life_cycle.py

[blueorangutan]

Trillian test result (tid-360)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37268 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4099-t360-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestKubernetesCluster>:teardown	Error	76.54	test_kubernetes_clusters.py
test_01_migrate_VM_and_root_volume	Error	70.44	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	Error	48.30	test_vm_life_cycle.py

[blueorangutan]

Trillian test result (tid-359)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41883 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4099-t359-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_scale_vm.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 83 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestKubernetesCluster>:teardown	Error	85.20	test_kubernetes_clusters.py
test_01_scale_vm	Failure	11.27	test_scale_vm.py
test_01_cancel_host_maintenace_with_no_migration_jobs	Failure	305.89	test_host_maintenance.py
test_02_cancel_host_maintenace_with_migration_jobs	Failure	305.97	test_host_maintenance.py
test_03_cancel_host_maintenace_with_migration_jobs_failure	Failure	0.20	test_host_maintenance.py

[yadvr]

Comparing against baseline 4.15 health check PR, don't see any new failures. LGTM.

[RodrigoDLopez]

hey folks, after this merge i'm facing troubles to build my own branches.
somehow the maven are saying that the repo doesn't exist.
To fix this I need to change the pom.xml from utils project.
Am I doing something wrong?
this is the error i receive:

[ERROR] Failed to execute goal on project cloud-utils: Could not resolve dependencies for project org.apache.cloudstack:cloud-utils:jar:4.15.1.0-SNAPSHOT: Failed to collect dependencies at org.jenkins-ci:trilead-ssh2:jar:build-217-jenkins-27: Failed to read artifact descriptor for org.jenkins-ci:trilead-ssh2:jar:build-217-jenkins-27: Could not transfer artifact org.jenkins-ci:trilead-ssh2:pom:build-217-jenkins-27 from/to repo.jenkins-ci.org.releases (http://repo.jenkins-ci.org/releases/): Transfer failed for http://repo.jenkins-ci.org/releases/org/jenkins-ci/trilead-ssh2/build-217-jenkins-27/trilead-ssh2-build-217-jenkins-27.pom 308 Permanent Redirect -> [Help 1]

[Dajeong-Park]

Same as above, I checked the error when building in centOS 8.4 environment.

[ERROR] Failed to execute goal on project cloud-utils: Could not resolve dependencies for project org.apache.cloudstack:cloud-utils:jar:4.16.0.0-SNAPSHOT: Failed to collect dependencies at org.jenkins-ci:trilead-ssh2:jar:build-217-jenkins-27: Failed to read artifact descriptor for org.jenkins-ci:trilead-ssh2:jar:build-217-jenkins-27: Could not transfer artifact org.jenkins-ci:trilead-ssh2:pom:build-217-jenkins-27 from/to repo.jenkins-ci.org.releases (http://repo.jenkins-ci.org/releases/): Transfer failed for http://repo.jenkins-ci.org/releases/org/jenkins-ci/trilead-ssh2/build-217-jenkins-27/trilead-ssh2-build-217-jenkins-27.pom 308 Permanent Redirect -> [Help 1]

[yadvr]

@RodrigoDLopez @Dajeong-Park are you using 4.15 or master branch? Can you try updating the jenkins repo url to use https, as in f1c83a0

[Dajeong-Park]

It is being used as a forked source by reflecting the merged sources in real time, and the build was successful by changing to https. Thank you.

[yadvr]

Alright @Dajeong-Park I'll cherry-pick the fix from master and get it on 4.15.

[RodrigoDLopez]

@RodrigoDLopez @Dajeong-Park are you using 4.15 or master branch? Can you try updating the jenkins repo url to use https, as in f1c83a0

I tried one of my own branches, and then i tried with 4.15. same error into both branches. I'm using the last maven version

It is being used as a forked source by reflecting the merged sources in real time, and the build was successful by changing to https. Thank you.

@rhtyd as @Dajeong-Park said.
we need to change the pom.xml in order to successfully build the project.

Can I do a small PR to fix this in 4.15?

[yadvr]

@RodrigoDLopez the issue was recently fixed in 4.15,I've backported the commit to 4.15. Pl try using latest 4.15, if still hit the issue raise a PR thnx.