use retryable http client

[janisz]

As we may get rate limited it will be good to retry our API requests.
Looks like default settings are OK for us:
https://github.com/hashicorp/go-retryablehttp/blob/9dfd949154e507c91ee569367ffa461991b80950/client.go#L49-L51
and backoff policy https://pkg.go.dev/github.com/hashicorp/go-retryablehttp#DefaultBackoff

Tested:

go run ./cmd/junit2jira/... -junit-reports-dir ./cmd/junit2jira/testdata/jira -html-output index.html -debug
INFO[Mar  4 11:19:01] Found 7 failed tests                         
DEBU[Mar  4 11:19:01] Searching for issue                           ID="?" summary="DefaultPoliciesTest / Verify policy Apache Struts  CVE-2017-5638 is triggered FAILED"
DEBU[Mar  4 11:19:01] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22DefaultPoliciesTest+%2F+Verify+policy+Apache+Struts++CVE-2017-5638+is+triggered+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:01] Issue not found. Creating new issue...        ID="?" summary="DefaultPoliciesTest / Verify policy Apache Struts  CVE-2017-5638 is triggered FAILED"
DEBU[Mar  4 11:19:01] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
ERRO[Mar  4 11:19:03] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:03] Searching for issue                           ID="?" summary="github.com/stackrox/rox/pkg/grpc / Test_APIServerSuite/Test_TwoTestsStartingAPIs FAILED"
DEBU[Mar  4 11:19:03] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22github.com%2Fstackrox%2Frox%2Fpkg%2Fgrpc+%2F+Test_APIServerSuite%2FTest_TwoTestsStartingAPIs+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:03] Issue not found. Creating new issue...        ID="?" summary="github.com/stackrox/rox/pkg/grpc / Test_APIServerSuite/Test_TwoTestsStartingAPIs FAILED"
DEBU[Mar  4 11:19:03] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
ERRO[Mar  4 11:19:04] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:04] Searching for issue                           ID="?" summary="central-basic / step 90-activate-scanner-v4 FAILED"
DEBU[Mar  4 11:19:04] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22central-basic+%2F+step+90-activate-scanner-v4+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:04] Issue not found. Creating new issue...        ID="?" summary="central-basic / step 90-activate-scanner-v4 FAILED"
DEBU[Mar  4 11:19:04] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
DEBU[Mar  4 11:19:04] retrying request                              remaining=4 request="POST https://issues.redhat.com/rest/api/2/issue (status: 429)" timeout=1s
ERRO[Mar  4 11:19:05] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:05] Searching for issue                           ID="?" summary="github.com/stackrox/rox/pkg/booleanpolicy/evaluator / TestDifferentBaseTypes FAILED"
DEBU[Mar  4 11:19:05] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22github.com%2Fstackrox%2Frox%2Fpkg%2Fbooleanpolicy%2Fevaluator+%2F+TestDifferentBaseTypes+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:05] Issue not found. Creating new issue...        ID="?" summary="github.com/stackrox/rox/pkg/booleanpolicy/evaluator / TestDifferentBaseTypes FAILED"
DEBU[Mar  4 11:19:05] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
DEBU[Mar  4 11:19:05] retrying request                              remaining=4 request="POST https://issues.redhat.com/rest/api/2/issue (status: 429)" timeout=1s
ERRO[Mar  4 11:19:07] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:07] Searching for issue                           ID="?" summary="github.com/stackrox/rox/sensor/kubernetes/localscanner / TestLocalScannerTLSIssuerIntegrationTests FAILED"
DEBU[Mar  4 11:19:07] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22github.com%2Fstackrox%2Frox%2Fsensor%2Fkubernetes%2Flocalscanner+%2F+TestLocalScannerTLSIssuerIntegrationTests+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:07] Issue not found. Creating new issue...        ID="?" summary="github.com/stackrox/rox/sensor/kubernetes/localscanner / TestLocalScannerTLSIssuerIntegrationTests FAILED"
DEBU[Mar  4 11:19:07] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
ERRO[Mar  4 11:19:07] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:07] Searching for issue                           ID="?" summary="github.com/stackrox/rox/central/resourcecollection/datastore/store/postgres / TestCollectionsStore FAILED"
DEBU[Mar  4 11:19:07] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22github.com%2Fstackrox%2Frox%2Fcentral%2Fresourcecollection%2Fdatastore%2Fstore%2Fpostgres+%2F+TestCollectionsStore+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:07] Issue not found. Creating new issue...        ID="?" summary="github.com/stackrox/rox/central/resourcecollection/datastore/store/postgres / TestCollectionsStore FAILED"
DEBU[Mar  4 11:19:07] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
ERRO[Mar  4 11:19:08] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
DEBU[Mar  4 11:19:08] Searching for issue                           ID="?" summary="command-line-arguments / TestTimeout FAILED"
DEBU[Mar  4 11:19:08] performing request                            method=GET url="https://issues.redhat.com/rest/api/2/search?jql=project+in+%28ROX%29%0AAND+issuetype+%3D+Bug%0AAND+status+%21%3D+Closed%0AAND+labels+%3D+CI_Failure%0AAND+summary+~+%22command-line-arguments+%2F+TestTimeout+FAILED%22%0AORDER+BY+created+DESC"
INFO[Mar  4 11:19:08] Issue not found. Creating new issue...        ID="?" summary="command-line-arguments / TestTimeout FAILED"
DEBU[Mar  4 11:19:08] performing request                            method=POST url="https://issues.redhat.com/rest/api/2/issue"
ERRO[Mar  4 11:19:08] {"errorMessages":[],"errors":{"summary":"Field 'summary' cannot be set. It is not on the appropriate screen, or unknown.","description":"Field 'description' cannot be set. It is not on the appropriate screen, or unknown.","labels":"Field 'labels' cannot be set. It is not on the appropriate screen, or unknown."}}  StatusCode=400 error="request failed. Please analyze the request body for more details. Status code: 400"
FATA[Mar  4 11:19:08] could not create issues or comments: 7 errors occurred:
	* could not create issue DefaultPoliciesTest / Verify policy Apache Struts  CVE-2017-5638 is triggered FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue github.com/stackrox/rox/pkg/grpc / Test_APIServerSuite/Test_TwoTestsStartingAPIs FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue central-basic / step 90-activate-scanner-v4 FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue github.com/stackrox/rox/pkg/booleanpolicy/evaluator / TestDifferentBaseTypes FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue github.com/stackrox/rox/sensor/kubernetes/localscanner / TestLocalScannerTLSIssuerIntegrationTests FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue github.com/stackrox/rox/central/resourcecollection/datastore/store/postgres / TestCollectionsStore FAILED: request failed. Please analyze the request body for more details. Status code: 400
	* could not create issue command-line-arguments / TestTimeout FAILED: request failed. Please analyze the request body for more details. Status code: 400
 
exit status 1

[porridge]

Please also add timestamps to the logger. 🙏🏻