Consider switching to GitHub Actions for automated testing

[lukpueh]

Description of issue or feature request:
We switched from automated testing on Travis CI (Linux) and Appveyor (Windows) to GitHub Actions in our sister-project in-toto-golang (see in-toto/in-toto-golang#72), mostly due to availability concerns about Appveyor and because it seems appealing to configure only one service for all platforms.

Let's consider switching here too and testing on more platforms than we currently do.

Current behavior:
Use Travis CI and Appveyor for automated testing on Linux for Windows (no tests on macOS)

Expected behavior:
Use GitHub Actions instead and also test on macOS.

[lukpueh]

Related: in-toto/in-toto#407 and secure-systems-lab/securesystemslib#297

[joshuagl]

Being able to test on Windows/Linux/macOS with the same CI make this a no-brainer to me.

[lukpueh]

Also consider injection vulnerabilities reported in https://crbug.com/project-zero/2070

[lukpueh]

See @jku's #1204 (comment) as backing data point for the "configure only one service for all platforms" argument.

[jku]

I think the next step is someone testing this to see if it's good enough.

Documenting a potential test matrix:
python: Latest release of all supported cpython 3.x versions, cpython 2.7. Currently that would be [2.7, 3.6, 3.7, 3.8, 3.9]
platform: Latest release of Linux, Windows, Mac: [ubuntu-latest, windows-latest, macos-latest]

Github does offer specific platform versions as well but I'm not sure it makes sense to use those? The matrix size as proposed is 15 test runs. As a bonus it might be nice to add pypy3 to python version list (not sure how much additional benefits it gives -- but it is something pip tests with and currently TUF tests pass with pypy3): that should probably be done separately as the build dependencies might require a bit of tweaking there.

There's also at least two special cases that we want to test (they do not need to be part of the matrix):
special: Securesystemslib master branch on "ubuntu-latest" with python "3.x" (this should not fail a build)
special: Lint on "ubuntu-latest" with python "3.x"

This should be a good starting point: https://docs.github.com/en/free-pro-team@latest/actions/guides/building-and-testing-python
Note this suggestion:

You'll need to invoke tox using the -e py option to choose the version of Python in your PATH, rather than specifying a specific version.

Something like this might work as the matrix setup in workflow yaml:

strategy:
  matrix:
    python-version: [2.7, 3.6, 3.7, 3.8, 3.9]
    os: [ubuntu-latest, windows-latest, macos-latest]
    toxenv: py
    include:
      - python-version: 3.x
        os: ubuntu-latest
        toxenv: with-sslib-master
        experimental: true
      - python-version: 3.x
        os: ubuntu-latest
        toxenv: lint

Things to keep in mind:

• caching pip seems to be fairly easy but requires setting up the path per OS (eww): https://github.com/actions/cache/blob/main/examples.md#python---pip
• I just realized we could cache even the tox directory: this would speed things up even more

[lukpueh]

Also, we just ran out of build minutes on Travis. :(

[joshuagl]

Also, we just ran out of build minutes on Travis. :(

Wow. I didn't even realise we had a build minutes limit. Does that mean we can't do any more Travis builds in December?

[lukpueh]

I didn't know either until I noticed that builds didn't finish and came across this message on https://travis-ci.com/organizations/theupdateframework:

Builds have been temporarily disabled for private and public repositories due to a negative credit balance. Please go to the Plan page to replenish your credit balance.

Not exactly sure when credits will reset.

[lukpueh]

IIUC, GitHub actions has no limits for open source projects (see first sentence in Billing for GitHub Actions). I am happy to set this up first thing next week.