Rework regtest runner

[McSinyx]

The C++ script is rewritten in Makefile and shell script for concision.

[McSinyx]

the entire regtest suite will pass on my system, but can fail on other systems. This should be fixed eventually.

I suppose I should remove ef728c8 from this PR then? Edit: I did.

[McSinyx]

The functional purpose of this patch is, like GH-89, to reliably check signals instead of depending on the shell used by system(3). Moving most of the logic to the Makefile allows caching (7a2210d) and separates test build from test execution.

Distributions would expect make check, while you might want to use make check-debug which tests against e9tool linked against bundled elfutils and Zydis.

[McSinyx]

@GJDuck, could you please test this and see if it fails anything on your setup?

[McSinyx]

@GJDuck, could you please take a look at this one? The missing make check is the only blocker I have before upstreaming my package to Guix.

[GJDuck]

This does not work when I try it. The make check prints a bunch of E9Tool output (the regtests I presume), then fails for me.

What is wrong with the original C++ regtest.cpp program? It is specialized to handle the test cases in a nice clean way. Can make check be set up to just call this program?

[McSinyx]

This does not work when I try it. The make check prints a bunch of E9Tool output (the regtests I presume), then fails for me.

@GJDuck, E9Tool output was from make -C test/regtest, which now builds the patched binaries as well. They are cached and the log flood will not appear unless some particular patching fails. I suppose the actual failure was as follows:

./regtest: 2: Syntax error: "(" unexpected

I used the wrong shell for the regtest script: POSIX sh does not support arrays and on Guix /bin/sh is a minimal Bash so I missed it. Could you please try invoking make -j$(nproc) check-debug again (make check is intended for distributions (e.g. expecting installed Zydis, etc. and would fail for you after a make clean)? I added GitHub Action to this PR and it is successful.

What is wrong with the original C++ regtest.cpp program? It is specialized to handle the test cases in a nice clean way. Can make check be set up to just call this program?

C++ is not great as a shell, pipe logic got drown in a sea of string concatenations to construct commands. This is not a criticism towards your skills, but rather the opposite: it's unobvious for downstream to understand and patch basic test runner behaviors like printing expect/actual diffs. E9Patch's test suite is most likely to not pass entirely at the first packaging try in most distributions, and isolated environment for reproducible build is not great for interactive debugging, so test runners should be as user friendly to tweak as possible.

If this new regtest introduces any inconvenience to you, please let me know so we can archive the best of both worlds.

[GJDuck]

Thanks, I tried it out.

The new make -j$(nproc) check-debug does not print anything though, which I assume means everything passed. I would suggest keeping the old regtest runner, but then adding separate-but-minimal make/scripts that are necessary for distro builds.

I prefer to set up my projects so I do not have to remember how anything works. This is also why I use a build.sh script, since it means I do not have to remember how to build this and my other projects. It is very nice when coming back to a project after >1year.

[McSinyx]

I would suggest keeping the old regtest runner, but then adding separate-but-minimal make/scripts that are necessary for distro builds.

Done! The humongous 8c01d27 is there because GitHub updated the Ubuntu image and the default CC is more strict towards standard compliance.