framework: optimize test key fetching to fix build job going out of memory

[unnawut]

🗒️ Description

Following #320 that we had to limit the number of build workers. I think it was because the key manager is loading all keys even though most of them are not used. With parallel workers, this saturates the memory really fast.

This PR update the test key manager to lazy load individual keys only when they are needed.

Because -n auto was failing for all builds previously, so #320 set it to 2. If this PR sets it back to -n auto and the build passes then this solution fixes the original issue.

🔗 Related Issues or PRs

✅ Checklist

• Ran tox checks to avoid unnecessary CI fails:

  uvx tox

• Considered adding appropriate tests for the changes.
• Considered updating the online docs in the ./docs/ directory.

[anshalshukla]

lgtm

[tcoratger]

Sounds good