On 15/03/2023 12:18, Kevin Brodsky wrote:
On 15/03/2023 12:21, Beata Michalska wrote:
On Wed, Mar 15, 2023 at 10:50:11AM +0000, Teo Couprie Diaz wrote:
On 06/03/2023 11:59, Teo Couprie Diaz wrote:
Add two skip files for running the syscalls set of tests in compat. One skips the tests failing because of Musl, the second the tests that we skip or don't build at all.
Update the README to reflect those additions and explain how to use the skip files.
This is preparation for CI testing in Morello release 1.6 and later.
Signed-off-by: Teo Couprie Diaz teo.coupriediaz@arm.com
v3: - Added *_16 tests to the skip list - Added tests from kernel/security as we don't build them yet - Mentioned Morello 1.6 release v2: - Removed mentions of COMPAT - Reworded comment around Alpine - Added bpf_prog02 to the musl skips after further testing README.rst | 25 ++++ runtest/syscalls_aarch64_pcuabi.skip | 172 +++++++++++++++++++++++++++
Going through the tests in purecap it turns out we would need a skip list for it as well (`process_vm_{read,write}v` tests pass capabilities between processes through memory mappings, which is explicitly disallowed in purecap) Thus I wonder if the name I chose here is good enough ? Would it make sense to have `syscalls_aarch64_pcuabi.skip` and `syscalls_pcuabi.skip` ? Or maybe `syscalls_purecap_pcuabi.skip` ?
Given that in purecap we need to skip all those that are present in the compat skip list as it's just system issues (leapsec) and tests that aren't built, I'm not sure of the naming anymore. I'm not too keen on moving out the not built tests outside of the aarch64 skip list to an additional one.
What do you think ?
It's good this has been brought up (great timing as well as the patch was about to be merged).
Glad I didn't think about it longer then, hopefully the timing wasn't too inconvenient !
So it looks like we need 3 flavours of skip tests for different configuration, plus additional set for tests we are deliberately ignoring due to test environment limitations (leapsec) or due to build setup. I was thinking we could have 3 skip files as:
- syscalls_morello_skip - for purecap skip tests
- syscalls_morello_aarch64_skip - for plan aarch64 skip tests
I think using "morello" as a general suffix is fine, but then I would swap those two: syscalls_morello_skip for what we always skip regardless of the ABI, and then syscalls_morello_purecap_skip for what we only skip in purecap. There should be no test we skip in the standard AArch64 ABI but not in purecap.
I didn't think of using Morello, I like it. I'd agree with Kevin on the swap though: in theory if we skip something in AArch64 it should be skipped in purecap too, but not the reverse.
Kevin
- syscalls_morello_musl_skip - for Musl related skip tests
(note the *-morello-* tagging, which might be nice to have as we have already that notion in runtest cmd files) As per those tests that are not being build or have dependencies on run env, how about adding install step to append those automatically to either syscalls_morello_skip or syscalls_morello_aarch64_skip, depending on which variant is being considered. (note the actual naming can be different)
Mmmh... It would make it cleaner in a clean repository, but at the same time it would make it a bit less explicit what tests are being skipped and why as it would end up in the build scripts. We could always add some documentation detailing what isn't built and why, which could help. There's also the fact that you would need the AArch64 skip list in purecap as mentioned above. But I guess while we're doing build time shenanigans we could append it to the purecap list as well.
Instead of having a list of *_16 and *_64 we could use the make targets to clear those entries from syscalls list ?
I'm not too sure on removing the entries from the syscalls list : it's an additional step if you want to do anything with them and you lose track that they are part of the "regular" list but are not run here. Maybe it's not that important, I don't really know.
BR B.
Thanks both, Téo
runtest/syscalls_musl.skip | 18 +++ 3 files changed, 215 insertions(+) create mode 100644 runtest/syscalls_aarch64_pcuabi.skip create mode 100644 runtest/syscalls_musl.skip
Thanks,
Téo
linux-morello-ltp mailing list -- linux-morello-ltp@op-lists.linaro.org To unsubscribe send an email to linux-morello-ltp-leave@op-lists.linaro.org
linux-morello-ltp mailing list -- linux-morello-ltp@op-lists.linaro.org To unsubscribe send an email to linux-morello-ltp-leave@op-lists.linaro.org