Hi Beata,
On 23/02/2023 11:05, Beata Michalska wrote:
Hi Teo On Tue, Feb 21, 2023 at 04:43:33PM +0000, Teo Couprie Diaz wrote:
Add two skip files for running the syscalls set of tests in compat. One skips the tests failing because of Musl, the second the tests that we skip or don't build at all.
Update the README to reflect those additions and explain how to use the skip files.
Signed-off-by: Teo Couprie Diaz teo.coupriediaz@arm.com
README.rst | 21 ++++++++ runtest/syscalls_compat.skip | 96 ++++++++++++++++++++++++++++++++++++ runtest/syscalls_musl.skip | 16 ++++++ 3 files changed, 133 insertions(+) create mode 100644 runtest/syscalls_compat.skip create mode 100644 runtest/syscalls_musl.skip
diff --git a/README.rst b/README.rst index 8b6e3a776..4efa6d38f 100644 --- a/README.rst +++ b/README.rst @@ -136,6 +136,27 @@ Running tests: runltp -f morello_transitional,morello_transitional_extended +Skipping tests: +---------------
+In certain cases, you might want to skip tests that are known to be failing or are yet to be supported.
+This is done by passing a file containing the names of the tests to be skip with the option ``-S`` to +``runltp``.
+We currently provide the following skip files under ``runtest/``:
- ``syscalls_musl.skip``: Skips all tests failing on a regular Aarch64 Musl based system.
- ``syscalls_compat.skip``: Skips all tests not considered for COMPAT64 with a PCuABI kernel.
I think it would be better to replace all COMPAT64 mentions here (as this is kernel specific term) and use 'plain aarch64' instead (that might include renaming the skip file as well)
I see, that does make sense. Will update in a v2.
+``runltp`` does not support passing multiple skip files, however this can be circumvented by using +process substitution.
+For example, in COMPAT64 on a Musl system to test all syscalls but skip all known failures, one +could run the following command:
+.. code-block::
- ./runltp -f syscalls -S <( cat runtest/syscalls_{compat,musl}.skip )
Developers playground
diff --git a/runtest/syscalls_compat.skip b/runtest/syscalls_compat.skip new file mode 100644 index 000000000..b9af14a3a --- /dev/null +++ b/runtest/syscalls_compat.skip @@ -0,0 +1,96 @@ +# This skip file contains all the tests skipped from the syscalls command file +# for testing the PCuABI kernel in COMPAT64.
+# leapsec01 is skipped as systemd-timesyncd interfers with it and it doesn't +# test any worthwhile functionality. +leapsec01
+# We do not build most *_64 tests, skip them. +posix_fadvise01_64 +posix_fadvise02_64 +posix_fadvise03_64 +posix_fadvise04_64 +fcntl01_64 +fcntl02_64 +fcntl03_64 +fcntl04_64 +fcntl05_64 +fcntl06_64 +fcntl07_64 +fcntl08_64 +fcntl09_64 +fcntl10_64 +fcntl11_64 +fcntl12_64 +fcntl13_64 +fcntl14_64 +fcntl15_64 +fcntl16_64 +fcntl17_64 +fcntl18_64 +fcntl19_64 +fcntl20_64 +fcntl21_64 +fcntl22_64 +fcntl23_64 +fcntl24_64 +fcntl25_64 +fcntl26_64 +fcntl27_64 +fcntl28_64 +fcntl29_64 +fcntl30_64 +fcntl31_64 +fcntl32_64 +fcntl33_64 +fcntl34_64 +fcntl35_64 +fcntl36_64 +fcntl37_64 +fcntl38_64 +fcntl39_64 +fstat02_64 +fstat03_64 +fstatfs01_64 +fstatfs02_64 +ftruncate01_64 +ftruncate03_64 +ftruncate04_64 +lstat01A_64 +lstat02_64 +pread01_64 +pread02_64 +preadv01_64 +preadv02_64 +preadv03_64 +preadv201_64 +preadv202_64 +preadv203_64 +pselect01_64 +pselect02_64 +pselect03_64 +pwrite01_64 +pwrite02_64 +pwrite03_64 +pwrite04_64 +pwritev01_64 +pwritev02_64 +pwritev03_64 +pwritev201_64 +pwritev202_64 +sendfile02_64 +sendfile03_64 +sendfile04_64 +sendfile05_64 +sendfile06_64 +sendfile07_64 +sendfile08_64 +sendfile09_64 +stat01_64 +stat02_64 +stat03_64 +statfs01_64 +statfs02_64 +statfs03_64 +truncate02_64 +truncate03_64 diff --git a/runtest/syscalls_musl.skip b/runtest/syscalls_musl.skip new file mode 100644 index 000000000..92c952dac --- /dev/null +++ b/runtest/syscalls_musl.skip @@ -0,0 +1,16 @@ +# All the following tests from the syscalls test list are failing in a standard +# Aarch64 Musl-based distribution (Alpine Linux). Thus they depend on Musl
Is Alpine Linux a standard distribution?
I would say it is quite well known: it's the second most downloaded Docker image for a Linux distribution (Behind Ubuntu) and a lot of others use it as a base, for example. It's also somewhat supported by LTP as it's part of the build CI. But I guess it really depends on what you consider standard ?
Are those known failures with reported bugs against either ?
On the other hand, Musl is not officially supported in LTP. There doesn't seem to be anything related to those issues on the LTP side, and not much more on the Alpine side either. LTP is available as a package there but I couldn't find the failing tests on the issue tracker, although there was some recent activity from one of the LTP maintainers who fixed some lint issues for the Alpine package.
BR B.
Thanks for having a look !
Téo
+# or LTP, rather than the kernel, to be fixed. +gethostbyname_r01 +mq_open01 +nftw01 +nftw6401 +open13 +perf_event_open02 +recvmmsg01 +sbrk01 +sendmsg01 +setgroups03 +sigrelse01 +sigtimedwait01
+sigwaitinfo01
2.25.1
linux-morello-ltp mailing list -- linux-morello-ltp@op-lists.linaro.org To unsubscribe send an email to linux-morello-ltp-leave@op-lists.linaro.org