See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/818/display/redirect
Changes:
------------------------------------------ [...truncated 7.02 KiB...]
TASK [tensorflow : Record git commit hash to variable] ************************* Saturday 01 June 2024 01:11:14 +0100 (0:00:00.036) 0:01:10.344 ********* ok: [localhost]
TASK [tensorflow : Save git commit hash to file] ******************************* Saturday 01 June 2024 01:11:14 +0100 (0:00:00.058) 0:01:10.403 ********* changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] ********************* Saturday 01 June 2024 01:11:15 +0100 (0:00:01.103) 0:01:11.506 *********
TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Saturday 01 June 2024 01:11:16 +0100 (0:00:00.411) 0:01:11.917 ********* changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Saturday 01 June 2024 01:11:16 +0100 (0:00:00.690) 0:01:12.608 ********* skipping: [localhost]
TASK [Fetch TensorFlow] ******************************************************** Saturday 01 June 2024 01:11:16 +0100 (0:00:00.029) 0:01:12.638 ********* skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] ************************************** Saturday 01 June 2024 01:11:17 +0100 (0:00:00.030) 0:01:12.668 *********
TASK [tensorflow : Create linklibs shell script] ******************************* Saturday 01 June 2024 01:11:17 +0100 (0:00:00.082) 0:01:12.751 ********* changed: [localhost]
TASK [tensorflow : Use nightly package requirements] *************************** Saturday 01 June 2024 01:11:17 +0100 (0:00:00.695) 0:01:13.446 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Saturday 01 June 2024 01:11:18 +0100 (0:00:00.638) 0:01:14.084 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Saturday 01 June 2024 01:11:19 +0100 (0:00:00.688) 0:01:14.773 ********* ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Saturday 01 June 2024 01:11:20 +0100 (0:00:01.629) 0:01:16.403 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Saturday 01 June 2024 01:11:22 +0100 (0:00:02.163) 0:01:18.567 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Saturday 01 June 2024 01:11:24 +0100 (0:00:01.345) 0:01:19.912 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Saturday 01 June 2024 01:11:25 +0100 (0:00:00.905) 0:01:20.818 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Saturday 01 June 2024 01:11:26 +0100 (0:00:00.843) 0:01:21.661 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Saturday 01 June 2024 01:11:26 +0100 (0:00:00.865) 0:01:22.526 ********* changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Saturday 01 June 2024 01:11:29 +0100 (0:00:02.342) 0:01:24.869 ********* ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Saturday 01 June 2024 01:11:29 +0100 (0:00:00.037) 0:01:24.907 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] ************* Saturday 01 June 2024 02:20:32 +0100 (1:09:02.754) 1:10:27.662 ********* skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] ***************** Saturday 01 June 2024 02:20:32 +0100 (0:00:00.040) 1:10:27.702 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] *** Saturday 01 June 2024 02:20:33 +0100 (0:00:01.579) 1:10:29.282 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] ******* Saturday 01 June 2024 02:20:36 +0100 (0:00:02.703) 1:10:31.985 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] *** Saturday 01 June 2024 02:20:37 +0100 (0:00:00.966) 1:10:32.952 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] *** Saturday 01 June 2024 04:41:53 +0100 (2:21:16.641) 3:31:49.593 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] *** Saturday 01 June 2024 04:41:55 +0100 (0:00:01.870) 3:31:51.463 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ******** Saturday 01 June 2024 04:41:56 +0100 (0:00:00.690) 3:31:52.154 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] *** Saturday 01 June 2024 04:41:57 +0100 (0:00:00.698) 3:31:52.853 ********* changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ****** Saturday 01 June 2024 04:41:57 +0100 (0:00:00.704) 3:31:53.557 ********* ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] ********* Saturday 01 June 2024 04:41:57 +0100 (0:00:00.076) 3:31:53.634 ********* ok: [localhost] => ut_tests_line: 'Executed 3086 out of 3086 tests: 3086 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] ************* Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.674 ********* ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] ******* Saturday 01 June 2024 04:41:58 +0100 (0:00:00.127) 3:31:53.802 ********* ok: [localhost] => tests_flaky: - //tensorflow/core/kernels/mkl:mkl_qmatmul_op_test FLAKY, failed in 1 out of 2 in 0.3s
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] ********** Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.842 ********* ok: [localhost] => tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] *** Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.881 ********* fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- + for wheel in /tf/pkg/*.whl + echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...' + python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg + tee check.txt
real 1m27.249s user 1m11.201s sys 0m15.533s stderr_lines: <omitted> stdout: |- Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl... INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel raise ValueError( ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main rval = args.func(args, p) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute out_wheel = repair_wheel( File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel strip_symbols(itertools.chain(libs_to_strip, extensions)) File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__ dir2zip(self.name, self.out_wheel, date_time) File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip with zipfile.ZipFile(zip_fname, "w", compression=compression) as z: File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__ self.fp = io.open(file, filemode) PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl' stdout_lines: <omitted>
PLAY RECAP ********************************************************************* localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Saturday 01 June 2024 04:43:26 +0100 (0:01:28.052) 3:33:21.934 ********* =============================================================================== tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 8476.64s tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 4142.75s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 88.05s Cleanup after previous build ------------------------------------------- 38.04s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 22.84s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.94s tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 2.70s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.34s tensorflow_in_container : (Python 3.9) start build container ------------ 2.16s tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 1.87s Gathering Facts --------------------------------------------------------- 1.69s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.63s tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 1.58s Create cache dirs ------------------------------------------------------- 1.43s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.35s tensorflow : Save git commit hash to file ------------------------------- 1.10s tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.97s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.91s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.87s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.84s Build step 'Execute shell' marked build as failure
See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/819/display/redirect
Changes:
------------------------------------------ [...truncated 7.02 KiB...]
TASK [tensorflow : Record git commit hash to variable] ************************* Sunday 02 June 2024 01:11:13 +0100 (0:00:00.036) 0:01:03.796 *********** ok: [localhost]
TASK [tensorflow : Save git commit hash to file] ******************************* Sunday 02 June 2024 01:11:13 +0100 (0:00:00.054) 0:01:03.851 *********** changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] ********************* Sunday 02 June 2024 01:11:14 +0100 (0:00:01.128) 0:01:04.980 ***********
TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Sunday 02 June 2024 01:11:14 +0100 (0:00:00.432) 0:01:05.412 *********** changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Sunday 02 June 2024 01:11:15 +0100 (0:00:00.642) 0:01:06.054 *********** skipping: [localhost]
TASK [Fetch TensorFlow] ******************************************************** Sunday 02 June 2024 01:11:15 +0100 (0:00:00.029) 0:01:06.084 *********** skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] ************************************** Sunday 02 June 2024 01:11:15 +0100 (0:00:00.030) 0:01:06.114 ***********
TASK [tensorflow : Create linklibs shell script] ******************************* Sunday 02 June 2024 01:11:15 +0100 (0:00:00.081) 0:01:06.196 *********** changed: [localhost]
TASK [tensorflow : Use nightly package requirements] *************************** Sunday 02 June 2024 01:11:16 +0100 (0:00:00.686) 0:01:06.882 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Sunday 02 June 2024 01:11:17 +0100 (0:00:00.641) 0:01:07.523 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Sunday 02 June 2024 01:11:17 +0100 (0:00:00.679) 0:01:08.203 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Sunday 02 June 2024 01:11:19 +0100 (0:00:01.526) 0:01:09.729 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Sunday 02 June 2024 01:28:12 +0100 (0:16:53.334) 0:18:03.063 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Sunday 02 June 2024 01:28:13 +0100 (0:00:01.214) 0:18:04.277 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Sunday 02 June 2024 01:28:14 +0100 (0:00:00.797) 0:18:05.075 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Sunday 02 June 2024 01:28:15 +0100 (0:00:00.827) 0:18:05.902 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Sunday 02 June 2024 01:28:16 +0100 (0:00:00.806) 0:18:06.709 *********** changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Sunday 02 June 2024 01:28:18 +0100 (0:00:01.866) 0:18:08.576 *********** ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Sunday 02 June 2024 01:28:18 +0100 (0:00:00.036) 0:18:08.613 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] ************* Sunday 02 June 2024 02:20:18 +0100 (0:52:00.398) 1:10:09.011 *********** skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] ***************** Sunday 02 June 2024 02:20:18 +0100 (0:00:00.033) 1:10:09.044 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] *** Sunday 02 June 2024 02:20:19 +0100 (0:00:00.931) 1:10:09.976 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] ******* Sunday 02 June 2024 02:20:20 +0100 (0:00:00.894) 1:10:10.871 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] *** Sunday 02 June 2024 02:20:21 +0100 (0:00:00.711) 1:10:11.582 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] *** Sunday 02 June 2024 04:22:29 +0100 (2:02:08.739) 3:12:20.322 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] *** Sunday 02 June 2024 04:22:31 +0100 (0:00:01.764) 3:12:22.087 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ******** Sunday 02 June 2024 04:22:32 +0100 (0:00:00.690) 3:12:22.777 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] *** Sunday 02 June 2024 04:22:32 +0100 (0:00:00.694) 3:12:23.472 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ****** Sunday 02 June 2024 04:22:33 +0100 (0:00:00.704) 3:12:24.176 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] ********* Sunday 02 June 2024 04:22:33 +0100 (0:00:00.075) 3:12:24.252 *********** ok: [localhost] => ut_tests_line: 'Executed 3086 out of 3086 tests: 3086 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] ************* Sunday 02 June 2024 04:22:33 +0100 (0:00:00.038) 3:12:24.291 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] ******* Sunday 02 June 2024 04:22:33 +0100 (0:00:00.128) 3:12:24.419 *********** ok: [localhost] => tests_flaky: - //tensorflow/python/kernel_tests/linalg:matrix_triangular_solve_op_test_cpu FLAKY, failed in 1 out of 4 in 900.3s
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] ********** Sunday 02 June 2024 04:22:33 +0100 (0:00:00.038) 3:12:24.458 *********** ok: [localhost] => tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] *** Sunday 02 June 2024 04:22:34 +0100 (0:00:00.038) 3:12:24.497 *********** fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- + for wheel in /tf/pkg/*.whl + echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...' + python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg + tee check.txt
real 1m26.902s user 1m10.280s sys 0m16.224s stderr_lines: <omitted> stdout: |- Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl... INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel raise ValueError( ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main rval = args.func(args, p) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute out_wheel = repair_wheel( File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel strip_symbols(itertools.chain(libs_to_strip, extensions)) File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__ dir2zip(self.name, self.out_wheel, date_time) File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip with zipfile.ZipFile(zip_fname, "w", compression=compression) as z: File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__ self.fp = io.open(file, filemode) PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl' stdout_lines: <omitted>
PLAY RECAP ********************************************************************* localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Sunday 02 June 2024 04:24:01 +0100 (0:01:27.707) 3:13:52.204 *********** =============================================================================== tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 7328.74s tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 3120.40s tensorflow_in_container : (Python 3.9) start build container --------- 1013.33s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 87.71s Cleanup after previous build ------------------------------------------- 31.29s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 22.57s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.47s Gathering Facts --------------------------------------------------------- 2.61s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.87s tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 1.76s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.53s Create cache dirs ------------------------------------------------------- 1.41s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.21s tensorflow : Save git commit hash to file ------------------------------- 1.13s tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 0.93s tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.89s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.83s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.81s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.80s tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.71s Build step 'Execute shell' marked build as failure
See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/820/display/redirect
Changes:
------------------------------------------ [...truncated 7.00 KiB...] git_log.stdout: a96e3431f6f97fa248a289ca82b9b2c262dc3de4 Implements the `DisassembleIntoSingleDeviceArrays` method for the BasicStringArray class.
TASK [tensorflow : Record git commit hash to variable] ************************* Monday 03 June 2024 01:11:07 +0100 (0:00:00.036) 0:01:03.173 *********** ok: [localhost]
TASK [tensorflow : Save git commit hash to file] ******************************* Monday 03 June 2024 01:11:07 +0100 (0:00:00.054) 0:01:03.228 *********** changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] ********************* Monday 03 June 2024 01:11:08 +0100 (0:00:01.104) 0:01:04.333 ***********
TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Monday 03 June 2024 01:11:09 +0100 (0:00:00.435) 0:01:04.768 *********** changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Monday 03 June 2024 01:11:10 +0100 (0:00:00.650) 0:01:05.419 *********** skipping: [localhost]
TASK [Fetch TensorFlow] ******************************************************** Monday 03 June 2024 01:11:10 +0100 (0:00:00.031) 0:01:05.450 *********** skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] ************************************** Monday 03 June 2024 01:11:10 +0100 (0:00:00.031) 0:01:05.482 ***********
TASK [tensorflow : Create linklibs shell script] ******************************* Monday 03 June 2024 01:11:10 +0100 (0:00:00.082) 0:01:05.565 *********** changed: [localhost]
TASK [tensorflow : Use nightly package requirements] *************************** Monday 03 June 2024 01:11:10 +0100 (0:00:00.686) 0:01:06.252 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Monday 03 June 2024 01:11:11 +0100 (0:00:00.626) 0:01:06.878 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Monday 03 June 2024 01:11:12 +0100 (0:00:00.680) 0:01:07.559 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Monday 03 June 2024 01:11:13 +0100 (0:00:01.496) 0:01:09.056 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Monday 03 June 2024 01:28:55 +0100 (0:17:42.197) 0:18:51.254 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Monday 03 June 2024 01:28:57 +0100 (0:00:01.245) 0:18:52.499 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Monday 03 June 2024 01:28:58 +0100 (0:00:00.854) 0:18:53.354 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Monday 03 June 2024 01:28:58 +0100 (0:00:00.819) 0:18:54.173 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Monday 03 June 2024 01:28:59 +0100 (0:00:00.765) 0:18:54.939 *********** changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Monday 03 June 2024 01:29:01 +0100 (0:00:01.843) 0:18:56.782 *********** ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Monday 03 June 2024 01:29:01 +0100 (0:00:00.036) 0:18:56.819 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] ************* Monday 03 June 2024 02:21:03 +0100 (0:52:02.483) 1:10:59.302 *********** skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] ***************** Monday 03 June 2024 02:21:03 +0100 (0:00:00.033) 1:10:59.335 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] *** Monday 03 June 2024 02:21:04 +0100 (0:00:00.992) 1:11:00.327 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] ******* Monday 03 June 2024 02:21:05 +0100 (0:00:00.825) 1:11:01.153 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] *** Monday 03 June 2024 02:21:06 +0100 (0:00:00.706) 1:11:01.859 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] *** Monday 03 June 2024 04:26:16 +0100 (2:05:09.675) 3:16:11.534 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] *** Monday 03 June 2024 04:26:17 +0100 (0:00:01.690) 3:16:13.225 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ******** Monday 03 June 2024 04:26:18 +0100 (0:00:00.694) 3:16:13.920 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] *** Monday 03 June 2024 04:26:19 +0100 (0:00:00.693) 3:16:14.613 *********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ****** Monday 03 June 2024 04:26:19 +0100 (0:00:00.702) 3:16:15.315 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] ********* Monday 03 June 2024 04:26:20 +0100 (0:00:00.074) 3:16:15.390 *********** ok: [localhost] => ut_tests_line: 'Executed 3086 out of 3086 tests: 3086 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] ************* Monday 03 June 2024 04:26:20 +0100 (0:00:00.039) 3:16:15.429 *********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] ******* Monday 03 June 2024 04:26:20 +0100 (0:00:00.125) 3:16:15.554 *********** ok: [localhost] => tests_flaky: []
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] ********** Monday 03 June 2024 04:26:20 +0100 (0:00:00.039) 3:16:15.594 *********** ok: [localhost] => tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] *** Monday 03 June 2024 04:26:20 +0100 (0:00:00.039) 3:16:15.633 *********** fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- + for wheel in /tf/pkg/*.whl + echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...' + python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg + tee check.txt
real 1m27.266s user 1m10.297s sys 0m16.354s stderr_lines: <omitted> stdout: |- Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl... INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel raise ValueError( ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main rval = args.func(args, p) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute out_wheel = repair_wheel( File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel strip_symbols(itertools.chain(libs_to_strip, extensions)) File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__ dir2zip(self.name, self.out_wheel, date_time) File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip with zipfile.ZipFile(zip_fname, "w", compression=compression) as z: File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__ self.fp = io.open(file, filemode) PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl' stdout_lines: <omitted>
PLAY RECAP ********************************************************************* localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Monday 03 June 2024 04:27:48 +0100 (0:01:28.046) 3:17:43.679 *********** =============================================================================== tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 7509.68s tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 3122.48s tensorflow_in_container : (Python 3.9) start build container --------- 1062.20s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 88.05s Cleanup after previous build ------------------------------------------- 31.03s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 23.09s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.57s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.84s tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 1.69s Gathering Facts --------------------------------------------------------- 1.62s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.50s Create cache dirs ------------------------------------------------------- 1.42s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.25s tensorflow : Save git commit hash to file ------------------------------- 1.10s tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 0.99s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.85s tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.83s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.82s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.77s tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.71s Build step 'Execute shell' marked build as failure
ldcg-ai-ml@op-lists.linaro.org
-
ci_notify＠linaro.org