See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/1188/display/redirect

Changes:

------------------------------------------ [...truncated 6.63 KiB...] git_log.stdout: f2535930fb518a3996cac26d59c3dd2bf9776e77 [XLA:GPU] Clang-tidy cleanup for xla/service/gpu/fusions/mlir/mlir_fusion_emitter.h

TASK [tensorflow : Record git commit hash to variable] ************************* Saturday 01 June 2024 19:11:14 +0100 (0:00:00.037) 0:01:09.722 ********* ok: [localhost]

TASK [tensorflow : Save git commit hash to file] ******************************* Saturday 01 June 2024 19:11:14 +0100 (0:00:00.055) 0:01:09.778 ********* changed: [localhost]

TASK [Build TensorFlow for each Python version (dual abi)] ********************* Saturday 01 June 2024 19:11:15 +0100 (0:00:01.137) 0:01:10.915 *********

TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Saturday 01 June 2024 19:11:15 +0100 (0:00:00.412) 0:01:11.328 ********* changed: [localhost]

TASK [tensorflow_in_container : Remove TensorFlow git sources from previous build] *** Saturday 01 June 2024 19:11:16 +0100 (0:00:00.646) 0:01:11.974 ********* skipping: [localhost]

TASK [Fetch TensorFlow] ******************************************************** Saturday 01 June 2024 19:11:16 +0100 (0:00:00.029) 0:01:12.004 ********* skipping: [localhost]

TASK [(Python 3.9) patch TensorFlow tree] ************************************** Saturday 01 June 2024 19:11:16 +0100 (0:00:00.030) 0:01:12.034 *********

TASK [tensorflow : Create linklibs shell script] ******************************* Saturday 01 June 2024 19:11:16 +0100 (0:00:00.081) 0:01:12.116 ********* changed: [localhost]

TASK [tensorflow : Use nightly package requirements] *************************** Saturday 01 June 2024 19:11:17 +0100 (0:00:00.685) 0:01:12.802 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Saturday 01 June 2024 19:11:17 +0100 (0:00:00.626) 0:01:13.429 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Saturday 01 June 2024 19:11:18 +0100 (0:00:00.680) 0:01:14.109 ********* ok: [localhost]

TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Saturday 01 June 2024 19:11:19 +0100 (0:00:01.486) 0:01:15.596 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Saturday 01 June 2024 19:27:00 +0100 (0:15:40.311) 0:16:55.907 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Saturday 01 June 2024 19:27:01 +0100 (0:00:01.205) 0:16:57.113 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Saturday 01 June 2024 19:27:02 +0100 (0:00:00.823) 0:16:57.936 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Saturday 01 June 2024 19:27:03 +0100 (0:00:00.866) 0:16:58.803 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Saturday 01 June 2024 19:27:03 +0100 (0:00:00.800) 0:16:59.603 ********* changed: [localhost]

TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Saturday 01 June 2024 19:27:05 +0100 (0:00:01.857) 0:17:01.461 ********* ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip

TASK [tensorflow_in_container : (Python 3.9) build TensorFlow git] ************* Saturday 01 June 2024 19:27:05 +0100 (0:00:00.037) 0:17:01.498 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) build Python package] ************* Saturday 01 June 2024 20:32:04 +0100 (1:04:58.846) 1:22:00.345 ********* skipping: [localhost]

TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] ***************** Saturday 01 June 2024 20:32:04 +0100 (0:00:00.031) 1:22:00.376 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] *** Saturday 01 June 2024 20:32:05 +0100 (0:00:00.909) 1:22:01.285 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create script to run tests] ******* Saturday 01 June 2024 20:32:06 +0100 (0:00:00.869) 1:22:02.155 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) run TensorFlow git tests] ********* Saturday 01 June 2024 20:32:07 +0100 (0:00:00.703) 1:22:02.858 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] *** Saturday 01 June 2024 22:37:19 +0100 (2:05:11.868) 3:27:14.727 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] *** Saturday 01 June 2024 22:37:21 +0100 (0:00:02.366) 3:27:17.093 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ******** Saturday 01 June 2024 22:37:22 +0100 (0:00:00.679) 3:27:17.773 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] *** Saturday 01 June 2024 22:37:22 +0100 (0:00:00.694) 3:27:18.467 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ****** Saturday 01 June 2024 22:37:23 +0100 (0:00:00.700) 3:27:19.167 ********* ok: [localhost]

TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] ********* Saturday 01 June 2024 22:37:23 +0100 (0:00:00.075) 3:27:19.243 ********* ok: [localhost] => ut_tests_line: 'Executed 3087 out of 3087 tests: 3087 tests pass.'

TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] ************* Saturday 01 June 2024 22:37:23 +0100 (0:00:00.062) 3:27:19.305 ********* ok: [localhost]

TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] ******* Saturday 01 June 2024 22:37:23 +0100 (0:00:00.123) 3:27:19.429 ********* ok: [localhost] => tests_flaky: []

TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] ********** Saturday 01 June 2024 22:37:23 +0100 (0:00:00.038) 3:27:19.467 ********* ok: [localhost] => tests_failed: []

TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] *** Saturday 01 June 2024 22:37:23 +0100 (0:00:00.039) 3:27:19.507 ********* fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- + for wheel in /tf/pkg/*.whl + echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...' + python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg + tee check.txt

real 1m22.581s user 1m8.234s sys 0m13.796s stderr_lines: <omitted> stdout: |- Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl... INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel raise ValueError( ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main rval = args.func(args, p) File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute out_wheel = repair_wheel( File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel strip_symbols(itertools.chain(libs_to_strip, extensions)) File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__ dir2zip(self.name, self.out_wheel, date_time) File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip with zipfile.ZipFile(zip_fname, "w", compression=compression) as z: File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__ self.fp = io.open(file, filemode) PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl' stdout_lines: <omitted>

PLAY RECAP ********************************************************************* localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0

Saturday 01 June 2024 22:38:47 +0100 (0:01:23.357) 3:28:42.864 ********* =============================================================================== tensorflow_in_container : (Python 3.9) run TensorFlow git tests ------ 7511.87s tensorflow_in_container : (Python 3.9) build TensorFlow git ---------- 3898.85s tensorflow_in_container : (Python 3.9) start build container ---------- 940.31s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 83.36s Cleanup after previous build ------------------------------------------- 36.54s tensorflow : Fetch TensorFlow from git --------------------------------- 23.26s tensorflow : Remove TensorFlow git sources from previous build ---------- 4.43s Gathering Facts --------------------------------------------------------- 2.63s tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 2.37s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.86s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.49s Create cache dirs ------------------------------------------------------- 1.41s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.21s tensorflow : Save git commit hash to file ------------------------------- 1.14s tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 0.91s tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.87s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.87s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.82s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.80s tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.70s Build step 'Execute shell' marked build as failure