See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/858/display/redirect

Changes:

------------------------------------------ [...truncated 6.96 KiB...] ok: [localhost] => git_log.stdout: 0aa90c2e29b9f1e374b6da9ee15e109eba5f7561 [xla:cpu] Optimize CopyThunk for small shapes

TASK [tensorflow : Record git commit hash to variable] ************************* Thursday 11 July 2024 01:11:07 +0100 (0:00:00.036) 0:01:02.788 ********* ok: [localhost]

TASK [tensorflow : Save git commit hash to file] ******************************* Thursday 11 July 2024 01:11:07 +0100 (0:00:00.055) 0:01:02.844 ********* changed: [localhost]

TASK [Build TensorFlow for each Python version (dual abi)] ********************* Thursday 11 July 2024 01:11:08 +0100 (0:00:01.143) 0:01:03.988 *********

TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Thursday 11 July 2024 01:11:09 +0100 (0:00:00.438) 0:01:04.427 ********* changed: [localhost]

TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Thursday 11 July 2024 01:11:09 +0100 (0:00:00.655) 0:01:05.082 ********* skipping: [localhost]

TASK [Fetch TensorFlow] ******************************************************** Thursday 11 July 2024 01:11:09 +0100 (0:00:00.030) 0:01:05.113 ********* skipping: [localhost]

TASK [(Python 3.9) patch TensorFlow tree] ************************************** Thursday 11 July 2024 01:11:09 +0100 (0:00:00.031) 0:01:05.144 *********

TASK [tensorflow : Create linklibs shell script] ******************************* Thursday 11 July 2024 01:11:09 +0100 (0:00:00.080) 0:01:05.224 ********* changed: [localhost]

TASK [tensorflow : Use nightly package requirements] *************************** Thursday 11 July 2024 01:11:10 +0100 (0:00:00.688) 0:01:05.913 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Thursday 11 July 2024 01:11:11 +0100 (0:00:00.622) 0:01:06.535 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Thursday 11 July 2024 01:11:11 +0100 (0:00:00.688) 0:01:07.224 ********* ok: [localhost]

TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Thursday 11 July 2024 01:11:13 +0100 (0:00:01.483) 0:01:08.707 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Thursday 11 July 2024 01:11:51 +0100 (0:00:38.056) 0:01:46.764 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Thursday 11 July 2024 01:11:52 +0100 (0:00:01.235) 0:01:47.999 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Thursday 11 July 2024 01:11:53 +0100 (0:00:00.795) 0:01:48.795 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Thursday 11 July 2024 01:11:54 +0100 (0:00:00.790) 0:01:49.586 ********* changed: [localhost]

TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Thursday 11 July 2024 01:11:55 +0100 (0:00:00.866) 0:01:50.452 ********* changed: [localhost]

TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Thursday 11 July 2024 01:11:56 +0100 (0:00:01.854) 0:01:52.307 ********* ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip

TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Thursday 11 July 2024 01:11:56 +0100 (0:00:00.038) 0:01:52.345 ********* fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Invocation ID: fc992184-8a7b-4426-ad26-edbf32fdb4dd INFO: Reading 'startup' options from /tf/tensorflow/.bazelrc: --windows_enable_symlinks INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility INFO: Reading rc options for 'build' from /usertools/aarch64_clang.bazelrc: 'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/local/bin/python3 --python_path=/usr/local/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --linkopt=-fuse-ld=lld --linkopt=-lm --linkopt=-Wl,--undefined-version --host_linkopt=-fuse-ld=lld --host_linkopt=-lm --host_linkopt=-Wl,--undefined-version --distinct_host_configuration=false --copt=-Wno-gnu-offsetof-extensions --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_clang_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3 INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64_clang.bazelrc: --disk_cache=/tf/cache INFO: Found applicable config definition build:mkl_aarch64_threadpool in file /tf/tensorflow/.bazelrc: --define=build_with_mkl_aarch64=true -c opt INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS Loading: Loading: DEBUG: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/local_xla/third_party/py/python_repo.bzl:109:10: Using hermetic Python 3.9 Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: 0 packages loaded Analyzing: target //tensorflow/tools/pip_package:wheel (1 packages loaded, 0 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (10 packages loaded, 12 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (157 packages loaded, 306 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (274 packages loaded, 4204 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (277 packages loaded, 4204 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (277 packages loaded, 4204 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (278 packages loaded, 4230 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (396 packages loaded, 5060 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (625 packages loaded, 22801 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (662 packages loaded, 30663 targets configured) WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/zlib.net/fossils/zlib-1... failed: class java.io.FileNotFoundException GET returned 404 Not Found Analyzing: target //tensorflow/tools/pip_package:wheel (665 packages loaded, 35604 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (681 packages loaded, 40809 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (682 packages loaded, 42554 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (691 packages loaded, 43201 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (692 packages loaded, 43711 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (692 packages loaded, 44119 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (693 packages loaded, 44550 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (695 packages loaded, 44677 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (695 packages loaded, 44677 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (697 packages loaded, 44800 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (697 packages loaded, 44800 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (698 packages loaded, 45564 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (700 packages loaded, 45983 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (700 packages loaded, 45983 targets configured) INFO: Analyzed target //tensorflow/tools/pip_package:wheel (701 packages loaded, 46082 targets configured). INFO: Found 1 target... [0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (3 actions, 1 running) ERROR: /tf/tensorflow/tensorflow/python/util/BUILD:170:27: Compiling tensorflow/python/util/tensor_float_32.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/util:_pywrap_tensor_float_32_execution.so) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/util/_objs/_pywrap_tensor_float_32_execution.so/tensor_float_32.pic.d ... (remaining 82 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/com_google_absl/absl/time/internal/cctz/BUILD.bazel:21:11: Compiling absl/time/internal/cctz/src/civil_time_detail.cc failed: (Exit 1): clang failed: error executing command (from target @com_google_absl//absl/time/internal/cctz:civil_time) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/com_google_absl/absl/time/internal/cctz/_objs/civil_time/civil_time_detail.pic.d ... (remaining 46 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/zlib/BUILD.bazel:5:11: Compiling zutil.c failed: (Exit 1): clang failed: error executing command (from target @zlib//:zlib) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/zlib/_objs/zlib/zutil.pic.d '-frandom-seed=bazel-out/aarch64-opt/bin/external/zlib/_objs/zlib/zutil.pic.o' ... (remaining 49 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/zlib/BUILD.bazel:5:11: Compiling zutil.c [for tool] failed: (Exit 1): clang failed: error executing command (from target @zlib//:zlib) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt-exec-50AE0418/bin/external/zlib/_objs/zlib/zutil.d '-frandom-seed=bazel-out/aarch64-opt-exec-50AE0418/bin/external/zlib/_objs/zlib/zutil.o' ... (remaining 35 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/org_brotli/BUILD:121:11: Compiling c/enc/utf8_util.c failed: (Exit 1): clang failed: error executing command (from target @org_brotli//:brotlienc) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/org_brotli/_objs/brotlienc/utf8_util.pic.d ... (remaining 56 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/org_brotli/BUILD:121:11: Compiling c/enc/static_dict.c failed: (Exit 1): clang failed: error executing command (from target @org_brotli//:brotlienc) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/org_brotli/_objs/brotlienc/static_dict.pic.d ... (remaining 56 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /tf/tensorflow/tensorflow/python/util/BUILD:185:27: Compiling tensorflow/python/util/determinism.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/util:_pywrap_determinism.so) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/util/_objs/_pywrap_determinism.so/determinism.pic.d ... (remaining 78 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /tf/tensorflow/tensorflow/python/lib/core/BUILD:193:11: Compiling tensorflow/python/lib/core/safe_pyobject_ptr.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/lib/core:safe_pyobject_ptr) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/lib/core/_objs/safe_pyobject_ptr/safe_pyobject_ptr.pic.d ... (remaining 62 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory ERROR: /tf/tensorflow/tensorflow/core/common_runtime/BUILD:1828:11: Compiling tensorflow/core/common_runtime/local_session_selection.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/core/common_runtime:local_session_selection) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/core/common_runtime/_objs/local_session_selection/local_session_selection.pic.d ... (remaining 61 arguments skipped) src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory Target //tensorflow/tools/pip_package:wheel failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 410.122s, Critical Path: 0.61s INFO: 209 processes: 173 internal, 36 local. FAILED: Build did NOT complete successfully stderr_lines: <omitted> stdout: '' stdout_lines: <omitted>

PLAY RECAP ********************************************************************* localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0

Thursday 11 July 2024 01:18:48 +0100 (0:06:51.082) 0:08:43.428 ********* =============================================================================== tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git - 411.08s tensorflow_in_container : (Python 3.9) start build container ----------- 38.06s Cleanup after previous build ------------------------------------------- 32.73s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 21.54s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.00s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.85s Gathering Facts --------------------------------------------------------- 1.63s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.48s Create cache dirs ------------------------------------------------------- 1.44s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.24s tensorflow : Save git commit hash to file ------------------------------- 1.14s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.87s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.80s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.79s tensorflow : Create linklibs shell script ------------------------------- 0.69s tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.69s tensorflow_in_container : Reset checkout to clean state ----------------- 0.66s tensorflow : Use nightly package requirements --------------------------- 0.62s tensorflow : Get oneline version of git log for this checkout ----------- 0.58s Build TensorFlow for each Python version (dual abi) --------------------- 0.44s Build step 'Execute shell' marked build as failure