ldcg-ai-ml April 2024

ldcg-ai-ml@op-lists.linaro.org

1 participants
3 discussions

Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #782

by ci_notify＠linaro.org

See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/7…> Changes: ------------------------------------------ [...truncated 6.74 KiB...] changed: [localhost] TASK [tensorflow : Show git log] *********************************************** Friday 26 April 2024 01:11:25 +0100 (0:00:00.576) 0:01:20.940 ********** ok: [localhost] => git_log.stdout: c977e63b30d098344351b48e61c173afe5a2c6a9 Use the input's shapes directly instead of getting them from the op TASK [tensorflow : Record git commit hash to variable] ************************* Friday 26 April 2024 01:11:25 +0100 (0:00:00.036) 0:01:20.977 ********** ok: [localhost] TASK [tensorflow : Save git commit hash to file] ******************************* Friday 26 April 2024 01:11:25 +0100 (0:00:00.054) 0:01:21.032 ********** changed: [localhost] TASK [Build TensorFlow for each Python version (dual abi)] ********************* Friday 26 April 2024 01:11:26 +0100 (0:00:01.130) 0:01:22.162 ********** TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Friday 26 April 2024 01:11:27 +0100 (0:00:00.435) 0:01:22.597 ********** changed: [localhost] TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Friday 26 April 2024 01:11:27 +0100 (0:00:00.651) 0:01:23.249 ********** skipping: [localhost] TASK [Fetch TensorFlow] ******************************************************** Friday 26 April 2024 01:11:27 +0100 (0:00:00.029) 0:01:23.279 ********** skipping: [localhost] TASK [(Python 3.9) patch TensorFlow tree] ************************************** Friday 26 April 2024 01:11:27 +0100 (0:00:00.030) 0:01:23.309 ********** TASK [tensorflow : Create linklibs shell script] ******************************* Friday 26 April 2024 01:11:27 +0100 (0:00:00.081) 0:01:23.391 ********** changed: [localhost] TASK [tensorflow : Use nightly package requirements] *************************** Friday 26 April 2024 01:11:28 +0100 (0:00:00.688) 0:01:24.079 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Friday 26 April 2024 01:11:29 +0100 (0:00:00.624) 0:01:24.703 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Friday 26 April 2024 01:11:29 +0100 (0:00:00.679) 0:01:25.383 ********** ok: [localhost] TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Friday 26 April 2024 01:11:31 +0100 (0:00:01.710) 0:01:27.094 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Friday 26 April 2024 01:11:45 +0100 (0:00:14.301) 0:01:41.395 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Friday 26 April 2024 01:11:47 +0100 (0:00:01.232) 0:01:42.628 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Friday 26 April 2024 01:11:47 +0100 (0:00:00.848) 0:01:43.476 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Friday 26 April 2024 01:11:48 +0100 (0:00:00.853) 0:01:44.330 ********** changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Friday 26 April 2024 01:11:49 +0100 (0:00:00.883) 0:01:45.213 ********** changed: [localhost] TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Friday 26 April 2024 01:11:51 +0100 (0:00:02.150) 0:01:47.363 ********** ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Friday 26 April 2024 01:11:51 +0100 (0:00:00.038) 0:01:47.402 ********** fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Invocation ID: b31cd5f3-1f73-47b3-8c40-53c592c12ccd INFO: Reading 'startup' options from /tf/tensorflow/.bazelrc: --windows_enable_symlinks INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility INFO: Reading rc options for 'build' from /usertools/aarch64_clang.bazelrc: 'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/local/bin/python3 --python_path=/usr/local/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --linkopt=-fuse-ld=lld --linkopt=-lm --linkopt=-Wl,--undefined-version --host_linkopt=-fuse-ld=lld --host_linkopt=-lm --host_linkopt=-Wl,--undefined-version --distinct_host_configuration=false --copt=-Wno-gnu-offsetof-extensions --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_clang_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3 INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64_clang.bazelrc: --disk_cache=/tf/cache INFO: Found applicable config definition build:mkl_aarch64_threadpool in file /tf/tensorflow/.bazelrc: --define=build_with_mkl_aarch64=true -c opt INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: 0 packages loaded Analyzing: target //tensorflow/tools/pip_package:wheel (1 packages loaded, 0 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (10 packages loaded, 12 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (152 packages loaded, 302 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (268 packages loaded, 4191 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (271 packages loaded, 4216 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (271 packages loaded, 4216 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (391 packages loaded, 5046 targets configured) INFO: Repository pthreadpool instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:166:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository gif instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:340:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository XNNPACK instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:151:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository riegeli instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:868:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository cpuinfo instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:173:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> ERROR: /tf/tensorflow/tensorflow/compiler/mlir/lite/stablehlo/BUILD:758:11: no such package 'buildenv/target': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package. - /tf/tensorflow/buildenv/target and referenced by '//tensorflow/compiler/mlir/lite/stablehlo:optimize_layout' ERROR: Analysis of target '//tensorflow/tools/pip_package:wheel' failed; build aborted: INFO: Elapsed time: 91.644s INFO: 0 processes. FAILED: Build did NOT complete successfully (598 packages loaded, 21661 targets configured) stderr_lines: <omitted> stdout: '' stdout_lines: <omitted> PLAY RECAP ********************************************************************* localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0 Friday 26 April 2024 01:13:24 +0100 (0:01:32.693) 0:03:20.096 ********** =============================================================================== tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git -- 92.69s Cleanup after previous build ------------------------------------------- 49.86s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 21.48s tensorflow_in_container : (Python 3.9) start build container ----------- 14.30s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 5.04s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.15s Gathering Facts --------------------------------------------------------- 1.71s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.71s Create cache dirs ------------------------------------------------------- 1.44s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.23s tensorflow : Save git commit hash to file ------------------------------- 1.13s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.88s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.85s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.85s tensorflow : Create linklibs shell script ------------------------------- 0.69s tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.68s tensorflow_in_container : Reset checkout to clean state ----------------- 0.65s tensorflow : Use nightly package requirements --------------------------- 0.62s tensorflow : Get oneline version of git log for this checkout ----------- 0.58s Build TensorFlow for each Python version (dual abi) --------------------- 0.44s Build step 'Execute shell' marked build as failure

6 months

Build failed in Jenkins: Tensorflow Python package (manylinux2014 nightly) #1142

by ci_notify＠linaro.org

See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/1142/dis…> Changes: ------------------------------------------ [...truncated 314.91 KiB...] [19,191 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/partitioned_topological_sort.cc; 72s local, remote-cache ... (111 actions, 110 running) [19,194 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/partitioned_topological_sort.cc; 73s local, remote-cache ... (111 actions, 110 running) [19,196 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/optimize_functional_ops.cc; 74s local, remote-cache ... (111 actions, 110 running) [19,197 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/optimize_functional_ops.cc; 75s local, remote-cache ... (111 actions, 110 running) [19,197 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/optimize_functional_ops.cc; 76s local, remote-cache ... (112 actions running) [19,200 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 61s local, remote-cache ... (111 actions, 110 running) [19,201 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 62s local, remote-cache ... (111 actions, 110 running) [19,204 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 63s local, remote-cache ... (111 actions, 110 running) [19,205 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 64s local, remote-cache ... (112 actions running) [19,208 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 65s local, remote-cache ... (112 actions running) [19,210 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 66s local, remote-cache ... (112 actions running) [19,211 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 67s local, remote-cache ... (112 actions running) [19,212 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 68s local, remote-cache ... (112 actions running) [19,212 / 19,692] Compiling tensorflow/compiler/mlir/lite/tf_to_tfl_flatbuffer.cc; 73s local, remote-cache ... (112 actions running) [19,213 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 73s local, remote-cache ... (112 actions running) [19,216 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 74s local, remote-cache ... (111 actions, 110 running) [19,216 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 75s local, remote-cache ... (112 actions running) [19,218 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 76s local, remote-cache ... (112 actions running) [19,219 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 77s local, remote-cache ... (111 actions, 110 running) [19,221 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 79s local, remote-cache ... (111 actions, 110 running) [19,221 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 80s local, remote-cache ... (112 actions running) [19,223 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 81s local, remote-cache ... (112 actions running) [19,224 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 82s local, remote-cache ... (112 actions running) [19,225 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 83s local, remote-cache ... (111 actions, 110 running) [19,225 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 85s local, remote-cache ... (112 actions running) [19,226 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 86s local, remote-cache ... (112 actions running) [19,227 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 87s local, remote-cache ... (112 actions running) [19,227 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 88s local, remote-cache ... (112 actions running) [19,228 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 89s local, remote-cache ... (112 actions running) [19,229 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/lower_static_tensor_list.cc; 91s local, remote-cache ... (111 actions, 110 running) [19,231 / 19,692] Compiling tensorflow/compiler/mlir/quantization/stablehlo/passes/lift_quantizable_spots_as_functions.cc; 82s local, remote-cache ... (112 actions running) [19,232 / 19,692] Compiling tensorflow/compiler/mlir/quantization/stablehlo/passes/lift_quantizable_spots_as_functions.cc; 83s local, remote-cache ... (111 actions, 110 running) [19,233 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 76s local, remote-cache ... (111 actions, 110 running) [19,236 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 77s local, remote-cache ... (111 actions, 110 running) [19,238 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 79s local, remote-cache ... (112 actions running) [19,239 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 82s local, remote-cache ... (111 actions, 110 running) [19,241 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 83s local, remote-cache ... (112 actions running) [19,242 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 86s local, remote-cache ... (111 actions, 110 running) [19,244 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 87s local, remote-cache ... (111 actions, 110 running) [19,248 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 88s local, remote-cache ... (112 actions running) [19,251 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 89s local, remote-cache ... (112 actions running) [19,253 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 90s local, remote-cache ... (112 actions running) [19,257 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 91s local, remote-cache ... (112 actions running) [19,257 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 92s local, remote-cache ... (112 actions running) [19,259 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 93s local, remote-cache ... (112 actions running) [19,260 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 94s local, remote-cache ... (112 actions running) [19,262 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 95s local, remote-cache ... (112 actions running) [19,263 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 96s local, remote-cache ... (112 actions running) [19,264 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 97s local, remote-cache ... (112 actions running) [19,267 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 98s local, remote-cache ... (112 actions running) [19,269 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 99s local, remote-cache ... (112 actions running) [19,271 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 100s local, remote-cache ... (112 actions running) [19,271 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 102s local, remote-cache ... (112 actions running) [19,275 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 103s local, remote-cache ... (112 actions running) [19,276 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 105s local, remote-cache ... (111 actions, 110 running) [19,279 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 106s local, remote-cache ... (112 actions running) [19,280 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 111s local, remote-cache ... (111 actions, 110 running) [19,282 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 112s local, remote-cache ... (111 actions, 110 running) [19,287 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 113s local, remote-cache ... (111 actions, 110 running) [19,291 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 114s local, remote-cache ... (111 actions, 110 running) [19,293 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 115s local, remote-cache ... (112 actions running) [19,295 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 116s local, remote-cache ... (112 actions running) [19,296 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 117s local, remote-cache ... (112 actions running) [19,297 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 120s local, remote-cache ... (111 actions, 110 running) [19,303 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 121s local, remote-cache ... (111 actions, 110 running) [19,305 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 122s local, remote-cache ... (111 actions, 110 running) [19,308 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 123s local, remote-cache ... (111 actions, 110 running) [19,311 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 124s local, remote-cache ... (112 actions running) [19,312 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 127s local, remote-cache ... (111 actions, 110 running) [19,314 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 128s local, remote-cache ... (112 actions running) [19,320 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 129s local, remote-cache ... (112 actions running) [19,322 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 131s local, remote-cache ... (112 actions running) [19,331 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 132s local, remote-cache ... (111 actions, 110 running) [19,335 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 133s local, remote-cache ... (112 actions running) [19,340 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 134s local, remote-cache ... (111 actions, 110 running) [19,344 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 135s local, remote-cache ... (112 actions running) [19,347 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 136s local, remote-cache ... (111 actions, 110 running) [19,349 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 137s local, remote-cache ... (112 actions running) [19,351 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 138s local, remote-cache ... (112 actions running) [19,354 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 139s local, remote-cache ... (112 actions running) [19,360 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 140s local, remote-cache ... (112 actions running) [19,362 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 141s local, remote-cache ... (112 actions running) [19,365 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 142s local, remote-cache ... (112 actions running) [19,369 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 143s local, remote-cache ... (112 actions running) [19,369 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 145s local, remote-cache ... (112 actions running) [19,371 / 19,692] Compiling tensorflow/compiler/mlir/lite/transforms/legalize_tf.cc; 146s local, remote-cache ... (112 actions running) [19,373 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/quantize_and_dequantize_op.cc; 44s local, remote-cache ... (112 actions running) [19,374 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/quantize_and_dequantize_op.cc; 45s local, remote-cache ... (111 actions, 110 running) [19,380 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/pad_op.cc; 43s local, remote-cache ... (111 actions, 110 running) [19,381 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/pad_op.cc; 45s local, remote-cache ... (112 actions running) [19,383 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/pad_op.cc; 46s local, remote-cache ... (112 actions running) [19,387 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateless_random_ops.cc; 41s local, remote-cache ... (111 actions, 110 running) [19,387 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateless_random_ops.cc; 42s local, remote-cache ... (112 actions running) [19,389 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 42s local, remote-cache ... (112 actions running) [19,391 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 43s local, remote-cache ... (112 actions running) [19,393 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 44s local, remote-cache ... (112 actions running) [19,396 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 45s local, remote-cache ... (112 actions running) [19,397 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 46s local, remote-cache ... (112 actions running) [19,400 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 47s local, remote-cache ... (111 actions, 110 running) [19,403 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/stateful_random_ops.cc; 49s local, remote-cache ... (112 actions running) [19,405 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matrix_band_part_op.cc; 42s local, remote-cache ... (112 actions running) [19,406 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matrix_band_part_op.cc; 44s local, remote-cache ... (111 actions, 110 running) [19,409 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 44s local, remote-cache ... (112 actions running) [19,409 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 45s local, remote-cache ... (112 actions running) [19,411 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 46s local, remote-cache ... (112 actions running) [19,413 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 48s local, remote-cache ... (111 actions, 110 running) [19,416 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 49s local, remote-cache ... (110 actions running) [19,418 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 50s local, remote-cache ... (112 actions running) [19,420 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 51s local, remote-cache ... (112 actions running) [19,421 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/matmul_op.cc; 52s local, remote-cache ... (112 actions running) [19,424 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/image_resize_ops.cc; 47s local, remote-cache ... (112 actions running) [19,427 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/gather_scatter_ops.cc; 46s local, remote-cache ... (111 actions, 110 running) [19,427 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/gather_scatter_ops.cc; 47s local, remote-cache ... (112 actions running) [19,431 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/gather_op.cc; 48s local, remote-cache ... (111 actions, 110 running) [19,433 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/gather_op.cc; 49s local, remote-cache ... (112 actions running) [19,436 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/fused_conv_ops.cc; 49s local, remote-cache ... (112 actions running) [19,441 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/fused_conv_ops.cc; 50s local, remote-cache ... (111 actions, 110 running) [19,441 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/fused_conv_ops.cc; 51s local, remote-cache ... (112 actions running) [19,445 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/fused_conv_ops.cc; 52s local, remote-cache ... (112 actions running) [19,451 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_stitch_op.cc; 48s local, remote-cache ... (112 actions running) [19,457 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_slice_ops.cc; 48s local, remote-cache ... (112 actions running) [19,458 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_slice_ops.cc; 49s local, remote-cache ... (112 actions running) [19,462 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_slice_ops.cc; 50s local, remote-cache ... (112 actions running) [19,466 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_slice_ops.cc; 51s local, remote-cache ... (111 actions, 110 running) [19,469 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/dynamic_slice_ops.cc; 52s local, remote-cache ... (112 actions running) [19,473 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 50s local, remote-cache ... (112 actions running) [19,476 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 51s local, remote-cache ... (112 actions running) [19,480 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 52s local, remote-cache ... (112 actions running) [19,482 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 53s local, remote-cache ... (112 actions running) [19,483 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 54s local, remote-cache ... (111 actions, 110 running) [19,486 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/depthtospace_op.cc; 55s local, remote-cache ... (111 actions, 110 running) [19,488 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/const_op.cc; 47s local, remote-cache ... (112 actions running) [19,496 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/const_op.cc; 48s local, remote-cache ... (112 actions running) [19,497 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/cholesky_op.cc; 49s local, remote-cache ... (111 actions, 110 running) [19,504 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/cholesky_op.cc; 50s local, remote-cache ... (111 actions, 110 running) [19,510 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/binary_ops.cc; 46s local, remote-cache ... (112 actions running) [19,512 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/binary_ops.cc; 47s local, remote-cache ... (112 actions running) [19,513 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/binary_ops.cc; 48s local, remote-cache ... (112 actions running) [19,516 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/binary_ops.cc; 49s local, remote-cache ... (112 actions running) [19,519 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/binary_ops.cc; 50s local, remote-cache ... (112 actions running) [19,528 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/bcast_ops.cc; 47s local, remote-cache ... (112 actions running) [19,531 / 19,692] Compiling tensorflow/compiler/tf2xla/kernels/bcast_ops.cc; 48s local, remote-cache ... (110 actions running) [19,538 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 45s local, remote-cache ... (104 actions running) [19,546 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 47s local, remote-cache ... (95 actions, 94 running) [19,548 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 48s local, remote-cache ... (92 actions running) [19,548 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 49s local, remote-cache ... (92 actions running) [19,549 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 51s local, remote-cache ... (90 actions running) [19,550 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 52s local, remote-cache ... (90 actions running) [19,557 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 53s local, remote-cache ... (82 actions running) [19,562 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 54s local, remote-cache ... (77 actions, 76 running) [19,574 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 55s local, remote-cache ... (65 actions, 64 running) [19,579 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 57s local, remote-cache ... (59 actions, 58 running) [19,582 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 58s local, remote-cache ... (56 actions running) [19,589 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 60s local, remote-cache ... (49 actions, 48 running) [19,592 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 61s local, remote-cache ... (45 actions, 44 running) [19,596 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 62s local, remote-cache ... (40 actions running) [19,598 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 63s local, remote-cache ... (38 actions running) [19,602 / 19,692] Compiling tensorflow/compiler/jit/xla_compile_on_demand_op.cc; 65s local, remote-cache ... (33 actions, 32 running) [19,604 / 19,692] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 63s local, remote-cache ... (30 actions running) [19,605 / 19,692] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 64s local, remote-cache ... (28 actions running) [19,610 / 19,692] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 66s local, remote-cache ... (21 actions, 20 running) [19,617 / 19,692] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 67s local, remote-cache ... (14 actions running) [19,620 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 59s local, remote-cache ... (11 actions, 10 running) [19,621 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 60s local, remote-cache ... (10 actions running) [19,623 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 61s local, remote-cache ... (8 actions running) [19,623 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 71s local, remote-cache ... (8 actions running) [19,624 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 72s local, remote-cache ... (7 actions, 6 running) [19,625 / 19,692] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 74s local, remote-cache ... (6 actions running) [19,626 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 61s local, remote-cache ... (5 actions, 4 running) [19,627 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 63s local, remote-cache ... (4 actions running) [19,627 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 73s local, remote-cache ... (4 actions running) [19,628 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 87s local, remote-cache ... (3 actions, 2 running) [19,629 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 89s local, remote-cache [19,629 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 99s local, remote-cache [19,629 / 19,692] Compiling tensorflow/compiler/mlir/python/mlir.cc; 129s local, remote-cache [19,630 / 19,692] [Prepa] Linking tensorflow/compiler/mlir/python/libmlir.pic.lo [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 0s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 10s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 40s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 101s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 161s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 221s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 281s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 341s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 401s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 461s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 521s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 581s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 641s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 701s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 761s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 801s local, remote-cache [19,631 / 19,692] Linking tensorflow/libtensorflow_cc.so.2.17.0; 808s local, remote-cache [19,635 / 19,692] Linking tensorflow/python/_pywrap_tensorflow_internal.so; 0s local, remote-cache [19,636 / 19,692] [Prepa] action 'SolibSymlink _solib_local/_Utensorflow_Spython/_pywrap_tensorflow_internal.so' [19,685 / 19,692] Linking tensorflow/python/_pywrap_tfcompile.so; 1s local, remote-cache ... (12 actions running) [19,689 / 19,692] Linking tensorflow/python/tpu/_pywrap_tpu_embedding.so; 2s local, remote-cache ... (4 actions running) [19,690 / 19,692] Linking tensorflow/python/tpu/_pywrap_tpu_embedding.so; 4s local, remote-cache [19,691 / 19,692] Action tensorflow/tools/pip_package/wheel_house; 0s local, remote-cache [19,691 / 19,692] Action tensorflow/tools/pip_package/wheel_house; 10s local, remote-cache ERROR: /tf/tensorflow/tensorflow/tools/pip_package/BUILD:265:9: Action tensorflow/tools/pip_package/wheel_house failed: (Exit 1): build_pip_package_py failed: error executing command (from target //tensorflow/tools/pip_package:wheel) bazel-out/aarch64-opt-exec-50AE0418/bin/tensorflow/tools/pip_package/build_pip_package_py @bazel-out/aarch64-opt/bin/tensorflow/tools/pip_package/wheel_house-0.params patchelf: getting info about '/tmp/tensorflow_wheel3j1pofb4/tensorflow/compiler/mlir/quantization/tensorflow/calibrator/pywrap_calibration.so': No such file or directory Traceback (most recent call last): File "/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt-exec-50AE0418/bin/tensorflow/tools/pip_package/build_pip_package_py.runfiles/org_tensorflow/tensorflow/tools/pip_package/build_pip_package.py", line 340, in <module> prepare_wheel_srcs(args.headers, args.srcs, args.xla_aot, File "/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt-exec-50AE0418/bin/tensorflow/tools/pip_package/build_pip_package_py.runfiles/org_tensorflow/tensorflow/tools/pip_package/build_pip_package.py", line 197, in prepare_wheel_srcs patch_so(srcs_dir) File "/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt-exec-50AE0418/bin/tensorflow/tools/pip_package/build_pip_package_py.runfiles/org_tensorflow/tensorflow/tools/pip_package/build_pip_package.py", line 239, in patch_so rpath = subprocess.check_output( File "/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/python_aarch64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/python_aarch64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['patchelf', '--print-rpath', '/tmp/tensorflow_wheel3j1pofb4/tensorflow/compiler/mlir/quantization/tensorflow/calibrator/pywrap_calibration.so']' returned non-zero exit status 1. Target //tensorflow/tools/pip_package:wheel failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 3818.825s, Critical Path: 1498.78s INFO: 19692 processes: 1223 internal, 18469 local. FAILED: Build did NOT complete successfully stderr_lines: <omitted> stdout: '' stdout_lines: <omitted> PLAY RECAP ********************************************************************* localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0 Tuesday 16 April 2024 20:15:26 +0100 (1:03:40.285) 1:05:21.938 ********* =============================================================================== tensorflow_in_container : (Python 3.9) build TensorFlow git ---------- 3820.29s Cleanup after previous build ------------------------------------------- 31.14s tensorflow_in_container : (Python 3.9) start build container ----------- 26.57s tensorflow : Fetch TensorFlow from git --------------------------------- 22.96s tensorflow : Remove TensorFlow git sources from previous build ---------- 4.04s Gathering Facts --------------------------------------------------------- 2.61s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.83s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.51s Create cache dirs ------------------------------------------------------- 1.43s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.25s tensorflow : Save git commit hash to file ------------------------------- 1.09s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.87s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.81s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.81s tensorflow : Create linklibs shell script ------------------------------- 0.69s tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.68s tensorflow_in_container : Reset checkout to clean state ----------------- 0.64s tensorflow : Use nightly package requirements --------------------------- 0.62s tensorflow : Get oneline version of git log for this checkout ----------- 0.59s Build TensorFlow for each Python version (dual abi) --------------------- 0.41s Build step 'Execute shell' marked build as failure

6 months, 2 weeks

Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #772

by ci_notify＠linaro.org

See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/7…> Changes: ------------------------------------------ [...truncated 6.52 KiB...] TASK [tensorflow : Set world write access to source] *************************** Tuesday 16 April 2024 01:11:25 +0100 (0:00:00.030) 0:01:21.157 ********* changed: [localhost] TASK [tensorflow : Get oneline version of git log for this checkout] *********** Tuesday 16 April 2024 01:11:26 +0100 (0:00:00.385) 0:01:21.542 ********* changed: [localhost] TASK [tensorflow : Show git log] *********************************************** Tuesday 16 April 2024 01:11:26 +0100 (0:00:00.559) 0:01:22.101 ********* ok: [localhost] => git_log.stdout: 'e9d460922aa8d54b2f48a9b2dff47085726c89e5 Merge pull request #65568 from Intel-tensorflow:mraunak/win_config' TASK [tensorflow : Record git commit hash to variable] ************************* Tuesday 16 April 2024 01:11:26 +0100 (0:00:00.037) 0:01:22.139 ********* ok: [localhost] TASK [tensorflow : Save git commit hash to file] ******************************* Tuesday 16 April 2024 01:11:26 +0100 (0:00:00.057) 0:01:22.196 ********* changed: [localhost] TASK [Build TensorFlow for each Python version (dual abi)] ********************* Tuesday 16 April 2024 01:11:28 +0100 (0:00:01.113) 0:01:23.310 ********* TASK [tensorflow_in_container : Reset checkout to clean state] ***************** Tuesday 16 April 2024 01:11:28 +0100 (0:00:00.417) 0:01:23.728 ********* changed: [localhost] TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] *** Tuesday 16 April 2024 01:11:29 +0100 (0:00:00.827) 0:01:24.556 ********* skipping: [localhost] TASK [Fetch TensorFlow] ******************************************************** Tuesday 16 April 2024 01:11:29 +0100 (0:00:00.029) 0:01:24.586 ********* skipping: [localhost] TASK [(Python 3.9) patch TensorFlow tree] ************************************** Tuesday 16 April 2024 01:11:29 +0100 (0:00:00.031) 0:01:24.617 ********* TASK [tensorflow : Create linklibs shell script] ******************************* Tuesday 16 April 2024 01:11:29 +0100 (0:00:00.082) 0:01:24.700 ********* changed: [localhost] TASK [tensorflow : Use nightly package requirements] *************************** Tuesday 16 April 2024 01:11:30 +0100 (0:00:00.695) 0:01:25.396 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] *** Tuesday 16 April 2024 01:11:30 +0100 (0:00:00.627) 0:01:26.023 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] *** Tuesday 16 April 2024 01:11:31 +0100 (0:00:00.685) 0:01:26.708 ********* ok: [localhost] TASK [tensorflow_in_container : (Python 3.9) start build container] ************ Tuesday 16 April 2024 01:11:32 +0100 (0:00:01.552) 0:01:28.260 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] *** Tuesday 16 April 2024 01:11:41 +0100 (0:00:08.472) 0:01:36.733 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ****** Tuesday 16 April 2024 01:11:42 +0100 (0:00:01.232) 0:01:37.965 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] *** Tuesday 16 April 2024 01:11:43 +0100 (0:00:00.843) 0:01:38.809 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ******** Tuesday 16 April 2024 01:11:44 +0100 (0:00:00.873) 0:01:39.682 ********* changed: [localhost] TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] *** Tuesday 16 April 2024 01:11:45 +0100 (0:00:00.840) 0:01:40.523 ********* changed: [localhost] TASK [tensorflow_in_container : Show list of installed Python packages for build] *** Tuesday 16 April 2024 01:11:47 +0100 (0:00:02.642) 0:01:43.165 ********* ok: [localhost] => pip_list_output.stdout: |- Package Version Location Installer ------------------- -------------------- -------------------------------------- --------- auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip certifi 2019.11.28 /usr/lib/python3/dist-packages cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip chardet 3.0.4 /usr/lib/python3/dist-packages cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip dbus-python 1.2.16 /usr/lib/python3/dist-packages docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip idna 2.8 /usr/lib/python3/dist-packages importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip pip 24.0 /usr/local/lib/python3.9/dist-packages pip pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip PyGObject 3.36.0 /usr/lib/python3/dist-packages python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip requests 2.22.0 /usr/lib/python3/dist-packages requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip six 1.14.0 /usr/lib/python3/dist-packages twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] *** Tuesday 16 April 2024 01:11:47 +0100 (0:00:00.038) 0:01:43.204 ********* fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Invocation ID: 87b5868d-81e1-4350-8962-9af9210f24f3 INFO: Reading 'startup' options from /tf/tensorflow/.bazelrc: --windows_enable_symlinks INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility INFO: Reading rc options for 'build' from /usertools/aarch64_clang.bazelrc: 'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/local/bin/python3 --python_path=/usr/local/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --linkopt=-fuse-ld=lld --linkopt=-lm --linkopt=-Wl,--undefined-version --host_linkopt=-fuse-ld=lld --host_linkopt=-lm --host_linkopt=-Wl,--undefined-version --distinct_host_configuration=false --copt=-Wno-gnu-offsetof-extensions --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_clang_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3 INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64_clang.bazelrc: --disk_cache=/tf/cache INFO: Found applicable config definition build:mkl_aarch64_threadpool in file /tf/tensorflow/.bazelrc: --define=build_with_mkl_aarch64=true -c opt INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: Loading: 0 packages loaded Analyzing: target //tensorflow/tools/pip_package:wheel (1 packages loaded, 0 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (11 packages loaded, 12 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (157 packages loaded, 302 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (270 packages loaded, 4172 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (270 packages loaded, 4172 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (270 packages loaded, 4172 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (271 packages loaded, 4197 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (271 packages loaded, 4197 targets configured) Analyzing: target //tensorflow/tools/pip_package:wheel (391 packages loaded, 5027 targets configured) INFO: Repository dlpack instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:931:28: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:67:11: in _initialize_third_party /tf/tensorflow/third_party/dlpack/workspace.bzl:6:20: in repo /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository pthreadpool instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:166:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository mkl_dnn_acl_compatible instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:213:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository riegeli instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:868:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository cpuinfo instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:173:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: Repository XNNPACK instantiated at: /tf/tensorflow/WORKSPACE:95:14: in <toplevel> /tf/tensorflow/tensorflow/workspace2.bzl:938:21: in workspace /tf/tensorflow/tensorflow/workspace2.bzl:151:20: in _tf_repositories /tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive Repository rule _tf_http_archive defined at: /tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel> INFO: repository @riegeli' used the following cache hits instead of downloading the corresponding file. * Hash '870ca080cdfc5eba696a72ccc3a54cbf0f2271befc0d459eafa8f065edfaadb2' for https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/rieg… If the definition of 'repository @riegeli' was updated, verify that the hashes were also updated. ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/tf_runtime/BUILD:204:16: no such package '@local_tsl//tsl/concurrency': BUILD file not found in directory 'tsl/concurrency' of external repository @local_tsl. Add a BUILD file to a directory to mark it as a package. and referenced by '@tf_runtime//:async_value' ERROR: Analysis of target '//tensorflow/tools/pip_package:wheel' failed; build aborted: INFO: Elapsed time: 71.961s INFO: 0 processes. FAILED: Build did NOT complete successfully (589 packages loaded, 20499 targets configured) stderr_lines: <omitted> stdout: '' stdout_lines: <omitted> PLAY RECAP ********************************************************************* localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0 Tuesday 16 April 2024 01:13:00 +0100 (0:01:12.894) 0:02:56.098 ********* =============================================================================== tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git -- 72.89s Cleanup after previous build ------------------------------------------- 51.14s tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 21.57s tensorflow_in_container : (Python 3.9) start build container ------------ 8.47s tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.83s tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.64s Gathering Facts --------------------------------------------------------- 1.69s tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.55s Create cache dirs ------------------------------------------------------- 1.44s tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.23s tensorflow : Save git commit hash to file ------------------------------- 1.11s tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.87s tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.84s tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.84s tensorflow_in_container : Reset checkout to clean state ----------------- 0.83s tensorflow : Create linklibs shell script ------------------------------- 0.70s tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.69s tensorflow : Use nightly package requirements --------------------------- 0.63s tensorflow : Get oneline version of git log for this checkout ----------- 0.56s Build TensorFlow for each Python version (dual abi) --------------------- 0.42s Build step 'Execute shell' marked build as failure

6 months, 2 weeks

2024

2023

2022

2021

ldcg-ai-ml April 2024