Jenkins build is back to normal : Tensorflow oneDNN Python package (manylinux2014 nightly) #868
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
3 days, 21 hours
- 1
- 0
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #866
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 3.15 KiB...]
skipping: [localhost] => (item={'key': '2.15', 'value': {'tensorflow_version': '2.15.1', 'bazel_version': 'bazelisk', 'old_pip_package': True, 'image_prefix': '2.15', 'run_unit_tests': True, 'run_serial_unit_tests': True, 'onednn_ut_exclude': ['//tensorflow/core/grappler/optimizers:auto_mixed_precision_test', '//tensorflow/core/grappler/optimizers:remapper_test'], 'unit_tests_exclude': ['//tensorflow/lite/...', '//tensorflow/tools/api/tests:api_compatibility_test'], 'unit_tests_gcc_exclude': ['//tensorflow/compiler/mlir/lite/debug:debug_test'], 'unit_tests_clang_exclude': ['//tensorflow/compiler/mlir/lite/quantization/lite:quantize_model_test', '//tensorflow/compiler/mlir/lite/quantization/lite:quantize_weights_test', '//tensorflow/compiler/mlir/lite/sparsity:sparsify_model_test', '//tensorflow/core/kernels/image:resize_bicubic_op_test'], 'serial_unit_tests_exclude': '', 'unit_tests_filter_flags': 'NOT_USED', 'python_versions': ['3.9', '3.10', '3.11']}})
TASK [tensorflow : Set TensorFlow and other package versions] ******************
Thursday 18 July 2024 01:11:07 +0100 (0:00:00.116) 0:01:00.595 *********
ok: [localhost]
TASK [tensorflow : Set build dir for TensorFlow-oneDNN-ACL git] ****************
Thursday 18 July 2024 01:11:07 +0100 (0:00:00.069) 0:01:00.665 *********
ok: [localhost]
TASK [tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build] ***
Thursday 18 July 2024 01:11:07 +0100 (0:00:00.039) 0:01:00.705 *********
changed: [localhost]
TASK [tensorflow : Fetch TensorFlow] *******************************************
Thursday 18 July 2024 01:11:14 +0100 (0:00:07.242) 0:01:07.948 *********
included: /home/buildslave/workspace/ldcg-python-manylinux-tensorflow-onednn-nightly/configs/ldcg-python-manylinux-tensorflow/ansible/roles/tensorflow/tasks/fetch.yml for localhost
TASK [tensorflow : Fetch TensorFlow-oneDNN-ACL git source] *********************
Thursday 18 July 2024 01:11:14 +0100 (0:00:00.079) 0:01:08.028 *********
skipping: [localhost]
TASK [tensorflow : Unpack TensorFlow-oneDNN-ACL git source] ********************
Thursday 18 July 2024 01:11:14 +0100 (0:00:00.033) 0:01:08.062 *********
skipping: [localhost]
TASK [tensorflow : Rename source dir for oneDNN build] *************************
Thursday 18 July 2024 01:11:14 +0100 (0:00:00.031) 0:01:08.093 *********
skipping: [localhost]
TASK [tensorflow : Fetch TensorFlow-oneDNN-ACL from git] ***********************
Thursday 18 July 2024 01:11:14 +0100 (0:00:00.034) 0:01:08.127 *********
changed: [localhost]
TASK [tensorflow : Fetch TensorFlow-oneDNN-ACL from git at commit] *************
Thursday 18 July 2024 01:11:38 +0100 (0:00:23.593) 0:01:31.721 *********
skipping: [localhost]
TASK [tensorflow : Set world write access to source] ***************************
Thursday 18 July 2024 01:11:38 +0100 (0:00:00.030) 0:01:31.751 *********
changed: [localhost]
TASK [tensorflow : Get oneline version of git log for this checkout] ***********
Thursday 18 July 2024 01:11:38 +0100 (0:00:00.371) 0:01:32.122 *********
changed: [localhost]
TASK [tensorflow : Show git log] ***********************************************
Thursday 18 July 2024 01:11:39 +0100 (0:00:00.555) 0:01:32.678 *********
ok: [localhost] =>
git_log.stdout: e359cc6b2a0ac0a7bed6cdefa28216c6e8a6aa96 [xla:ffi] Don't forget to register handlers with canonical platform name
TASK [tensorflow : Record git commit hash to variable] *************************
Thursday 18 July 2024 01:11:39 +0100 (0:00:00.036) 0:01:32.715 *********
ok: [localhost]
TASK [tensorflow : Save git commit hash to file] *******************************
Thursday 18 July 2024 01:11:39 +0100 (0:00:00.055) 0:01:32.770 *********
changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] *********************
Thursday 18 July 2024 01:11:40 +0100 (0:00:01.103) 0:01:33.874 *********
TASK [tensorflow_in_container : Reset checkout to clean state] *****************
Thursday 18 July 2024 01:11:40 +0100 (0:00:00.443) 0:01:34.317 *********
changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] ***
Thursday 18 July 2024 01:11:41 +0100 (0:00:00.649) 0:01:34.967 *********
skipping: [localhost]
TASK [Fetch TensorFlow] ********************************************************
Thursday 18 July 2024 01:11:41 +0100 (0:00:00.029) 0:01:34.996 *********
skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] **************************************
Thursday 18 July 2024 01:11:41 +0100 (0:00:00.030) 0:01:35.027 *********
TASK [tensorflow : Create linklibs shell script] *******************************
Thursday 18 July 2024 01:11:41 +0100 (0:00:00.082) 0:01:35.109 *********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Thursday 18 July 2024 01:11:42 +0100 (0:00:00.693) 0:01:35.802 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Thursday 18 July 2024 01:11:42 +0100 (0:00:00.626) 0:01:36.429 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Thursday 18 July 2024 01:11:43 +0100 (0:00:00.684) 0:01:37.113 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Thursday 18 July 2024 01:11:45 +0100 (0:00:02.095) 0:01:39.209 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Thursday 18 July 2024 01:11:51 +0100 (0:00:06.161) 0:01:45.370 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Thursday 18 July 2024 01:11:53 +0100 (0:00:01.246) 0:01:46.617 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Thursday 18 July 2024 01:11:53 +0100 (0:00:00.868) 0:01:47.486 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Thursday 18 July 2024 01:11:54 +0100 (0:00:00.854) 0:01:48.341 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Thursday 18 July 2024 01:11:55 +0100 (0:00:00.853) 0:01:49.194 *********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Thursday 18 July 2024 01:11:57 +0100 (0:00:01.482) 0:01:50.677 *********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
backports.tarfile 1.2.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.8 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.10 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.21.2 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib_metadata 8.0.0 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.4.0 /usr/local/lib/python3.9/dist-packages pip
jaraco.context 5.3.0 /usr/local/lib/python3.9/dist-packages pip
jaraco.functools 4.0.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 25.2.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.3.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.18 /usr/local/lib/python3.9/dist-packages pip
pip 24.1.2 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.11.1 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 6.0.0 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.22 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.31 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.18.0 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 44.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 45.2.0 /usr/lib/python3/dist-packages
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.2 /usr/local/lib/python3.9/dist-packages pip
wheel 0.43.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.19.2 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] ***
Thursday 18 July 2024 01:11:57 +0100 (0:00:00.037) 0:01:50.715 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] *************
Thursday 18 July 2024 02:47:42 +0100 (1:35:45.330) 1:37:36.045 *********
skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] *****************
Thursday 18 July 2024 02:47:42 +0100 (0:00:00.033) 1:37:36.079 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] ***
Thursday 18 July 2024 02:47:43 +0100 (0:00:00.971) 1:37:37.050 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] *******
Thursday 18 July 2024 02:47:44 +0100 (0:00:01.123) 1:37:38.174 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] ***
Thursday 18 July 2024 02:47:45 +0100 (0:00:00.751) 1:37:38.926 *********
FATAL: command execution failed
java.io.EOFException
at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
at java.base/java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.base/java.io.ObjectInputStream.<init>(Unknown Source)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:50)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused: java.io.IOException: Backing channel 'ldcg-aarch64-02' is disconnected.
at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at jdk.proxy2/jdk.proxy2.$Proxy89.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1212)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1204)
at hudson.tasks.CommandInterpreter.join(CommandInterpreter.java:195)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:145)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1895)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
FATAL: Unable to delete script file /tmp/jenkins10562564789596221331.sh
java.io.EOFException
at java.base/java.io.ObjectInputStream$PeekInputStream.readFully(Unknown Source)
at java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(Unknown Source)
at java.base/java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.base/java.io.ObjectInputStream.<init>(Unknown Source)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:50)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@3d426df8:ldcg-aarch64-02": Remote call on ldcg-aarch64-02 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:996)
at hudson.FilePath.act(FilePath.java:1230)
at hudson.FilePath.act(FilePath.java:1219)
at hudson.FilePath.delete(FilePath.java:1766)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:163)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:92)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1895)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Build step 'Execute shell' marked build as failure
1 week, 2 days
- 1
- 0
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #858
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 6.96 KiB...]
ok: [localhost] =>
git_log.stdout: 0aa90c2e29b9f1e374b6da9ee15e109eba5f7561 [xla:cpu] Optimize CopyThunk for small shapes
TASK [tensorflow : Record git commit hash to variable] *************************
Thursday 11 July 2024 01:11:07 +0100 (0:00:00.036) 0:01:02.788 *********
ok: [localhost]
TASK [tensorflow : Save git commit hash to file] *******************************
Thursday 11 July 2024 01:11:07 +0100 (0:00:00.055) 0:01:02.844 *********
changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] *********************
Thursday 11 July 2024 01:11:08 +0100 (0:00:01.143) 0:01:03.988 *********
TASK [tensorflow_in_container : Reset checkout to clean state] *****************
Thursday 11 July 2024 01:11:09 +0100 (0:00:00.438) 0:01:04.427 *********
changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] ***
Thursday 11 July 2024 01:11:09 +0100 (0:00:00.655) 0:01:05.082 *********
skipping: [localhost]
TASK [Fetch TensorFlow] ********************************************************
Thursday 11 July 2024 01:11:09 +0100 (0:00:00.030) 0:01:05.113 *********
skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] **************************************
Thursday 11 July 2024 01:11:09 +0100 (0:00:00.031) 0:01:05.144 *********
TASK [tensorflow : Create linklibs shell script] *******************************
Thursday 11 July 2024 01:11:09 +0100 (0:00:00.080) 0:01:05.224 *********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Thursday 11 July 2024 01:11:10 +0100 (0:00:00.688) 0:01:05.913 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Thursday 11 July 2024 01:11:11 +0100 (0:00:00.622) 0:01:06.535 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Thursday 11 July 2024 01:11:11 +0100 (0:00:00.688) 0:01:07.224 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Thursday 11 July 2024 01:11:13 +0100 (0:00:01.483) 0:01:08.707 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Thursday 11 July 2024 01:11:51 +0100 (0:00:38.056) 0:01:46.764 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Thursday 11 July 2024 01:11:52 +0100 (0:00:01.235) 0:01:47.999 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Thursday 11 July 2024 01:11:53 +0100 (0:00:00.795) 0:01:48.795 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Thursday 11 July 2024 01:11:54 +0100 (0:00:00.790) 0:01:49.586 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Thursday 11 July 2024 01:11:55 +0100 (0:00:00.866) 0:01:50.452 *********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Thursday 11 July 2024 01:11:56 +0100 (0:00:01.854) 0:01:52.307 *********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip
pip 24.0 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip
wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] ***
Thursday 11 July 2024 01:11:56 +0100 (0:00:00.038) 0:01:52.345 *********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: fc992184-8a7b-4426-ad26-edbf32fdb4dd
INFO: Reading 'startup' options from /tf/tensorflow/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc:
'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /usertools/aarch64_clang.bazelrc:
'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/local/bin/python3 --python_path=/usr/local/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --linkopt=-fuse-ld=lld --linkopt=-lm --linkopt=-Wl,--undefined-version --host_linkopt=-fuse-ld=lld --host_linkopt=-lm --host_linkopt=-Wl,--undefined-version --distinct_host_configuration=false --copt=-Wno-gnu-offsetof-extensions --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_clang_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3
INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64_clang.bazelrc: --disk_cache=/tf/cache
INFO: Found applicable config definition build:mkl_aarch64_threadpool in file /tf/tensorflow/.bazelrc: --define=build_with_mkl_aarch64=true -c opt
INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
Loading:
Loading:
DEBUG: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/local_xla/third_party/py/python_repo.bzl:109:10: Using hermetic Python 3.9
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading: 0 packages loaded
Analyzing: target //tensorflow/tools/pip_package:wheel (1 packages loaded, 0 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (10 packages loaded, 12 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (157 packages loaded, 306 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (274 packages loaded, 4204 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (277 packages loaded, 4204 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (277 packages loaded, 4204 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (278 packages loaded, 4230 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (396 packages loaded, 5060 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (625 packages loaded, 22801 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (662 packages loaded, 30663 targets configured)
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/zlib.net/fossils/zlib-… failed: class java.io.FileNotFoundException GET returned 404 Not Found
Analyzing: target //tensorflow/tools/pip_package:wheel (665 packages loaded, 35604 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (681 packages loaded, 40809 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (682 packages loaded, 42554 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (691 packages loaded, 43201 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (692 packages loaded, 43711 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (692 packages loaded, 44119 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (693 packages loaded, 44550 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (695 packages loaded, 44677 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (695 packages loaded, 44677 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (697 packages loaded, 44800 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (697 packages loaded, 44800 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (698 packages loaded, 45564 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (700 packages loaded, 45983 targets configured)
Analyzing: target //tensorflow/tools/pip_package:wheel (700 packages loaded, 45983 targets configured)
INFO: Analyzed target //tensorflow/tools/pip_package:wheel (701 packages loaded, 46082 targets configured).
INFO: Found 1 target...
[0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (3 actions, 1 running)
ERROR: /tf/tensorflow/tensorflow/python/util/BUILD:170:27: Compiling tensorflow/python/util/tensor_float_32.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/util:_pywrap_tensor_float_32_execution.so) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/util/_objs/_pywrap_tensor_float_32_execution.so/tensor_float_32.pic.d ... (remaining 82 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/com_google_absl/absl/time/internal/cctz/BUILD.bazel:21:11: Compiling absl/time/internal/cctz/src/civil_time_detail.cc failed: (Exit 1): clang failed: error executing command (from target @com_google_absl//absl/time/internal/cctz:civil_time) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/com_google_absl/absl/time/internal/cctz/_objs/civil_time/civil_time_detail.pic.d ... (remaining 46 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/zlib/BUILD.bazel:5:11: Compiling zutil.c failed: (Exit 1): clang failed: error executing command (from target @zlib//:zlib) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/zlib/_objs/zlib/zutil.pic.d '-frandom-seed=bazel-out/aarch64-opt/bin/external/zlib/_objs/zlib/zutil.pic.o' ... (remaining 49 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/zlib/BUILD.bazel:5:11: Compiling zutil.c [for tool] failed: (Exit 1): clang failed: error executing command (from target @zlib//:zlib) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt-exec-50AE0418/bin/external/zlib/_objs/zlib/zutil.d '-frandom-seed=bazel-out/aarch64-opt-exec-50AE0418/bin/external/zlib/_objs/zlib/zutil.o' ... (remaining 35 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/org_brotli/BUILD:121:11: Compiling c/enc/utf8_util.c failed: (Exit 1): clang failed: error executing command (from target @org_brotli//:brotlienc) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/org_brotli/_objs/brotlienc/utf8_util.pic.d ... (remaining 56 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/external/org_brotli/BUILD:121:11: Compiling c/enc/static_dict.c failed: (Exit 1): clang failed: error executing command (from target @org_brotli//:brotlienc) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/external/org_brotli/_objs/brotlienc/static_dict.pic.d ... (remaining 56 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /tf/tensorflow/tensorflow/python/util/BUILD:185:27: Compiling tensorflow/python/util/determinism.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/util:_pywrap_determinism.so) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/util/_objs/_pywrap_determinism.so/determinism.pic.d ... (remaining 78 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /tf/tensorflow/tensorflow/python/lib/core/BUILD:193:11: Compiling tensorflow/python/lib/core/safe_pyobject_ptr.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/python/lib/core:safe_pyobject_ptr) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/python/lib/core/_objs/safe_pyobject_ptr/safe_pyobject_ptr.pic.d ... (remaining 62 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
ERROR: /tf/tensorflow/tensorflow/core/common_runtime/BUILD:1828:11: Compiling tensorflow/core/common_runtime/local_session_selection.cc failed: (Exit 1): clang failed: error executing command (from target //tensorflow/core/common_runtime:local_session_selection) /usr/lib/llvm-18/bin/clang -MD -MF bazel-out/aarch64-opt/bin/tensorflow/core/common_runtime/_objs/local_session_selection/local_session_selection.pic.d ... (remaining 61 arguments skipped)
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/lib/llvm-18/bin/clang, ...)": No such file or directory
Target //tensorflow/tools/pip_package:wheel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 410.122s, Critical Path: 0.61s
INFO: 209 processes: 173 internal, 36 local.
FAILED: Build did NOT complete successfully
stderr_lines: <omitted>
stdout: ''
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0
Thursday 11 July 2024 01:18:48 +0100 (0:06:51.082) 0:08:43.428 *********
===============================================================================
tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git - 411.08s
tensorflow_in_container : (Python 3.9) start build container ----------- 38.06s
Cleanup after previous build ------------------------------------------- 32.73s
tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 21.54s
tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.00s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.85s
Gathering Facts --------------------------------------------------------- 1.63s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.48s
Create cache dirs ------------------------------------------------------- 1.44s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.24s
tensorflow : Save git commit hash to file ------------------------------- 1.14s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.87s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.80s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.79s
tensorflow : Create linklibs shell script ------------------------------- 0.69s
tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.69s
tensorflow_in_container : Reset checkout to clean state ----------------- 0.66s
tensorflow : Use nightly package requirements --------------------------- 0.62s
tensorflow : Get oneline version of git log for this checkout ----------- 0.58s
Build TensorFlow for each Python version (dual abi) --------------------- 0.44s
Build step 'Execute shell' marked build as failure
2 weeks
- 1
- 2
Build failed in Jenkins: Tensorflow Python package (manylinux2014 nightly) #1226
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/1226/dis…>
Changes:
------------------------------------------
[...truncated 217.73 KiB...]
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:532: note: this is the location of the previous definition
532 | #define DCHECK_NE(x, y) _TF_DCHECK_NOP(x, y)
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:30,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/check.h:144: warning: "DCHECK_LE" redefined
144 | #define DCHECK_LE(val1, val2) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:533: note: this is the location of the previous definition
533 | #define DCHECK_LE(x, y) _TF_DCHECK_NOP(x, y)
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:30,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/check.h:146: warning: "DCHECK_LT" redefined
146 | #define DCHECK_LT(val1, val2) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:534: note: this is the location of the previous definition
534 | #define DCHECK_LT(x, y) _TF_DCHECK_NOP(x, y)
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:30,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/check.h:148: warning: "DCHECK_GE" redefined
148 | #define DCHECK_GE(val1, val2) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:535: note: this is the location of the previous definition
535 | #define DCHECK_GE(x, y) _TF_DCHECK_NOP(x, y)
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:30,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/check.h:150: warning: "DCHECK_GT" redefined
150 | #define DCHECK_GT(val1, val2) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:536: note: this is the location of the previous definition
536 | #define DCHECK_GT(x, y) _TF_DCHECK_NOP(x, y)
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:31,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/log.h:199: warning: "LOG" redefined
199 | #define LOG(severity) ABSL_LOG_INTERNAL_LOG_IMPL(_##severity)
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:165: note: this is the location of the previous definition
165 | #define LOG(severity) _TF_LOG_##severity
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:31,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/log.h:237: warning: "LOG_EVERY_N" redefined
237 | #define LOG_EVERY_N(severity, n) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:278: note: this is the location of the previous definition
278 | #define LOG_EVERY_N(severity, n) \
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:31,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/log.h:245: warning: "LOG_FIRST_N" redefined
245 | #define LOG_FIRST_N(severity, n) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:284: note: this is the location of the previous definition
284 | #define LOG_FIRST_N(severity, n) \
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:31,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/log.h:253: warning: "LOG_EVERY_POW_2" redefined
253 | #define LOG_EVERY_POW_2(severity) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:290: note: this is the location of the previous definition
290 | #define LOG_EVERY_POW_2(severity) \
|
In file included from external/local_xla/xla/hlo/ir/hlo_computation.h:31,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/com_google_absl/absl/log/log.h:265: warning: "LOG_EVERY_N_SEC" redefined
265 | #define LOG_EVERY_N_SEC(severity, n_seconds) \
|
In file included from external/local_tsl/tsl/platform/logging.h:26,
from external/local_xla/xla/ffi/execution_context.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk.h:35,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_tsl/tsl/platform/default/logging.h:300: note: this is the location of the previous definition
300 | #define LOG_EVERY_N_SEC(severity, n_seconds) \
|
In file included from external/local_xla/xla/shape.h:28,
from external/local_xla/xla/index_util.h:24,
from external/local_xla/xla/literal.h:45,
from external/local_xla/xla/hlo/ir/dfs_hlo_visitor.h:27,
from external/local_xla/xla/hlo/ir/hlo_computation.h:36,
from external/local_xla/xla/service/buffer_assignment.h:36,
from external/local_xla/xla/runtime/buffer_use.h:21,
from external/local_xla/xla/service/cpu/runtime/thunk.h:36,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:34,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_xla/xla/layout.h:464:18: warning: 'xla::Layout::DimInfo::dim_level_type' is too small to hold all values of 'enum xla::DimLevelType'
464 | DimLevelType dim_level_type : 6;
| ^~~~~~~~~~~~~~
external/local_xla/xla/layout.h:476:17: warning: 'xla::Layout::index_primitive_type_' is too small to hold all values of 'enum xla::PrimitiveType'
476 | PrimitiveType index_primitive_type_ : 8;
| ^~~~~~~~~~~~~~~~~~~~~
external/local_xla/xla/layout.h:477:17: warning: 'xla::Layout::pointer_primitive_type_' is too small to hold all values of 'enum xla::PrimitiveType'
477 | PrimitiveType pointer_primitive_type_ : 8;
| ^~~~~~~~~~~~~~~~~~~~~~~
In file included from /dt10/usr/include/unistd.h:226,
from external/com_google_absl/absl/base/internal/thread_identity.h:27,
from external/com_google_absl/absl/synchronization/mutex.h:70,
from external/com_google_absl/absl/strings/internal/cordz_info.h:31,
from external/com_google_absl/absl/strings/cord.h:91,
from external/com_google_absl/absl/status/internal/status_internal.h:23,
from external/com_google_absl/absl/status/status.h:59,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.h:30,
from external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:16:
external/local_xla/xla/tsl/concurrency/async_value.h: In instantiation of 'static void tsl::internal::ConcreteAsyncValue<T>::VerifyOffsets() [with T = tsl::DummyValueForErrorAsyncValue]':
external/local_xla/xla/tsl/concurrency/async_value.h:547:18: required from 'tsl::internal::ConcreteAsyncValue<T>::ConcreteAsyncValue(absl::lts_20230802::Status) [with T = tsl::DummyValueForErrorAsyncValue]'
external/local_xla/xla/tsl/concurrency/async_value.h:738:30: required from here
external/local_xla/xla/tsl/concurrency/async_value.h:717:28: warning: 'offsetof' within non-standard-layout type 'tsl::internal::ConcreteAsyncValue<tsl::DummyValueForErrorAsyncValue>' is conditionally-supported [-Winvalid-offsetof]
717 | static_assert(offsetof(ConcreteAsyncValue<T>, data_store_.data_) ==
| ^
external/local_xla/xla/tsl/concurrency/async_value.h: In instantiation of 'static void tsl::internal::ConcreteAsyncValue<T>::VerifyOffsets() [with T = tsl::Chain]':
external/local_xla/xla/tsl/concurrency/async_value.h:556:18: required from 'tsl::internal::ConcreteAsyncValue<T>::ConcreteAsyncValue(tsl::internal::ConcreteAsyncValue<T>::ConstructedPayload, Args&& ...) [with Args = {}; T = tsl::Chain]'
external/local_xla/xla/tsl/concurrency/async_value_ref.h:867:10: required from 'T* tsl::internal::PlacementConstruct(void*, Args&& ...) [with T = tsl::internal::ConcreteAsyncValue<tsl::Chain>; Args = {tsl::internal::ConcreteAsyncValue<tsl::Chain>::ConstructedPayload}]'
external/local_xla/xla/tsl/concurrency/async_value_ref.h:873:40: required from 'T* tsl::internal::AllocateAndConstruct(Args&& ...) [with T = tsl::internal::ConcreteAsyncValue<tsl::Chain>; Args = {tsl::internal::ConcreteAsyncValue<tsl::Chain>::ConstructedPayload}]'
external/local_xla/xla/tsl/concurrency/async_value_ref.h:900:70: required from 'tsl::AsyncValueRef<T> tsl::MakeConstructedAsyncValueRef(Args&& ...) [with T = tsl::Chain; Args = {}]'
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:120:69: required from here
external/local_xla/xla/tsl/concurrency/async_value.h:717:28: warning: 'offsetof' within non-standard-layout type 'tsl::internal::ConcreteAsyncValue<tsl::Chain>' is conditionally-supported [-Winvalid-offsetof]
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:295:35: warning: 'always_inline' function might not be inlinable [-Wattributes]
295 | void ABSL_ATTRIBUTE_ALWAYS_INLINE ThunkExecutor::SplitReadyQueue(
| ^~~~~~~~~~~~~
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc: In member function 'void xla::cpu::ThunkExecutor::Execute(xla::cpu::ThunkExecutor::ExecuteState*, const xla::cpu::Thunk::ExecuteParams&, xla::cpu::ThunkExecutor::ReadyQueue, xla::cpu::Thunk::ExecuteSession::Lock)':
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:295:35: error: inlining failed in call to 'always_inline' 'void xla::cpu::ThunkExecutor::SplitReadyQueue(xla::cpu::ThunkExecutor::ExecuteState*, const xla::cpu::Thunk::ExecuteParams&, int64_t, xla::cpu::ThunkExecutor::ReadyQueue&)': function body can be overwritten at link time
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:254:22: note: called from here
254 | SplitReadyQueue(state, params, /*start_index=*/i + 1, ready_queue);
| ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:295:35: error: inlining failed in call to 'always_inline' 'void xla::cpu::ThunkExecutor::SplitReadyQueue(xla::cpu::ThunkExecutor::ExecuteState*, const xla::cpu::Thunk::ExecuteParams&, int64_t, xla::cpu::ThunkExecutor::ReadyQueue&)': function body can be overwritten at link time
295 | void ABSL_ATTRIBUTE_ALWAYS_INLINE ThunkExecutor::SplitReadyQueue(
| ^~~~~~~~~~~~~
external/local_xla/xla/service/cpu/runtime/thunk_executor.cc:254:22: note: called from here
254 | SplitReadyQueue(state, params, /*start_index=*/i + 1, ready_queue);
| ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Target //tensorflow/tools/pip_package:wheel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1939.210s, Critical Path: 479.60s
INFO: 16926 processes: 1264 internal, 15662 local.
FAILED: Build did NOT complete successfully
stderr_lines: <omitted>
stdout: ''
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0
Tuesday 09 July 2024 19:43:49 +0100 (0:32:20.445) 0:33:43.463 **********
===============================================================================
tensorflow_in_container : (Python 3.9) build TensorFlow git ---------- 1940.45s
Cleanup after previous build ------------------------------------------- 31.84s
tensorflow : Fetch TensorFlow from git --------------------------------- 23.86s
tensorflow_in_container : (Python 3.9) start build container ------------ 5.66s
tensorflow : Remove TensorFlow git sources from previous build ---------- 4.27s
Gathering Facts --------------------------------------------------------- 2.69s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.04s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.56s
Create cache dirs ------------------------------------------------------- 1.43s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.23s
tensorflow : Save git commit hash to file ------------------------------- 1.10s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.90s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.87s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.82s
tensorflow : Create linklibs shell script ------------------------------- 0.69s
tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.68s
tensorflow_in_container : Reset checkout to clean state ----------------- 0.65s
tensorflow : Use nightly package requirements --------------------------- 0.62s
tensorflow : Get oneline version of git log for this checkout ----------- 0.59s
Build TensorFlow for each Python version (dual abi) --------------------- 0.42s
Build step 'Execute shell' marked build as failure
2 weeks, 1 day
- 1
- 2
Build failed in Jenkins: Tensorflow Python package (manylinux2014 nightly) #1205
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/1205/dis…>
Changes:
------------------------------------------
[...truncated 1.57 MiB...]
Stats over 12 runs: max = 28.4s, min = 8.2s, avg = 17.5s, dev = 6.4s
//tensorflow/python/data/experimental/kernel_tests:group_by_reducer_test PASSED in 24.6s
Stats over 12 runs: max = 24.6s, min = 7.1s, avg = 13.8s, dev = 5.6s
//tensorflow/python/data/kernel_tests:choose_from_datasets_test PASSED in 18.7s
Stats over 12 runs: max = 18.7s, min = 4.9s, avg = 8.4s, dev = 3.9s
//tensorflow/python/data/kernel_tests:memory_cleanup_test_cpu PASSED in 13.9s
Stats over 12 runs: max = 13.9s, min = 4.4s, avg = 6.9s, dev = 2.7s
//tensorflow/python/distribute:moving_averages_test_2gpu PASSED in 28.0s
Stats over 12 runs: max = 28.0s, min = 12.6s, avg = 16.2s, dev = 4.1s
//tensorflow/python/distribute:moving_averages_test_cpu PASSED in 27.4s
Stats over 12 runs: max = 27.4s, min = 13.9s, avg = 17.5s, dev = 3.5s
//tensorflow/python/eager/polymorphic_function:polymorphic_function_test_cpu PASSED in 32.2s
Stats over 15 runs: max = 32.2s, min = 12.6s, avg = 18.5s, dev = 4.4s
//tensorflow/python/kernel_tests/linalg:linear_operator_low_rank_update_test_cpu PASSED in 146.9s
Stats over 15 runs: max = 146.9s, min = 109.3s, avg = 127.4s, dev = 10.3s
//tensorflow/python/kernel_tests/nn_ops:rnn_cell_test_cpu PASSED in 62.9s
Stats over 15 runs: max = 62.9s, min = 8.3s, avg = 16.9s, dev = 14.2s
//tensorflow/python/data/experimental/kernel_tests/service:dynamic_sharding_test PASSED in 16.9s
Stats over 16 runs: max = 16.9s, min = 5.1s, avg = 10.6s, dev = 3.5s
//tensorflow/python/data/kernel_tests:snapshot_test PASSED in 47.3s
Stats over 16 runs: max = 47.3s, min = 12.6s, avg = 27.4s, dev = 9.9s
//tensorflow/python/kernel_tests/control_flow:control_flow_ops_py_test_cpu PASSED in 28.0s
Stats over 16 runs: max = 28.0s, min = 6.1s, avg = 10.8s, dev = 5.0s
//tensorflow/python/kernel_tests/linalg:matrix_exponential_op_test PASSED in 10.7s
Stats over 16 runs: max = 10.7s, min = 4.5s, avg = 6.4s, dev = 1.9s
//tensorflow/python/kernel_tests/signal:dct_ops_test_cpu PASSED in 20.7s
Stats over 16 runs: max = 20.7s, min = 8.4s, avg = 12.5s, dev = 3.3s
//tensorflow/python/ops:image_ops_test_cpu PASSED in 26.3s
Stats over 16 runs: max = 26.3s, min = 8.5s, avg = 16.1s, dev = 4.2s
//tensorflow/python/data/experimental/kernel_tests/service:distributed_save_ft_test PASSED in 81.1s
Stats over 17 runs: max = 81.1s, min = 10.7s, avg = 35.5s, dev = 21.9s
//tensorflow/python/data/kernel_tests:map_test PASSED in 54.9s
Stats over 19 runs: max = 54.9s, min = 28.8s, avg = 38.8s, dev = 6.9s
//tensorflow/compiler/tests:pooling_ops_3d_test_cpu PASSED in 11.2s
Stats over 20 runs: max = 11.2s, min = 4.6s, avg = 5.4s, dev = 1.4s
//tensorflow/compiler/tests:pooling_ops_3d_test_cpu_mlir_bridge_test PASSED in 11.1s
Stats over 20 runs: max = 11.1s, min = 4.3s, avg = 5.0s, dev = 1.4s
//tensorflow/compiler/tests:pooling_ops_test_cpu PASSED in 18.5s
Stats over 20 runs: max = 18.5s, min = 3.6s, avg = 5.5s, dev = 3.2s
//tensorflow/compiler/tests:pooling_ops_test_cpu_mlir_bridge_test PASSED in 12.2s
Stats over 20 runs: max = 12.2s, min = 3.6s, avg = 5.2s, dev = 2.1s
//tensorflow/compiler/tests:stochastic_cast_op_test_cpu PASSED in 14.3s
Stats over 20 runs: max = 14.3s, min = 5.1s, avg = 6.0s, dev = 1.9s
//tensorflow/dtensor/python/tests:rng_test_cpu PASSED in 16.7s
Stats over 20 runs: max = 16.7s, min = 7.1s, avg = 8.7s, dev = 2.0s
//tensorflow/python/autograph/tests:loop_control_flow_test PASSED in 26.4s
Stats over 20 runs: max = 26.4s, min = 15.5s, avg = 18.5s, dev = 2.6s
//tensorflow/python/kernel_tests:metrics_test PASSED in 53.6s
Stats over 20 runs: max = 53.6s, min = 7.3s, avg = 22.6s, dev = 12.3s
//tensorflow/python/kernel_tests/array_ops:matrix_band_part_op_test_cpu PASSED in 10.3s
Stats over 20 runs: max = 10.3s, min = 4.3s, avg = 5.6s, dev = 1.4s
//tensorflow/python/kernel_tests/data_structures:barrier_ops_test PASSED in 14.8s
Stats over 20 runs: max = 14.8s, min = 4.3s, avg = 7.0s, dev = 3.1s
//tensorflow/python/kernel_tests/linalg:eig_op_test PASSED in 49.5s
Stats over 20 runs: max = 49.5s, min = 4.6s, avg = 15.8s, dev = 15.4s
//tensorflow/python/kernel_tests/linalg:linalg_grad_test_cpu PASSED in 119.4s
Stats over 20 runs: max = 119.4s, min = 28.7s, avg = 55.5s, dev = 21.3s
//tensorflow/python/kernel_tests/linalg:norm_op_test_cpu PASSED in 13.2s
Stats over 20 runs: max = 13.2s, min = 6.1s, avg = 7.8s, dev = 1.7s
//tensorflow/python/kernel_tests/linalg:normalize_op_test_cpu PASSED in 19.6s
Stats over 20 runs: max = 19.6s, min = 7.8s, avg = 12.5s, dev = 3.1s
//tensorflow/python/kernel_tests/linalg:qr_op_test_cpu PASSED in 319.1s
Stats over 20 runs: max = 319.1s, min = 43.5s, avg = 123.1s, dev = 78.0s
//tensorflow/python/kernel_tests/linalg:self_adjoint_eig_op_test_cpu PASSED in 26.1s
Stats over 20 runs: max = 26.1s, min = 5.2s, avg = 13.4s, dev = 7.7s
//tensorflow/python/kernel_tests/math_ops:batch_matmul_op_test_cpu PASSED in 23.9s
Stats over 20 runs: max = 23.9s, min = 7.6s, avg = 14.9s, dev = 5.7s
//tensorflow/python/kernel_tests/math_ops:matmul_op_test_cpu PASSED in 25.1s
Stats over 20 runs: max = 25.1s, min = 15.4s, avg = 19.1s, dev = 2.2s
//tensorflow/python/kernel_tests/math_ops:tensordot_op_test_cpu PASSED in 65.4s
Stats over 20 runs: max = 65.4s, min = 7.9s, avg = 32.0s, dev = 21.3s
//tensorflow/python/kernel_tests/nn_ops:embedding_ops_test_cpu PASSED in 29.8s
Stats over 20 runs: max = 29.8s, min = 12.6s, avg = 17.2s, dev = 4.1s
//tensorflow/compiler/tests:unary_ops_test_cpu FAILED in 3 out of 22 in 17.0s
Stats over 22 runs: max = 17.0s, min = 4.2s, avg = 8.6s, dev = 4.2s
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test_attempts/attempt_1.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test_attempts/attempt_2.log
//tensorflow/compiler/tests:unary_ops_test_cpu_mlir_bridge_test FAILED in 3 out of 22 in 19.2s
Stats over 22 runs: max = 19.2s, min = 3.7s, avg = 9.4s, dev = 5.0s
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test_attempts/attempt_1.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test_attempts/attempt_2.log
//tensorflow/python/data/kernel_tests:interleave_test PASSED in 41.4s
Stats over 24 runs: max = 41.4s, min = 11.5s, avg = 23.7s, dev = 8.4s
//tensorflow/python/data/kernel_tests:sample_from_datasets_test PASSED in 36.6s
Stats over 24 runs: max = 36.6s, min = 5.3s, avg = 12.7s, dev = 8.1s
//tensorflow/dtensor/python/tests:multi_device_spmd_test_cpu PASSED in 36.3s
Stats over 25 runs: max = 36.3s, min = 26.3s, avg = 30.3s, dev = 2.5s
//tensorflow/python/kernel_tests/nn_ops:conv_ops_3d_test_cpu PASSED in 23.1s
Stats over 30 runs: max = 23.1s, min = 4.6s, avg = 9.9s, dev = 4.9s
//tensorflow/python/data/experimental/kernel_tests/service:data_service_ops_test PASSED in 23.4s
Stats over 32 runs: max = 23.4s, min = 5.0s, avg = 11.0s, dev = 5.1s
//tensorflow/python/data/experimental/kernel_tests/service:worker_tags_test PASSED in 25.3s
Stats over 32 runs: max = 25.3s, min = 4.3s, avg = 11.8s, dev = 5.3s
//tensorflow/python/distribute:multi_process_runner_test_2gpu PASSED in 216.2s
Stats over 35 runs: max = 216.2s, min = 5.5s, avg = 24.5s, dev = 37.0s
//tensorflow/python/distribute:multi_process_runner_test_cpu PASSED in 220.9s
Stats over 35 runs: max = 220.9s, min = 5.4s, avg = 24.6s, dev = 37.8s
//tensorflow/core/kernels:stochastic_cast_op_test PASSED in 1.5s
Stats over 48 runs: max = 1.5s, min = 0.4s, avg = 0.6s, dev = 0.2s
//tensorflow/compiler/mlir/quantization/tensorflow/python:quantize_model_test PASSED in 78.8s
Stats over 50 runs: max = 78.8s, min = 42.9s, avg = 58.5s, dev = 9.0s
//tensorflow/compiler/tests:sort_ops_test_cpu PASSED in 16.7s
Stats over 50 runs: max = 16.7s, min = 3.8s, avg = 9.9s, dev = 3.5s
//tensorflow/compiler/tests:sort_ops_test_cpu_mlir_bridge_test PASSED in 16.4s
Stats over 50 runs: max = 16.4s, min = 4.0s, avg = 9.5s, dev = 3.4s
//tensorflow/python/kernel_tests/linalg:linear_operator_circulant_test_cpu PASSED in 69.1s
Stats over 50 runs: max = 69.1s, min = 44.5s, avg = 53.9s, dev = 6.2s
//tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_dense_mat_mul_grad_test_cpu PASSED in 21.8s
Stats over 50 runs: max = 21.8s, min = 5.6s, avg = 11.4s, dev = 3.9s
//tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_dense_mat_mul_onednn_grad_test PASSED in 20.5s
Stats over 50 runs: max = 20.5s, min = 5.6s, avg = 11.4s, dev = 3.6s
//tensorflow/python/kernel_tests/math_ops:cwise_ops_binary_test_cpu PASSED in 28.9s
Stats over 50 runs: max = 28.9s, min = 8.4s, avg = 16.0s, dev = 5.6s
//tensorflow/python/kernel_tests/math_ops:cwise_ops_test_cpu PASSED in 11.6s
Stats over 50 runs: max = 11.6s, min = 3.9s, avg = 4.8s, dev = 1.3s
Executed 3085 out of 3087 tests: 3083 tests pass, 2 fail to build and 2 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
stdout_lines: <omitted>
...ignoring
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] ***
Tuesday 18 June 2024 22:59:24 +0100 (2:15:05.506) 3:49:11.879 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] ***
Tuesday 18 June 2024 22:59:26 +0100 (0:00:01.868) 3:49:13.747 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ********
Tuesday 18 June 2024 22:59:27 +0100 (0:00:00.695) 3:49:14.442 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] ***
Tuesday 18 June 2024 22:59:28 +0100 (0:00:00.697) 3:49:15.139 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ******
Tuesday 18 June 2024 22:59:28 +0100 (0:00:00.704) 3:49:15.844 **********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] *********
Tuesday 18 June 2024 22:59:28 +0100 (0:00:00.081) 3:49:15.926 **********
ok: [localhost] =>
ut_tests_line: 'Executed 3085 out of 3087 tests: 3083 tests pass, 2 fail to build and 2 fail locally.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] *************
Tuesday 18 June 2024 22:59:28 +0100 (0:00:00.039) 3:49:15.965 **********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] *******
Tuesday 18 June 2024 22:59:29 +0100 (0:00:00.161) 3:49:16.126 **********
ok: [localhost] =>
tests_flaky:
- //tensorflow/python/kernel_tests/linalg:matrix_triangular_solve_op_test_cpu FLAKY, failed in 1 out of 4 in 900.4s
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] **********
Tuesday 18 June 2024 22:59:29 +0100 (0:00:00.038) 3:49:16.165 **********
ok: [localhost] =>
tests_failed:
- 'FAILED: //tensorflow/compiler/tests:unary_ops_test_cpu_mlir_bridge_test (Summary)'
- 'FAILED: //tensorflow/compiler/tests:unary_ops_test_cpu (Summary)'
- 'FAILED: Build did NOT complete successfully'
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] ***
Tuesday 18 June 2024 22:59:29 +0100 (0:00:00.038) 3:49:16.203 **********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
+ for wheel in /tf/pkg/*.whl
+ echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl...'
+ python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg
+ tee check.txt
real 2m30.267s
user 2m14.570s
sys 0m15.596s
++ grep --extended-regexp --only-matching '/tf/pkg/\S+.whl' check.txt
+ new_wheel=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ [[ /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl != \/\t\f\/\p\k\g\/\t\e\n\s\o\r\f\l\o\w\_\a\a\r\c\h\6\4\-\2\.\1\8\.\0\-\c\p\3\9\-\c\p\3\9\-\l\i\n\u\x\_\a\a\r\c\h\6\4\.\w\h\l ]]
+ rm /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl
+ wheel=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ TF_WHEEL=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ bats /usertools/wheel_verification.bats --timing
stderr_lines: <omitted>
stdout: |-
Checking and renaming /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl...
INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_aarch64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_aarch64, manylinux2014_aarch64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp39-cp39-linux_aarch64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp39-cp39-manylinux_2_17_aarch64, cp39-cp39-manylinux2014_aarch64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
1..5
ok 1 Wheel is manylinux2014 (manylinux_2_17) compliant in 68696ms
# Size of /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl is 263 / 580 megabytes.
ok 2 Wheel conforms to upstream size limitations in 19ms
ok 3 Wheel is installable in 57389ms
ok 4 TensorFlow is importable in 4073ms
not ok 5 TensorFlow has Keras in 3756ms
# (in test file /usertools/wheel_verification.bats, line 72)
# `python3 -c 'import sys; import tensorflow as tf; sys.exit(0 if "keras" in tf.keras.__name__ else 1)'' failed
# Traceback (most recent call last):
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 147, in _initialize
# import keras # pylint: disable=g-import-not-at-top
# File "/tf/venv/lib/python3.9/site-packages/keras/__init__.py", line 23, in <module>
# from keras.api import _tf_keras
# ImportError: cannot import name '_tf_keras' from 'keras.api' (/tf/venv/lib/python3.9/site-packages/keras/api/__init__.py)
#
# During handling of the above exception, another exception occurred:
#
# Traceback (most recent call last):
# File "<string>", line 1, in <module>
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 181, in __getattr__
# self._initialize()
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 161, in _initialize
# raise ImportError( # pylint: disable=raise-missing-from
# ImportError: Keras cannot be imported. Check that it is installed.
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=1
Tuesday 18 June 2024 23:04:17 +0100 (0:04:48.629) 3:54:04.833 **********
===============================================================================
tensorflow_in_container : (Python 3.9) run TensorFlow git tests ------ 8105.51s
tensorflow_in_container : (Python 3.9) build TensorFlow git ---------- 4448.16s
tensorflow_in_container : (Python 3.9) start build container --------- 1058.84s
tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one - 288.63s
Cleanup after previous build ------------------------------------------- 93.14s
tensorflow : Fetch TensorFlow from git --------------------------------- 22.83s
tensorflow : Remove TensorFlow git sources from previous build ---------- 4.22s
Gathering Facts --------------------------------------------------------- 1.95s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.89s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 1.87s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.58s
Create cache dirs ------------------------------------------------------- 1.42s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.21s
tensorflow : Save git commit hash to file ------------------------------- 1.15s
tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 1.07s
tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.89s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.84s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.82s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.82s
tensorflow_in_container : (Python 3.9) save tests stderr output to file --- 0.70s
Build step 'Execute shell' marked build as failure
3 weeks, 6 days
- 1
- 11
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #835
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 1.37 MiB...]
Stats over 10 runs: max = 63.3s, min = 32.7s, avg = 39.6s, dev = 8.8s
//tensorflow/compiler/tests:fft_test_cpu PASSED in 20.4s
Stats over 12 runs: max = 20.4s, min = 7.2s, avg = 14.7s, dev = 4.8s
//tensorflow/python/data/experimental/kernel_tests:group_by_reducer_test PASSED in 16.4s
Stats over 12 runs: max = 16.4s, min = 4.1s, avg = 9.5s, dev = 3.9s
//tensorflow/python/data/kernel_tests:choose_from_datasets_test PASSED in 15.8s
Stats over 12 runs: max = 15.8s, min = 5.2s, avg = 7.3s, dev = 2.8s
//tensorflow/python/data/kernel_tests:memory_cleanup_test_cpu PASSED in 10.8s
Stats over 12 runs: max = 10.8s, min = 4.4s, avg = 6.5s, dev = 1.9s
//tensorflow/python/distribute:moving_averages_test_2gpu PASSED in 23.4s
Stats over 12 runs: max = 23.4s, min = 11.4s, avg = 14.4s, dev = 3.1s
//tensorflow/python/distribute:moving_averages_test_cpu PASSED in 22.7s
Stats over 12 runs: max = 22.7s, min = 11.3s, avg = 15.2s, dev = 2.8s
//tensorflow/python/eager/polymorphic_function:polymorphic_function_test_cpu PASSED in 28.5s
Stats over 15 runs: max = 28.5s, min = 12.1s, avg = 17.1s, dev = 4.9s
//tensorflow/python/kernel_tests/linalg:linear_operator_low_rank_update_test_cpu PASSED in 176.0s
Stats over 15 runs: max = 176.0s, min = 135.5s, avg = 150.3s, dev = 11.0s
//tensorflow/python/kernel_tests/nn_ops:rnn_cell_test_cpu PASSED in 54.7s
Stats over 15 runs: max = 54.7s, min = 6.6s, avg = 13.4s, dev = 12.3s
//tensorflow/python/data/experimental/kernel_tests/service:dynamic_sharding_test PASSED in 11.9s
Stats over 16 runs: max = 11.9s, min = 4.2s, avg = 7.8s, dev = 2.7s
//tensorflow/python/data/kernel_tests:snapshot_test PASSED in 35.2s
Stats over 16 runs: max = 35.2s, min = 12.0s, avg = 19.6s, dev = 6.0s
//tensorflow/python/kernel_tests/control_flow:control_flow_ops_py_test_cpu PASSED in 28.1s
Stats over 16 runs: max = 28.1s, min = 5.8s, avg = 9.2s, dev = 5.3s
//tensorflow/python/kernel_tests/linalg:matrix_exponential_op_test PASSED in 16.8s
Stats over 16 runs: max = 16.8s, min = 4.3s, avg = 7.6s, dev = 3.0s
//tensorflow/python/kernel_tests/signal:dct_ops_test_cpu PASSED in 17.0s
Stats over 16 runs: max = 17.0s, min = 8.4s, avg = 11.5s, dev = 2.1s
//tensorflow/python/ops:image_ops_test_cpu PASSED in 26.9s
Stats over 16 runs: max = 26.9s, min = 8.6s, avg = 15.0s, dev = 4.5s
//tensorflow/python/data/experimental/kernel_tests/service:distributed_save_ft_test PASSED in 57.9s
Stats over 17 runs: max = 57.9s, min = 8.3s, avg = 25.3s, dev = 15.6s
//tensorflow/python/data/kernel_tests:map_test PASSED in 76.9s
Stats over 19 runs: max = 76.9s, min = 27.9s, avg = 40.2s, dev = 10.5s
//tensorflow/compiler/tests:pooling_ops_3d_test_cpu PASSED in 13.9s
Stats over 20 runs: max = 13.9s, min = 4.2s, avg = 5.6s, dev = 2.0s
//tensorflow/compiler/tests:pooling_ops_3d_test_cpu_mlir_bridge_test PASSED in 12.4s
Stats over 20 runs: max = 12.4s, min = 4.3s, avg = 5.9s, dev = 1.9s
//tensorflow/compiler/tests:pooling_ops_test_cpu PASSED in 11.3s
Stats over 20 runs: max = 11.3s, min = 4.1s, avg = 5.7s, dev = 1.8s
//tensorflow/compiler/tests:pooling_ops_test_cpu_mlir_bridge_test PASSED in 12.2s
Stats over 20 runs: max = 12.2s, min = 4.4s, avg = 5.9s, dev = 2.1s
//tensorflow/compiler/tests:stochastic_cast_op_test_cpu PASSED in 12.0s
Stats over 20 runs: max = 12.0s, min = 5.1s, avg = 6.0s, dev = 1.4s
//tensorflow/dtensor/python/tests:rng_test_cpu PASSED in 14.2s
Stats over 20 runs: max = 14.2s, min = 6.5s, avg = 7.9s, dev = 1.6s
//tensorflow/python/autograph/tests:loop_control_flow_test PASSED in 29.1s
Stats over 20 runs: max = 29.1s, min = 14.0s, avg = 16.8s, dev = 3.1s
//tensorflow/python/kernel_tests:metrics_test PASSED in 44.4s
Stats over 20 runs: max = 44.4s, min = 7.9s, avg = 20.0s, dev = 10.2s
//tensorflow/python/kernel_tests/array_ops:matrix_band_part_op_test_cpu PASSED in 10.5s
Stats over 20 runs: max = 10.5s, min = 3.8s, avg = 4.8s, dev = 1.3s
//tensorflow/python/kernel_tests/data_structures:barrier_ops_test PASSED in 12.9s
Stats over 20 runs: max = 12.9s, min = 4.1s, avg = 6.3s, dev = 2.9s
//tensorflow/python/kernel_tests/linalg:eig_op_test PASSED in 56.6s
Stats over 20 runs: max = 56.6s, min = 5.1s, avg = 17.5s, dev = 15.9s
//tensorflow/python/kernel_tests/linalg:linalg_grad_test_cpu PASSED in 136.5s
Stats over 20 runs: max = 136.5s, min = 42.0s, avg = 76.5s, dev = 27.9s
//tensorflow/python/kernel_tests/linalg:norm_op_test_cpu PASSED in 14.0s
Stats over 20 runs: max = 14.0s, min = 6.4s, avg = 9.1s, dev = 2.2s
//tensorflow/python/kernel_tests/linalg:normalize_op_test_cpu PASSED in 19.2s
Stats over 20 runs: max = 19.2s, min = 8.1s, avg = 12.7s, dev = 3.2s
//tensorflow/python/kernel_tests/linalg:qr_op_test_cpu PASSED in 153.2s
Stats over 20 runs: max = 153.2s, min = 42.6s, avg = 82.8s, dev = 32.0s
//tensorflow/python/kernel_tests/linalg:self_adjoint_eig_op_test_cpu PASSED in 25.5s
Stats over 20 runs: max = 25.5s, min = 5.0s, avg = 12.1s, dev = 6.5s
//tensorflow/python/kernel_tests/math_ops:batch_matmul_op_test_cpu PASSED in 23.1s
Stats over 20 runs: max = 23.1s, min = 8.7s, avg = 14.3s, dev = 3.7s
//tensorflow/python/kernel_tests/math_ops:matmul_op_test_cpu PASSED in 34.8s
Stats over 20 runs: max = 34.8s, min = 21.2s, avg = 26.1s, dev = 3.7s
//tensorflow/python/kernel_tests/math_ops:tensordot_op_test_cpu PASSED in 81.5s
Stats over 20 runs: max = 81.5s, min = 10.0s, avg = 38.1s, dev = 24.7s
//tensorflow/python/kernel_tests/nn_ops:embedding_ops_test_cpu PASSED in 42.4s
Stats over 20 runs: max = 42.4s, min = 11.8s, avg = 19.0s, dev = 7.2s
//tensorflow/compiler/tests:unary_ops_test_cpu FAILED in 3 out of 22 in 25.0s
Stats over 22 runs: max = 25.0s, min = 4.2s, avg = 10.4s, dev = 6.5s
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test_attempts/attempt_1.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu/shard_18_of_20/test_attempts/attempt_2.log
//tensorflow/compiler/tests:unary_ops_test_cpu_mlir_bridge_test FAILED in 3 out of 22 in 27.0s
Stats over 22 runs: max = 27.0s, min = 3.8s, avg = 11.0s, dev = 6.5s
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test_attempts/attempt_1.log
/home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/tests/unary_ops_test_cpu_mlir_bridge_test/shard_18_of_20/test_attempts/attempt_2.log
//tensorflow/python/data/kernel_tests:interleave_test PASSED in 31.1s
Stats over 24 runs: max = 31.1s, min = 9.2s, avg = 17.2s, dev = 6.3s
//tensorflow/python/data/kernel_tests:sample_from_datasets_test PASSED in 20.4s
Stats over 24 runs: max = 20.4s, min = 4.6s, avg = 9.8s, dev = 4.8s
//tensorflow/dtensor/python/tests:multi_device_spmd_test_cpu PASSED in 31.1s
Stats over 25 runs: max = 31.1s, min = 23.1s, avg = 27.2s, dev = 2.0s
//tensorflow/python/kernel_tests/nn_ops:conv_ops_3d_test_cpu PASSED in 43.9s
Stats over 30 runs: max = 43.9s, min = 4.4s, avg = 15.1s, dev = 10.3s
//tensorflow/python/data/experimental/kernel_tests/service:data_service_ops_test PASSED in 17.4s
Stats over 32 runs: max = 17.4s, min = 4.4s, avg = 8.7s, dev = 3.3s
//tensorflow/python/data/experimental/kernel_tests/service:worker_tags_test PASSED in 19.6s
Stats over 32 runs: max = 19.6s, min = 4.2s, avg = 9.9s, dev = 3.9s
//tensorflow/python/distribute:multi_process_runner_test_2gpu PASSED in 215.6s
Stats over 35 runs: max = 215.6s, min = 4.5s, avg = 22.8s, dev = 37.1s
//tensorflow/python/distribute:multi_process_runner_test_cpu PASSED in 218.8s
Stats over 35 runs: max = 218.8s, min = 4.3s, avg = 23.7s, dev = 37.5s
//tensorflow/core/kernels:stochastic_cast_op_test PASSED in 1.7s
Stats over 48 runs: max = 1.7s, min = 0.4s, avg = 0.5s, dev = 0.3s
//tensorflow/compiler/mlir/quantization/tensorflow/python:quantize_model_test PASSED in 63.3s
Stats over 50 runs: max = 63.3s, min = 31.5s, avg = 43.2s, dev = 8.0s
//tensorflow/compiler/tests:sort_ops_test_cpu PASSED in 21.2s
Stats over 50 runs: max = 21.2s, min = 3.9s, avg = 11.7s, dev = 4.5s
//tensorflow/compiler/tests:sort_ops_test_cpu_mlir_bridge_test PASSED in 18.3s
Stats over 50 runs: max = 18.3s, min = 4.2s, avg = 10.9s, dev = 4.0s
//tensorflow/python/kernel_tests/linalg:linear_operator_circulant_test_cpu PASSED in 83.4s
Stats over 50 runs: max = 83.4s, min = 48.2s, avg = 60.8s, dev = 7.9s
//tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_dense_mat_mul_grad_test_cpu PASSED in 21.4s
Stats over 50 runs: max = 21.4s, min = 6.2s, avg = 11.4s, dev = 3.6s
//tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_dense_mat_mul_onednn_grad_test PASSED in 25.8s
Stats over 50 runs: max = 25.8s, min = 5.5s, avg = 11.9s, dev = 4.2s
//tensorflow/python/kernel_tests/math_ops:cwise_ops_binary_test_cpu PASSED in 39.1s
Stats over 50 runs: max = 39.1s, min = 10.5s, avg = 21.3s, dev = 7.2s
//tensorflow/python/kernel_tests/math_ops:cwise_ops_test_cpu PASSED in 12.3s
Stats over 50 runs: max = 12.3s, min = 4.3s, avg = 6.3s, dev = 1.6s
Executed 3086 out of 3086 tests: 3084 tests pass and 2 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
stdout_lines: <omitted>
...ignoring
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] ***
Tuesday 18 June 2024 04:41:58 +0100 (2:05:25.924) 3:31:53.188 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] ***
Tuesday 18 June 2024 04:42:00 +0100 (0:00:02.409) 3:31:55.597 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ********
Tuesday 18 June 2024 04:42:01 +0100 (0:00:00.691) 3:31:56.288 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] ***
Tuesday 18 June 2024 04:42:02 +0100 (0:00:00.699) 3:31:56.988 **********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ******
Tuesday 18 June 2024 04:42:02 +0100 (0:00:00.704) 3:31:57.692 **********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] *********
Tuesday 18 June 2024 04:42:03 +0100 (0:00:00.080) 3:31:57.772 **********
ok: [localhost] =>
ut_tests_line: 'Executed 3086 out of 3086 tests: 3084 tests pass and 2 fail locally.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] *************
Tuesday 18 June 2024 04:42:03 +0100 (0:00:00.040) 3:31:57.813 **********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] *******
Tuesday 18 June 2024 04:42:03 +0100 (0:00:00.156) 3:31:57.969 **********
ok: [localhost] =>
tests_flaky: []
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] **********
Tuesday 18 June 2024 04:42:03 +0100 (0:00:00.039) 3:31:58.009 **********
ok: [localhost] =>
tests_failed:
- 'FAILED: //tensorflow/compiler/tests:unary_ops_test_cpu (Summary)'
- 'FAILED: //tensorflow/compiler/tests:unary_ops_test_cpu_mlir_bridge_test (Summary)'
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] ***
Tuesday 18 June 2024 04:42:03 +0100 (0:00:00.040) 3:31:58.050 **********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
+ for wheel in /tf/pkg/*.whl
+ echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl...'
+ python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg
+ tee check.txt
real 2m38.362s
user 2m11.527s
sys 0m20.972s
++ grep --extended-regexp --only-matching '/tf/pkg/\S+.whl' check.txt
+ new_wheel=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ [[ /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl != \/\t\f\/\p\k\g\/\t\e\n\s\o\r\f\l\o\w\_\a\a\r\c\h\6\4\-\2\.\1\8\.\0\-\c\p\3\9\-\c\p\3\9\-\l\i\n\u\x\_\a\a\r\c\h\6\4\.\w\h\l ]]
+ rm /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl
+ wheel=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ TF_WHEEL=/tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ bats /usertools/wheel_verification.bats --timing
stderr_lines: <omitted>
stdout: |-
Checking and renaming /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl...
INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.18.0-cp39-cp39-linux_aarch64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_aarch64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_aarch64, manylinux2014_aarch64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp39-cp39-linux_aarch64
INFO:auditwheel.wheeltools:New WHEEL info tags: cp39-cp39-manylinux_2_17_aarch64, cp39-cp39-manylinux2014_aarch64
INFO:auditwheel.main_repair:
Fixed-up wheel written to /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
1..5
ok 1 Wheel is manylinux2014 (manylinux_2_17) compliant in 75030ms
# Size of /tf/pkg/tensorflow_aarch64-2.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl is 209 / 580 megabytes.
ok 2 Wheel conforms to upstream size limitations in 18ms
ok 3 Wheel is installable in 58309ms
ok 4 TensorFlow is importable in 4345ms
not ok 5 TensorFlow has Keras in 3732ms
# (in test file /usertools/wheel_verification.bats, line 72)
# `python3 -c 'import sys; import tensorflow as tf; sys.exit(0 if "keras" in tf.keras.__name__ else 1)'' failed
# Traceback (most recent call last):
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 147, in _initialize
# import keras # pylint: disable=g-import-not-at-top
# File "/tf/venv/lib/python3.9/site-packages/keras/__init__.py", line 23, in <module>
# from keras.api import _tf_keras
# ImportError: cannot import name '_tf_keras' from 'keras.api' (/tf/venv/lib/python3.9/site-packages/keras/api/__init__.py)
#
# During handling of the above exception, another exception occurred:
#
# Traceback (most recent call last):
# File "<string>", line 1, in <module>
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 181, in __getattr__
# self._initialize()
# File "/tf/venv/lib/python3.9/site-packages/tensorflow/python/util/lazy_loader.py", line 161, in _initialize
# raise ImportError( # pylint: disable=raise-missing-from
# ImportError: Keras cannot be imported. Check that it is installed.
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=1
Tuesday 18 June 2024 04:47:05 +0100 (0:05:02.388) 3:37:00.438 **********
===============================================================================
tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 7525.92s
tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 5080.08s
tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one - 302.39s
Cleanup after previous build ------------------------------------------- 49.44s
tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 23.05s
tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 5.28s
tensorflow_in_container : (Python 3.9) start build container ------------ 4.74s
tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 3.00s
Gathering Facts --------------------------------------------------------- 2.72s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.53s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 2.41s
tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 2.23s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.70s
Create cache dirs ------------------------------------------------------- 1.44s
tensorflow_in_container : (Python 3.9) create script to run tests ------- 1.37s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.22s
tensorflow : Save git commit hash to file ------------------------------- 1.15s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.85s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.83s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.80s
Build step 'Execute shell' marked build as failure
1 month
- 1
- 3
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #833
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 8.40 KiB...]
Sunday 16 June 2024 01:11:38 +0100 (0:00:00.079) 0:01:33.266 ***********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Sunday 16 June 2024 01:11:39 +0100 (0:00:00.691) 0:01:33.957 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Sunday 16 June 2024 01:11:39 +0100 (0:00:00.645) 0:01:34.603 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Sunday 16 June 2024 01:11:40 +0100 (0:00:00.687) 0:01:35.291 ***********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Sunday 16 June 2024 01:11:42 +0100 (0:00:01.734) 0:01:37.026 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Sunday 16 June 2024 01:11:47 +0100 (0:00:05.454) 0:01:42.481 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Sunday 16 June 2024 01:11:49 +0100 (0:00:01.235) 0:01:43.716 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Sunday 16 June 2024 01:11:49 +0100 (0:00:00.899) 0:01:44.616 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Sunday 16 June 2024 01:11:50 +0100 (0:00:00.861) 0:01:45.478 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Sunday 16 June 2024 01:11:51 +0100 (0:00:00.857) 0:01:46.336 ***********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Sunday 16 June 2024 01:11:54 +0100 (0:00:03.279) 0:01:49.615 ***********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip
pip 24.0 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip
wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] ***
Sunday 16 June 2024 01:11:54 +0100 (0:00:00.037) 0:01:49.653 ***********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: d8965d29-38fd-466e-a52f-f1776b7bd719
INFO: Reading 'startup' options from /tf/tensorflow/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /tf/tensorflow/.bazelrc:
'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /usertools/aarch64_clang.bazelrc:
'build' options: --action_env=CACHEBUSTER=20220325 --action_env PYTHON_BIN_PATH=/usr/local/bin/python3 --python_path=/usr/local/bin/python3 --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 --linkopt=-fuse-ld=lld --linkopt=-lm --linkopt=-Wl,--undefined-version --host_linkopt=-fuse-ld=lld --host_linkopt=-lm --host_linkopt=-Wl,--undefined-version --distinct_host_configuration=false --copt=-Wno-gnu-offsetof-extensions --profile=/tf/pkg/profile.json.gz --crosstool_top=@ml2014_clang_aarch64_config_aarch64//crosstool:toolchain --copt=-mtune=generic --copt=-march=armv8-a --copt=-O3
INFO: Found applicable config definition build:short_logs in file /tf/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /tf/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:sigbuild_local_cache in file /usertools/aarch64_clang.bazelrc: --disk_cache=/tf/cache
INFO: Found applicable config definition build:mkl_aarch64_threadpool in file /tf/tensorflow/.bazelrc: --define=build_with_mkl_aarch64=true -c opt
INFO: Found applicable config definition build:linux in file /tf/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /tf/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
INFO: Repository llvm-raw instantiated at:
/tf/tensorflow/WORKSPACE:91:14: in <toplevel>
/tf/tensorflow/tensorflow/workspace3.bzl:63:9: in workspace
/tf/tensorflow/third_party/llvm/workspace.bzl:10:20: in repo
/tf/tensorflow/third_party/repo.bzl:136:21: in tf_http_archive
Repository rule _tf_http_archive defined at:
/tf/tensorflow/third_party/repo.bzl:89:35: in <toplevel>
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-p… failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: An error occurred during the fetch of repository 'llvm-raw':
Traceback (most recent call last):
File "/tf/tensorflow/third_party/repo.bzl", line 83, column 30, in _tf_http_archive_impl
ctx.patch(patch_file, strip = 1)
Error in patch: Error applying patch /tf/tensorflow/third_party/llvm/typo.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 1051
**Original Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformObjects",
],
)
**Revised Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformsObjects",
],
)
ERROR: /tf/tensorflow/WORKSPACE:91:14: fetching _tf_http_archive rule //external:llvm-raw: Traceback (most recent call last):
File "/tf/tensorflow/third_party/repo.bzl", line 83, column 30, in _tf_http_archive_impl
ctx.patch(patch_file, strip = 1)
Error in patch: Error applying patch /tf/tensorflow/third_party/llvm/typo.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 1051
**Original Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformObjects",
],
)
**Revised Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformsObjects",
],
)
ERROR: Error computing the main repository mapping: no such package '@llvm-raw//utils/bazel': Error applying patch /tf/tensorflow/third_party/llvm/typo.patch: Incorrect Chunk: the chunk content doesn't match the target
**Original Position**: 1051
**Original Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformObjects",
],
)
**Revised Content**:
":CAPIDebugObjects",
":CAPIIRObjects",
":CAPIInterfacesObjects",
":CAPITransformsObjects",
],
)
stderr_lines: <omitted>
stdout: ''
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=25 changed=17 unreachable=0 failed=1 skipped=6 rescued=0 ignored=0
Sunday 16 June 2024 01:13:21 +0100 (0:01:26.392) 0:03:16.045 ***********
===============================================================================
tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git -- 86.39s
Cleanup after previous build ------------------------------------------- 57.36s
tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 22.00s
tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 6.88s
tensorflow_in_container : (Python 3.9) start build container ------------ 5.45s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 3.28s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.73s
Gathering Facts --------------------------------------------------------- 1.70s
Create cache dirs ------------------------------------------------------- 1.45s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.24s
tensorflow : Save git commit hash to file ------------------------------- 1.14s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.90s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.86s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.86s
tensorflow : Create linklibs shell script ------------------------------- 0.69s
tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds --- 0.69s
tensorflow_in_container : Reset checkout to clean state ----------------- 0.66s
tensorflow : Use nightly package requirements --------------------------- 0.65s
tensorflow : Get oneline version of git log for this checkout ----------- 0.58s
Build TensorFlow for each Python version (dual abi) --------------------- 0.43s
Build step 'Execute shell' marked build as failure
1 month, 1 week
- 1
- 1
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #818
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 7.02 KiB...]
TASK [tensorflow : Record git commit hash to variable] *************************
Saturday 01 June 2024 01:11:14 +0100 (0:00:00.036) 0:01:10.344 *********
ok: [localhost]
TASK [tensorflow : Save git commit hash to file] *******************************
Saturday 01 June 2024 01:11:14 +0100 (0:00:00.058) 0:01:10.403 *********
changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] *********************
Saturday 01 June 2024 01:11:15 +0100 (0:00:01.103) 0:01:11.506 *********
TASK [tensorflow_in_container : Reset checkout to clean state] *****************
Saturday 01 June 2024 01:11:16 +0100 (0:00:00.411) 0:01:11.917 *********
changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] ***
Saturday 01 June 2024 01:11:16 +0100 (0:00:00.690) 0:01:12.608 *********
skipping: [localhost]
TASK [Fetch TensorFlow] ********************************************************
Saturday 01 June 2024 01:11:16 +0100 (0:00:00.029) 0:01:12.638 *********
skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] **************************************
Saturday 01 June 2024 01:11:17 +0100 (0:00:00.030) 0:01:12.668 *********
TASK [tensorflow : Create linklibs shell script] *******************************
Saturday 01 June 2024 01:11:17 +0100 (0:00:00.082) 0:01:12.751 *********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Saturday 01 June 2024 01:11:17 +0100 (0:00:00.695) 0:01:13.446 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Saturday 01 June 2024 01:11:18 +0100 (0:00:00.638) 0:01:14.084 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Saturday 01 June 2024 01:11:19 +0100 (0:00:00.688) 0:01:14.773 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Saturday 01 June 2024 01:11:20 +0100 (0:00:01.629) 0:01:16.403 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Saturday 01 June 2024 01:11:22 +0100 (0:00:02.163) 0:01:18.567 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Saturday 01 June 2024 01:11:24 +0100 (0:00:01.345) 0:01:19.912 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Saturday 01 June 2024 01:11:25 +0100 (0:00:00.905) 0:01:20.818 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Saturday 01 June 2024 01:11:26 +0100 (0:00:00.843) 0:01:21.661 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Saturday 01 June 2024 01:11:26 +0100 (0:00:00.865) 0:01:22.526 *********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Saturday 01 June 2024 01:11:29 +0100 (0:00:02.342) 0:01:24.869 *********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip
pip 24.0 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip
wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] ***
Saturday 01 June 2024 01:11:29 +0100 (0:00:00.037) 0:01:24.907 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] *************
Saturday 01 June 2024 02:20:32 +0100 (1:09:02.754) 1:10:27.662 *********
skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] *****************
Saturday 01 June 2024 02:20:32 +0100 (0:00:00.040) 1:10:27.702 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] ***
Saturday 01 June 2024 02:20:33 +0100 (0:00:01.579) 1:10:29.282 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] *******
Saturday 01 June 2024 02:20:36 +0100 (0:00:02.703) 1:10:31.985 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] ***
Saturday 01 June 2024 02:20:37 +0100 (0:00:00.966) 1:10:32.952 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] ***
Saturday 01 June 2024 04:41:53 +0100 (2:21:16.641) 3:31:49.593 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] ***
Saturday 01 June 2024 04:41:55 +0100 (0:00:01.870) 3:31:51.463 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ********
Saturday 01 June 2024 04:41:56 +0100 (0:00:00.690) 3:31:52.154 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] ***
Saturday 01 June 2024 04:41:57 +0100 (0:00:00.698) 3:31:52.853 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ******
Saturday 01 June 2024 04:41:57 +0100 (0:00:00.704) 3:31:53.557 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] *********
Saturday 01 June 2024 04:41:57 +0100 (0:00:00.076) 3:31:53.634 *********
ok: [localhost] =>
ut_tests_line: 'Executed 3086 out of 3086 tests: 3086 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] *************
Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.674 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] *******
Saturday 01 June 2024 04:41:58 +0100 (0:00:00.127) 3:31:53.802 *********
ok: [localhost] =>
tests_flaky:
- //tensorflow/core/kernels/mkl:mkl_qmatmul_op_test FLAKY, failed in 1 out of 2 in 0.3s
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] **********
Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.842 *********
ok: [localhost] =>
tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] ***
Saturday 01 June 2024 04:41:58 +0100 (0:00:00.039) 3:31:53.881 *********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
+ for wheel in /tf/pkg/*.whl
+ echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...'
+ python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg
+ tee check.txt
real 1m27.249s
user 1m11.201s
sys 0m15.533s
stderr_lines: <omitted>
stdout: |-
Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...
INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel
raise ValueError(
ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main
rval = args.func(args, p)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute
out_wheel = repair_wheel(
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel
strip_symbols(itertools.chain(libs_to_strip, extensions))
File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__
dir2zip(self.name, self.out_wheel, date_time)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip
with zipfile.ZipFile(zip_fname, "w", compression=compression) as z:
File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__
self.fp = io.open(file, filemode)
PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl'
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Saturday 01 June 2024 04:43:26 +0100 (0:01:28.052) 3:33:21.934 *********
===============================================================================
tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 8476.64s
tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 4142.75s
tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 88.05s
Cleanup after previous build ------------------------------------------- 38.04s
tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 22.84s
tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 4.94s
tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 2.70s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 2.34s
tensorflow_in_container : (Python 3.9) start build container ------------ 2.16s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 1.87s
Gathering Facts --------------------------------------------------------- 1.69s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.63s
tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 1.58s
Create cache dirs ------------------------------------------------------- 1.43s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.35s
tensorflow : Save git commit hash to file ------------------------------- 1.10s
tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.97s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.91s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.87s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.84s
Build step 'Execute shell' marked build as failure
1 month, 3 weeks
- 1
- 3
Build failed in Jenkins: Tensorflow Python package (manylinux2014 nightly) #1188
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/1188/dis…>
Changes:
------------------------------------------
[...truncated 6.63 KiB...]
git_log.stdout: f2535930fb518a3996cac26d59c3dd2bf9776e77 [XLA:GPU] Clang-tidy cleanup for xla/service/gpu/fusions/mlir/mlir_fusion_emitter.h
TASK [tensorflow : Record git commit hash to variable] *************************
Saturday 01 June 2024 19:11:14 +0100 (0:00:00.037) 0:01:09.722 *********
ok: [localhost]
TASK [tensorflow : Save git commit hash to file] *******************************
Saturday 01 June 2024 19:11:14 +0100 (0:00:00.055) 0:01:09.778 *********
changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] *********************
Saturday 01 June 2024 19:11:15 +0100 (0:00:01.137) 0:01:10.915 *********
TASK [tensorflow_in_container : Reset checkout to clean state] *****************
Saturday 01 June 2024 19:11:15 +0100 (0:00:00.412) 0:01:11.328 *********
changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow git sources from previous build] ***
Saturday 01 June 2024 19:11:16 +0100 (0:00:00.646) 0:01:11.974 *********
skipping: [localhost]
TASK [Fetch TensorFlow] ********************************************************
Saturday 01 June 2024 19:11:16 +0100 (0:00:00.029) 0:01:12.004 *********
skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] **************************************
Saturday 01 June 2024 19:11:16 +0100 (0:00:00.030) 0:01:12.034 *********
TASK [tensorflow : Create linklibs shell script] *******************************
Saturday 01 June 2024 19:11:16 +0100 (0:00:00.081) 0:01:12.116 *********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Saturday 01 June 2024 19:11:17 +0100 (0:00:00.685) 0:01:12.802 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Saturday 01 June 2024 19:11:17 +0100 (0:00:00.626) 0:01:13.429 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Saturday 01 June 2024 19:11:18 +0100 (0:00:00.680) 0:01:14.109 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Saturday 01 June 2024 19:11:19 +0100 (0:00:01.486) 0:01:15.596 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Saturday 01 June 2024 19:27:00 +0100 (0:15:40.311) 0:16:55.907 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Saturday 01 June 2024 19:27:01 +0100 (0:00:01.205) 0:16:57.113 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Saturday 01 June 2024 19:27:02 +0100 (0:00:00.823) 0:16:57.936 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Saturday 01 June 2024 19:27:03 +0100 (0:00:00.866) 0:16:58.803 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Saturday 01 June 2024 19:27:03 +0100 (0:00:00.800) 0:16:59.603 *********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Saturday 01 June 2024 19:27:05 +0100 (0:00:01.857) 0:17:01.461 *********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip
pip 24.0 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip
wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow git] *************
Saturday 01 June 2024 19:27:05 +0100 (0:00:00.037) 0:17:01.498 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] *************
Saturday 01 June 2024 20:32:04 +0100 (1:04:58.846) 1:22:00.345 *********
skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] *****************
Saturday 01 June 2024 20:32:04 +0100 (0:00:00.031) 1:22:00.376 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] ***
Saturday 01 June 2024 20:32:05 +0100 (0:00:00.909) 1:22:01.285 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] *******
Saturday 01 June 2024 20:32:06 +0100 (0:00:00.869) 1:22:02.155 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow git tests] *********
Saturday 01 June 2024 20:32:07 +0100 (0:00:00.703) 1:22:02.858 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] ***
Saturday 01 June 2024 22:37:19 +0100 (2:05:11.868) 3:27:14.727 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] ***
Saturday 01 June 2024 22:37:21 +0100 (0:00:02.366) 3:27:17.093 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ********
Saturday 01 June 2024 22:37:22 +0100 (0:00:00.679) 3:27:17.773 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] ***
Saturday 01 June 2024 22:37:22 +0100 (0:00:00.694) 3:27:18.467 *********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ******
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.700) 3:27:19.167 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] *********
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.075) 3:27:19.243 *********
ok: [localhost] =>
ut_tests_line: 'Executed 3087 out of 3087 tests: 3087 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] *************
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.062) 3:27:19.305 *********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] *******
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.123) 3:27:19.429 *********
ok: [localhost] =>
tests_flaky: []
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] **********
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.038) 3:27:19.467 *********
ok: [localhost] =>
tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] ***
Saturday 01 June 2024 22:37:23 +0100 (0:00:00.039) 3:27:19.507 *********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
+ for wheel in /tf/pkg/*.whl
+ echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...'
+ python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg
+ tee check.txt
real 1m22.581s
user 1m8.234s
sys 0m13.796s
stderr_lines: <omitted>
stdout: |-
Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...
INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel
raise ValueError(
ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main
rval = args.func(args, p)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute
out_wheel = repair_wheel(
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel
strip_symbols(itertools.chain(libs_to_strip, extensions))
File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__
dir2zip(self.name, self.out_wheel, date_time)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip
with zipfile.ZipFile(zip_fname, "w", compression=compression) as z:
File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__
self.fp = io.open(file, filemode)
PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl'
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Saturday 01 June 2024 22:38:47 +0100 (0:01:23.357) 3:28:42.864 *********
===============================================================================
tensorflow_in_container : (Python 3.9) run TensorFlow git tests ------ 7511.87s
tensorflow_in_container : (Python 3.9) build TensorFlow git ---------- 3898.85s
tensorflow_in_container : (Python 3.9) start build container ---------- 940.31s
tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 83.36s
Cleanup after previous build ------------------------------------------- 36.54s
tensorflow : Fetch TensorFlow from git --------------------------------- 23.26s
tensorflow : Remove TensorFlow git sources from previous build ---------- 4.43s
Gathering Facts --------------------------------------------------------- 2.63s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 2.37s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 1.86s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.49s
Create cache dirs ------------------------------------------------------- 1.41s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.21s
tensorflow : Save git commit hash to file ------------------------------- 1.14s
tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 0.91s
tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.87s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.87s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.82s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.80s
tensorflow_in_container : (Python 3.9) create script to run tests ------- 0.70s
Build step 'Execute shell' marked build as failure
1 month, 3 weeks
- 1
- 2
Build failed in Jenkins: Tensorflow oneDNN Python package (manylinux2014 nightly) #807
by ci_notify＠linaro.org
See <https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-onednn-nightly/8…>
Changes:
------------------------------------------
[...truncated 7.14 KiB...]
TASK [tensorflow : Record git commit hash to variable] *************************
Tuesday 21 May 2024 01:11:31 +0100 (0:00:00.035) 0:01:26.096 ***********
ok: [localhost]
TASK [tensorflow : Save git commit hash to file] *******************************
Tuesday 21 May 2024 01:11:31 +0100 (0:00:00.055) 0:01:26.152 ***********
changed: [localhost]
TASK [Build TensorFlow for each Python version (dual abi)] *********************
Tuesday 21 May 2024 01:11:33 +0100 (0:00:01.107) 0:01:27.260 ***********
TASK [tensorflow_in_container : Reset checkout to clean state] *****************
Tuesday 21 May 2024 01:11:33 +0100 (0:00:00.433) 0:01:27.693 ***********
changed: [localhost]
TASK [tensorflow_in_container : Remove TensorFlow-oneDNN-ACL git sources from previous build] ***
Tuesday 21 May 2024 01:11:34 +0100 (0:00:00.650) 0:01:28.344 ***********
skipping: [localhost]
TASK [Fetch TensorFlow] ********************************************************
Tuesday 21 May 2024 01:11:34 +0100 (0:00:00.029) 0:01:28.374 ***********
skipping: [localhost]
TASK [(Python 3.9) patch TensorFlow tree] **************************************
Tuesday 21 May 2024 01:11:34 +0100 (0:00:00.030) 0:01:28.405 ***********
TASK [tensorflow : Create linklibs shell script] *******************************
Tuesday 21 May 2024 01:11:34 +0100 (0:00:00.081) 0:01:28.487 ***********
changed: [localhost]
TASK [tensorflow : Use nightly package requirements] ***************************
Tuesday 21 May 2024 01:11:34 +0100 (0:00:00.686) 0:01:29.173 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) copy upper constraints for hermetic builds] ***
Tuesday 21 May 2024 01:11:35 +0100 (0:00:00.632) 0:01:29.805 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python versioned container] ***
Tuesday 21 May 2024 01:11:36 +0100 (0:00:00.682) 0:01:30.488 ***********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) start build container] ************
Tuesday 21 May 2024 01:11:37 +0100 (0:00:01.717) 0:01:32.205 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir] ***
Tuesday 21 May 2024 01:11:56 +0100 (0:00:18.198) 0:01:50.404 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazel cache] ******
Tuesday 21 May 2024 01:11:57 +0100 (0:00:01.212) 0:01:51.617 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for bazelisk cache] ***
Tuesday 21 May 2024 01:11:58 +0100 (0:00:00.842) 0:01:52.460 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create link for pip cache] ********
Tuesday 21 May 2024 01:11:59 +0100 (0:00:00.816) 0:01:53.276 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build] ***
Tuesday 21 May 2024 01:11:59 +0100 (0:00:00.899) 0:01:54.176 ***********
changed: [localhost]
TASK [tensorflow_in_container : Show list of installed Python packages for build] ***
Tuesday 21 May 2024 01:12:03 +0100 (0:00:03.435) 0:01:57.612 ***********
ok: [localhost] =>
pip_list_output.stdout: |-
Package Version Location Installer
------------------- -------------------- -------------------------------------- ---------
auditwheel 5.4.0 /usr/local/lib/python3.9/dist-packages pip
certifi 2019.11.28 /usr/lib/python3/dist-packages
cffi 1.16.0 /usr/local/lib/python3.9/dist-packages pip
chardet 3.0.4 /usr/lib/python3/dist-packages
cryptography 42.0.5 /usr/local/lib/python3.9/dist-packages pip
Cython 3.0.9 /usr/local/lib/python3.9/dist-packages pip
dbus-python 1.2.16 /usr/lib/python3/dist-packages
docutils 0.20.1 /usr/local/lib/python3.9/dist-packages pip
idna 2.8 /usr/lib/python3/dist-packages
importlib-metadata 7.0.1 /usr/local/lib/python3.9/dist-packages pip
jaraco.classes 3.3.1 /usr/local/lib/python3.9/dist-packages pip
jeepney 0.8.0 /usr/local/lib/python3.9/dist-packages pip
keyring 24.3.1 /usr/local/lib/python3.9/dist-packages pip
markdown-it-py 3.0.0 /usr/local/lib/python3.9/dist-packages pip
mdurl 0.1.2 /usr/local/lib/python3.9/dist-packages pip
more-itertools 10.2.0 /usr/local/lib/python3.9/dist-packages pip
nh3 0.2.15 /usr/local/lib/python3.9/dist-packages pip
pip 24.0 /usr/local/lib/python3.9/dist-packages pip
pkginfo 1.10.0 /usr/local/lib/python3.9/dist-packages pip
portpicker 1.6.0 /usr/local/lib/python3.9/dist-packages pip
psutil 5.9.8 /usr/local/lib/python3.9/dist-packages pip
pycparser 2.21 /usr/local/lib/python3.9/dist-packages pip
pyelftools 0.30 /usr/local/lib/python3.9/dist-packages pip
Pygments 2.17.2 /usr/local/lib/python3.9/dist-packages pip
PyGObject 3.36.0 /usr/lib/python3/dist-packages
python-apt 2.0.1+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
readme_renderer 43.0 /usr/local/lib/python3.9/dist-packages pip
requests 2.22.0 /usr/lib/python3/dist-packages
requests-toolbelt 1.0.0 /usr/local/lib/python3.9/dist-packages pip
requests-unixsocket 0.2.0 /usr/lib/python3/dist-packages
rfc3986 2.0.0 /usr/local/lib/python3.9/dist-packages pip
rich 13.7.1 /usr/local/lib/python3.9/dist-packages pip
SecretStorage 3.3.3 /usr/local/lib/python3.9/dist-packages pip
setuptools 69.1.1 /usr/local/lib/python3.9/dist-packages pip
six 1.14.0 /usr/lib/python3/dist-packages
twine 4.0.2 /usr/local/lib/python3.9/dist-packages pip
urllib3 2.2.1 /usr/local/lib/python3.9/dist-packages pip
wheel 0.42.0 /usr/local/lib/python3.9/dist-packages pip
zipp 3.17.0 /usr/local/lib/python3.9/dist-packages pip
TASK [tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git] ***
Tuesday 21 May 2024 01:12:03 +0100 (0:00:00.037) 0:01:57.649 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) build Python package] *************
Tuesday 21 May 2024 02:45:32 +0100 (1:33:28.823) 1:35:26.473 ***********
skipping: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Copy built wheel] *****************
Tuesday 21 May 2024 02:45:32 +0100 (0:00:00.032) 1:35:26.505 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) cleanup Bazel install directory] ***
Tuesday 21 May 2024 02:45:33 +0100 (0:00:01.010) 1:35:27.516 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) create script to run tests] *******
Tuesday 21 May 2024 02:45:34 +0100 (0:00:00.964) 1:35:28.481 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests] ***
Tuesday 21 May 2024 02:45:34 +0100 (0:00:00.706) 1:35:29.187 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) collect list of installed Python packages] ***
Tuesday 21 May 2024 04:54:45 +0100 (2:09:10.985) 3:44:40.173 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save list of installed Python packages] ***
Tuesday 21 May 2024 04:54:48 +0100 (0:00:02.215) 3:44:42.388 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests output to file] ********
Tuesday 21 May 2024 04:54:48 +0100 (0:00:00.712) 3:44:43.101 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) save tests stderr output to file] ***
Tuesday 21 May 2024 04:54:49 +0100 (0:00:00.712) 3:44:43.813 ***********
changed: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Extract number of tests run] ******
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.724) 3:44:44.537 ***********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show number of tests run] *********
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.079) 3:44:44.617 ***********
ok: [localhost] =>
ut_tests_line: 'Executed 3085 out of 3085 tests: 3085 tests pass.'
TASK [tensorflow_in_container : (Python 3.9) Extract failed tests] *************
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.041) 3:44:44.658 ***********
ok: [localhost]
TASK [tensorflow_in_container : (Python 3.9) Show which tests are flaky] *******
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.131) 3:44:44.790 ***********
ok: [localhost] =>
tests_flaky:
- //tensorflow/core/kernels:batch_kernels_test FLAKY, failed in 1 out of 2 in 48.5s
- //tensorflow/core/kernels/mkl:mkl_qmatmul_op_test FLAKY, failed in 1 out of 2 in 0.5s
TASK [tensorflow_in_container : (Python 3.9) Show which tests failed] **********
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.041) 3:44:44.832 ***********
ok: [localhost] =>
tests_failed: []
TASK [tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one] ***
Tuesday 21 May 2024 04:54:50 +0100 (0:00:00.041) 3:44:44.873 ***********
fatal: [localhost]: FAILED! => changed=true
rc: 1
stderr: |-
+ for wheel in /tf/pkg/*.whl
+ echo 'Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...'
+ python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl --wheel-dir /tf/pkg
+ tee check.txt
real 1m28.486s
user 1m10.791s
sys 0m17.263s
stderr_lines: <omitted>
stdout: |-
Checking and renaming /tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl...
INFO:auditwheel.main_repair:Repairing tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 80, in repair_wheel
raise ValueError(
ValueError: Cannot repair wheel, because required library "_pywrap_tensorflow_internal.so" could not be located
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/__main__.py", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main.py", line 59, in main
rval = args.func(args, p)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/main_repair.py", line 175, in execute
out_wheel = repair_wheel(
File "/usr/local/lib/python3.9/dist-packages/auditwheel/repair.py", line 118, in repair_wheel
strip_symbols(itertools.chain(libs_to_strip, extensions))
File "/usr/local/lib/python3.9/dist-packages/auditwheel/wheeltools.py", line 137, in __exit__
dir2zip(self.name, self.out_wheel, date_time)
File "/usr/local/lib/python3.9/dist-packages/auditwheel/tools.py", line 76, in dir2zip
with zipfile.ZipFile(zip_fname, "w", compression=compression) as z:
File "/usr/lib/python3.9/zipfile.py", line 1248, in __init__
self.fp = io.open(file, filemode)
PermissionError: [Errno 13] Permission denied: '/tf/pkg/tensorflow_aarch64-2.17.0-cp39-cp39-linux_aarch64.whl'
stdout_lines: <omitted>
PLAY RECAP *********************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Tuesday 21 May 2024 04:56:19 +0100 (0:01:29.341) 3:46:14.214 ***********
===============================================================================
tensorflow_in_container : (Python 3.9) run TensorFlow-oneDNN-ACL git tests 7750.99s
tensorflow_in_container : (Python 3.9) build TensorFlow-oneDNN-ACL git 5608.82s
tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one -- 89.34s
Cleanup after previous build ------------------------------------------- 47.84s
tensorflow : Fetch TensorFlow-oneDNN-ACL from git ---------------------- 23.31s
tensorflow_in_container : (Python 3.9) start build container ----------- 18.20s
tensorflow : Remove TensorFlow-oneDNN-ACL git sources from previous build --- 9.40s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages for build --- 3.44s
Gathering Facts --------------------------------------------------------- 2.66s
tensorflow_in_container : (Python 3.9) collect list of installed Python packages --- 2.22s
tensorflow_in_container : (Python 3.9) build Python versioned container --- 1.72s
Create cache dirs ------------------------------------------------------- 1.44s
tensorflow_in_container : (Python 3.9) create buildslave bazel cache dir --- 1.21s
tensorflow : Save git commit hash to file ------------------------------- 1.11s
tensorflow_in_container : (Python 3.9) Copy built wheel ----------------- 1.01s
tensorflow_in_container : (Python 3.9) cleanup Bazel install directory --- 0.96s
tensorflow_in_container : (Python 3.9) create link for pip cache -------- 0.90s
tensorflow_in_container : (Python 3.9) create link for bazel cache ------ 0.84s
tensorflow_in_container : (Python 3.9) create link for bazelisk cache --- 0.82s
tensorflow_in_container : (Python 3.9) save tests stderr output to file --- 0.72s
Build step 'Execute shell' marked build as failure
2 months
- 1
- 3