See https://ci.linaro.org/job/ldcg-python-manylinux-tensorflow-nightly/711/display/redirect
Changes:
------------------------------------------ [...truncated 5.13 MB...] Stats over 16 runs: max = 20.7s, min = 4.5s, avg = 14.2s, dev = 4.8s //tensorflow/python/data/kernel_tests:snapshot_test PASSED in 45.7s Stats over 16 runs: max = 45.7s, min = 24.1s, avg = 35.4s, dev = 5.7s //tensorflow/python/kernel_tests/control_flow:control_flow_ops_py_test_cpu PASSED in 30.6s Stats over 16 runs: max = 30.6s, min = 10.5s, avg = 13.0s, dev = 4.7s //tensorflow/python/kernel_tests/linalg:matrix_exponential_op_test PASSED in 12.1s Stats over 16 runs: max = 12.1s, min = 4.2s, avg = 7.5s, dev = 1.9s //tensorflow/python/kernel_tests/signal:dct_ops_test_cpu PASSED in 17.2s Stats over 16 runs: max = 17.2s, min = 13.2s, avg = 15.4s, dev = 1.1s //tensorflow/python/ops/parallel_for:control_flow_ops_test_cpu PASSED in 66.2s Stats over 16 runs: max = 66.2s, min = 14.7s, avg = 25.0s, dev = 11.3s //tensorflow/python/kernel_tests/linalg:normalize_op_test_cpu INCOMPLETE in 14.0s Stats over 18 runs: max = 14.0s, min = 5.9s, avg = 9.7s, dev = 2.3s /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/python/kernel_tests/linalg/normalize_op_test_cpu/shard_18_of_20/test_attempts/attempt_1.log //tensorflow/compiler/tests:pooling_ops_3d_test_cpu PASSED in 6.4s Stats over 20 runs: max = 6.4s, min = 2.8s, avg = 4.0s, dev = 1.3s //tensorflow/compiler/tests:pooling_ops_3d_test_cpu_mlir_bridge_test PASSED in 7.1s Stats over 20 runs: max = 7.1s, min = 4.0s, avg = 5.3s, dev = 1.2s //tensorflow/compiler/tests:pooling_ops_test_cpu PASSED in 10.2s Stats over 20 runs: max = 10.2s, min = 3.1s, avg = 4.9s, dev = 1.9s //tensorflow/compiler/tests:pooling_ops_test_cpu_mlir_bridge_test PASSED in 10.2s Stats over 20 runs: max = 10.2s, min = 3.7s, avg = 5.8s, dev = 1.7s //tensorflow/compiler/xla/tests:convolution_dimension_numbers_test_cpu PASSED in 8.8s Stats over 20 runs: max = 8.8s, min = 7.0s, avg = 7.7s, dev = 0.5s //tensorflow/compiler/xla/tests:dot_operation_single_threaded_runtime_test_cpu PASSED in 11.9s Stats over 20 runs: max = 11.9s, min = 8.8s, avg = 10.7s, dev = 0.7s //tensorflow/compiler/xla/tests:dot_operation_test_cpu PASSED in 13.6s Stats over 20 runs: max = 13.6s, min = 9.7s, avg = 11.3s, dev = 0.9s //tensorflow/compiler/xla/tests:reduce_window_test_cpu PASSED in 42.8s Stats over 20 runs: max = 42.8s, min = 7.4s, avg = 15.9s, dev = 10.8s //tensorflow/python/autograph/tests:loop_control_flow_test PASSED in 24.3s Stats over 20 runs: max = 24.3s, min = 15.8s, avg = 20.5s, dev = 2.3s //tensorflow/python/kernel_tests:metrics_test PASSED in 37.3s Stats over 20 runs: max = 37.3s, min = 8.4s, avg = 18.5s, dev = 7.4s //tensorflow/python/kernel_tests/array_ops:matrix_band_part_op_test_cpu PASSED in 10.1s Stats over 20 runs: max = 10.1s, min = 6.3s, avg = 7.6s, dev = 1.1s //tensorflow/python/kernel_tests/data_structures:barrier_ops_test PASSED in 11.0s Stats over 20 runs: max = 11.0s, min = 2.8s, avg = 6.3s, dev = 2.1s //tensorflow/python/kernel_tests/linalg:eig_op_test PASSED in 46.4s Stats over 20 runs: max = 46.4s, min = 5.1s, avg = 15.2s, dev = 13.1s //tensorflow/python/kernel_tests/linalg:linalg_grad_test_cpu PASSED in 100.2s Stats over 20 runs: max = 100.2s, min = 26.4s, avg = 49.1s, dev = 19.8s //tensorflow/python/kernel_tests/linalg:norm_op_test_cpu PASSED in 7.2s Stats over 20 runs: max = 7.2s, min = 3.3s, avg = 4.7s, dev = 1.5s //tensorflow/python/kernel_tests/linalg:qr_op_test_cpu PASSED in 106.3s Stats over 20 runs: max = 106.3s, min = 33.7s, avg = 73.2s, dev = 24.2s //tensorflow/python/kernel_tests/linalg:self_adjoint_eig_op_test_cpu PASSED in 21.5s Stats over 20 runs: max = 21.5s, min = 5.2s, avg = 11.3s, dev = 5.1s //tensorflow/python/kernel_tests/math_ops:batch_matmul_op_test_cpu PASSED in 17.7s Stats over 20 runs: max = 17.7s, min = 5.6s, avg = 11.7s, dev = 4.4s //tensorflow/python/kernel_tests/math_ops:matmul_op_test_cpu PASSED in 16.4s Stats over 20 runs: max = 16.4s, min = 11.6s, avg = 14.1s, dev = 1.5s //tensorflow/python/kernel_tests/math_ops:tensordot_op_test_cpu PASSED in 60.4s Stats over 20 runs: max = 60.4s, min = 6.4s, avg = 26.1s, dev = 18.6s //tensorflow/python/kernel_tests/nn_ops:embedding_ops_test_cpu PASSED in 22.8s Stats over 20 runs: max = 22.8s, min = 5.0s, avg = 8.6s, dev = 3.8s //tensorflow/python/data/kernel_tests:map_test FAILED in 3 out of 21 in 70.7s Stats over 21 runs: max = 70.7s, min = 15.2s, avg = 24.3s, dev = 11.5s /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/python/data/kernel_tests/map_test/shard_11_of_19/test.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/python/data/kernel_tests/map_test/shard_11_of_19/test_attempts/attempt_1.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/python/data/kernel_tests/map_test/shard_11_of_19/test_attempts/attempt_2.log //tensorflow/python/data/experimental/kernel_tests/service:local_workers_test PASSED in 20.2s Stats over 24 runs: max = 20.2s, min = 4.6s, avg = 12.5s, dev = 3.4s //tensorflow/python/data/kernel_tests:interleave_test PASSED in 23.0s Stats over 24 runs: max = 23.0s, min = 7.7s, avg = 13.9s, dev = 4.3s //tensorflow/python/data/kernel_tests:sample_from_datasets_test PASSED in 20.2s Stats over 24 runs: max = 20.2s, min = 3.8s, avg = 9.8s, dev = 4.6s //tensorflow/compiler/xla/tests:array_elementwise_ops_test_cpu PASSED in 9.3s Stats over 25 runs: max = 9.3s, min = 7.0s, avg = 8.1s, dev = 0.6s //tensorflow/compiler/xla/tests:select_and_scatter_test_cpu PASSED in 34.4s Stats over 25 runs: max = 34.4s, min = 5.9s, avg = 10.1s, dev = 7.2s //tensorflow/compiler/xla/tests:conv_depthwise_backprop_filter_test_cpu INCOMPLETE in 11.7s Stats over 28 runs: max = 11.7s, min = 5.6s, avg = 7.2s, dev = 1.9s /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/xla/tests/conv_depthwise_backprop_filter_test_cpu/shard_10_of_40/test_attempts/attempt_1.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/xla/tests/conv_depthwise_backprop_filter_test_cpu/shard_40_of_40/test_attempts/attempt_1.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/xla/tests/conv_depthwise_backprop_filter_test_cpu/shard_39_of_40/test_attempts/attempt_1.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/xla/tests/conv_depthwise_backprop_filter_test_cpu/shard_1_of_40/test_attempts/attempt_1.log /home/buildslave/.cache/bazel/_bazel_buildslave/fbac33eb30dbfb6b11b15a7ff5ac830d/execroot/org_tensorflow/bazel-out/aarch64-opt/testlogs/tensorflow/compiler/xla/tests/conv_depthwise_backprop_filter_test_cpu/shard_9_of_40/test_attempts/attempt_1.log //tensorflow/compiler/xla/tests:convolution_variants_test_cpu PASSED in 9.4s Stats over 30 runs: max = 9.4s, min = 6.9s, avg = 8.2s, dev = 0.7s //tensorflow/compiler/xla/tests:iota_test_cpu PASSED in 16.2s Stats over 30 runs: max = 16.2s, min = 11.8s, avg = 13.8s, dev = 1.3s //tensorflow/compiler/xla/tests:params_test_cpu PASSED in 8.3s Stats over 30 runs: max = 8.3s, min = 6.2s, avg = 7.4s, dev = 0.5s //tensorflow/compiler/xla/tests:reshape_test_cpu PASSED in 7.5s Stats over 30 runs: max = 7.5s, min = 5.7s, avg = 6.5s, dev = 0.3s //tensorflow/compiler/xla/tests:reduce_test_cpu PASSED in 8.8s Stats over 31 runs: max = 8.8s, min = 6.0s, avg = 7.6s, dev = 0.8s //tensorflow/compiler/xla/tests:scalar_computations_test_cpu PASSED in 8.4s Stats over 32 runs: max = 8.4s, min = 5.8s, avg = 7.0s, dev = 0.6s //tensorflow/python/data/experimental/kernel_tests/service:auto_shard_test PASSED in 17.7s Stats over 32 runs: max = 17.7s, min = 5.5s, avg = 12.4s, dev = 3.1s //tensorflow/python/data/experimental/kernel_tests/service:data_service_ops_test PASSED in 30.7s Stats over 32 runs: max = 30.7s, min = 11.0s, avg = 19.5s, dev = 4.5s //tensorflow/compiler/xla/tests:batch_normalization_test_cpu PASSED in 10.1s Stats over 40 runs: max = 10.1s, min = 8.1s, avg = 9.2s, dev = 0.4s //tensorflow/compiler/xla/tests:bfloat16_test_cpu PASSED in 9.6s Stats over 40 runs: max = 9.6s, min = 8.1s, avg = 8.9s, dev = 0.3s //tensorflow/compiler/xla/tests:slice_test_cpu PASSED in 11.5s Stats over 40 runs: max = 11.5s, min = 6.4s, avg = 8.3s, dev = 1.0s //tensorflow/compiler/mlir/quantization/tensorflow/python:quantize_model_test PASSED in 96.9s Stats over 50 runs: max = 96.9s, min = 15.5s, avg = 41.8s, dev = 29.4s //tensorflow/compiler/tests:sort_ops_test_cpu PASSED in 42.3s Stats over 50 runs: max = 42.3s, min = 2.7s, avg = 11.0s, dev = 8.7s //tensorflow/compiler/tests:sort_ops_test_cpu_mlir_bridge_test PASSED in 37.6s Stats over 50 runs: max = 37.6s, min = 2.4s, avg = 9.7s, dev = 7.9s //tensorflow/compiler/xla/tests:conv_depthwise_test_cpu PASSED in 10.1s Stats over 50 runs: max = 10.1s, min = 6.7s, avg = 8.5s, dev = 0.7s //tensorflow/compiler/xla/tests:convolution_test_1d_no_vmodule_cpu PASSED in 11.3s Stats over 50 runs: max = 11.3s, min = 8.7s, avg = 9.8s, dev = 0.7s //tensorflow/compiler/xla/tests:convolution_test_cpu PASSED in 16.2s Stats over 50 runs: max = 16.2s, min = 7.8s, avg = 10.8s, dev = 2.2s //tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_dense_mat_mul_grad_test_cpu PASSED in 25.4s Stats over 50 runs: max = 25.4s, min = 3.6s, avg = 7.5s, dev = 4.0s //tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_grad_test_cpu PASSED in 7.9s Stats over 50 runs: max = 7.9s, min = 2.8s, avg = 4.5s, dev = 1.3s //tensorflow/python/kernel_tests/linalg/sparse:csr_sparse_matrix_sparse_mat_mul_grad_test_cpu PASSED in 8.4s Stats over 50 runs: max = 8.4s, min = 2.9s, avg = 4.8s, dev = 1.7s //tensorflow/python/kernel_tests/math_ops:cwise_ops_binary_test_cpu PASSED in 28.3s Stats over 50 runs: max = 28.3s, min = 6.6s, avg = 13.9s, dev = 5.4s //tensorflow/python/kernel_tests/math_ops:cwise_ops_test_cpu PASSED in 15.0s Stats over 50 runs: max = 15.0s, min = 3.1s, avg = 6.0s, dev = 2.6s //tensorflow/python/kernel_tests/math_ops:cwise_ops_unary_test_cpu PASSED in 14.2s Stats over 50 runs: max = 14.2s, min = 3.1s, avg = 5.6s, dev = 3.0s
Executed 3537 out of 3614 tests: 3529 tests pass, 38 fail to build, 8 fail locally and 39 were skipped. There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are. stdout_lines: <omitted> ...ignoring
TASK [tensorflow_in_container : (Python 3.11) collect list of installed Python packages] *** Friday 03 March 2023 12:53:51 +0000 (0:36:56.702) 4:03:51.083 ********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.11) save list of installed Python packages] *** Friday 03 March 2023 12:53:53 +0000 (0:00:02.062) 4:03:53.145 ********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.11) save tests output to file] ******* Friday 03 March 2023 12:53:54 +0000 (0:00:00.695) 4:03:53.841 ********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.11) save tests stderr output to file] *** Friday 03 March 2023 12:53:54 +0000 (0:00:00.727) 4:03:54.568 ********** changed: [localhost]
TASK [tensorflow_in_container : (Python 3.11) Extract number of tests run] ***** Friday 03 March 2023 12:53:55 +0000 (0:00:00.710) 4:03:55.279 ********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.11) Show number of tests run] ******** Friday 03 March 2023 12:53:55 +0000 (0:00:00.130) 4:03:55.409 ********** ok: [localhost] => ut_tests_line: ''
TASK [tensorflow_in_container : (Python 3.11) Extract failed tests] ************ Friday 03 March 2023 12:53:55 +0000 (0:00:00.087) 4:03:55.497 ********** ok: [localhost]
TASK [tensorflow_in_container : (Python 3.11) Show which tests failed] ********* Friday 03 March 2023 12:53:55 +0000 (0:00:00.088) 4:03:55.585 ********** ok: [localhost] => tests_failed: 'FAILED: '
TASK [tensorflow_in_container : (Python 3.11) Convert wheel to manylinux2014 compatible one] *** Friday 03 March 2023 12:53:55 +0000 (0:00:00.048) 4:03:55.633 ********** fatal: [localhost]: FAILED! => changed=true rc: 1 stderr: |- + for wheel in /tf/pkg/*.whl + echo 'Checking and renaming /tf/pkg/tensorflow-2.13.0-cp311-cp311-linux_aarch64.whl...' + python3 -m auditwheel repair --plat manylinux2014_aarch64 /tf/pkg/tensorflow-2.13.0-cp311-cp311-linux_aarch64.whl --wheel-dir /tf/pkg + tee check.txt
real 0m3.861s user 0m2.805s sys 0m0.229s stderr_lines: <omitted> stdout: |- Checking and renaming /tf/pkg/tensorflow-2.13.0-cp311-cp311-linux_aarch64.whl... INFO:auditwheel.main_repair:Repairing tensorflow-2.13.0-cp311-cp311-linux_aarch64.whl Traceback (most recent call last): File "/usr/lib/python3.11/zipfile.py", line 1732, in _extract_member shutil.copyfileobj(source, target) File "/usr/lib/python3.11/shutil.py", line 200, in copyfileobj fdst_write(buf) OSError: [Errno 28] No space left on device
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/local/lib/python3.11/dist-packages/auditwheel/__main__.py", line 6, in <module> sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/dist-packages/auditwheel/main.py", line 59, in main rval = args.func(args, p) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/auditwheel/main_repair.py", line 125, in execute wheel_abi = analyze_wheel_abi(wheel_file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/auditwheel/wheel_abi.py", line 244, in analyze_wheel_abi ) = get_wheel_elfdata(wheel_fn) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/auditwheel/wheel_abi.py", line 69, in get_wheel_elfdata with InGenericPkgCtx(wheel_fn) as ctx: File "/usr/local/lib/python3.11/dist-packages/auditwheel/wheeltools.py", line 170, in __enter__ self.path = super().__enter__() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/auditwheel/wheeltools.py", line 120, in __enter__ zip2dir(self.in_wheel, self.name) File "/usr/local/lib/python3.11/dist-packages/auditwheel/tools.py", line 42, in zip2dir extracted_path = z.extract(member, out_dir) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/zipfile.py", line 1660, in extract return self._extract_member(member, path, pwd) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/zipfile.py", line 1730, in _extract_member with self.open(member, pwd=pwd) as source, \ OSError: [Errno 28] No space left on device stdout_lines: <omitted>
PLAY RECAP ********************************************************************* localhost : ok=144 changed=108 unreachable=0 failed=1 skipped=14 rescued=0 ignored=4
Friday 03 March 2023 12:54:00 +0000 (0:00:04.760) 4:04:00.393 ********** =============================================================================== tensorflow_in_container : (Python 3.8) run TensorFlow git tests ------ 4041.48s tensorflow_in_container : (Python 3.8) build TensorFlow git ---------- 2823.84s tensorflow_in_container : (Python 3.11) run TensorFlow git tests ----- 2216.70s tensorflow_in_container : (Python 3.10) run TensorFlow git tests ----- 1378.08s tensorflow_in_container : (Python 3.9) run TensorFlow git tests ------ 1373.12s tensorflow_in_container : (Python 3.8) Convert wheel to manylinux2014 compatible one - 460.60s tensorflow_in_container : (Python 3.9) Convert wheel to manylinux2014 compatible one - 277.95s tensorflow_in_container : (Python 3.10) Convert wheel to manylinux2014 compatible one - 274.18s tensorflow_in_container : (Python 3.10) stop build container ---------- 246.52s tensorflow_in_container : (Python 3.8) stop build container ----------- 238.78s tensorflow_in_container : (Python 3.9) stop build container ----------- 217.31s tensorflow_in_container : (Python 3.9) build TensorFlow git ----------- 189.25s tensorflow_in_container : (Python 3.11) build TensorFlow git ---------- 186.43s tensorflow_in_container : (Python 3.10) build TensorFlow git ---------- 185.98s tensorflow_in_container : (Python 3.9) build Python package ----------- 103.25s tensorflow_in_container : (Python 3.11) build Python package ---------- 101.73s tensorflow_in_container : (Python 3.8) build Python package ------------ 99.00s tensorflow_in_container : (Python 3.10) build Python package ----------- 97.37s tensorflow : Fetch TensorFlow from git --------------------------------- 23.84s tensorflow_in_container : (Python 3.11) Convert wheel to manylinux2014 compatible one --- 4.76s Build step 'Execute shell' marked build as failure