Description: krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb}

Log: http://qa-proxy.ceph.com/teuthology/dis-2022-02-02_10:19:01-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6655264/teuthology.log

Sentry event: https://sentry.ceph.com/organizations/ceph/?query=459d38f14ca04e3d9354606871d3faa4

Failure Reason:

Failure object was: {'smithi121.front.sepia.ceph.com': {'cache_updated': False, 'stdout': "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 95 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nErr:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.2\n Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nErr:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.2\n Unable to connect to archive.ubuntu.com:http:\nErr:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nIgn:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19\nErr:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\n", 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'autoclean': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1643799204, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'stdout_lines': ['Reading package lists...', 'Building dependency tree...', 'Reading state information...', 'git is already the newest version (1:2.25.1-1ubuntu3.2).', 'The following packages were automatically installed and are no longer required:', ' libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic', ' linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic', ' linux-modules-extra-5.4.0-66-generic', "Use 'sudo apt autoremove' to remove them.", 'The following additional packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data', ' qemu-system-gui qemu-utils', 'Suggested packages:', ' samba vde2 debootstrap', 'The following NEW packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86', ' qemu-utils', 'The following packages will be upgraded:', ' qemu-system-data qemu-system-gui', '2 upgraded, 6 newly installed, 0 to remove and 95 not upgraded.', 'Need to get 14.3 MB of archives.', 'After this operation, 62.7 MB of additional disk space will be used.', 'Err:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.2', ' Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out', 'Err:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.2', ' Unable to connect to archive.ubuntu.com:http:', 'Err:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Ign:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19', 'Err:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:'], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time')

  • log_href: http://qa-proxy.ceph.com/teuthology/dis-2022-02-02_10:19:01-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6655264/teuthology.log
  • archive_path: /home/teuthworker/archive/dis-2022-02-02_10:19:01-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6655264
  • description: krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb}
  • duration: 0:15:29
  • email: idryomov@gmail.com
  • failure_reason: Failure object was: {'smithi121.front.sepia.ceph.com': {'cache_updated': False, 'stdout': "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.25.1-1ubuntu3.2).\nThe following packages were automatically installed and are no longer required:\n libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic\n linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic\n linux-modules-extra-5.4.0-66-generic\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n qemu-system-data qemu-system-gui\n2 upgraded, 6 newly installed, 0 to remove and 95 not upgraded.\nNeed to get 14.3 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nErr:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.2\n Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nErr:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.2\n Unable to connect to archive.ubuntu.com:http:\nErr:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nIgn:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19\nErr:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\nErr:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19\n Unable to connect to archive.ubuntu.com:http:\n", 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'autoclean': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1643799204, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'stdout_lines': ['Reading package lists...', 'Building dependency tree...', 'Reading state information...', 'git is already the newest version (1:2.25.1-1ubuntu3.2).', 'The following packages were automatically installed and are no longer required:', ' libllvm11 linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic', ' linux-image-5.4.0-66-generic linux-modules-5.4.0-66-generic', ' linux-modules-extra-5.4.0-66-generic', "Use 'sudo apt autoremove' to remove them.", 'The following additional packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data', ' qemu-system-gui qemu-utils', 'Suggested packages:', ' samba vde2 debootstrap', 'The following NEW packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86', ' qemu-utils', 'The following packages will be upgraded:', ' qemu-system-data qemu-system-gui', '2 upgraded, 6 newly installed, 0 to remove and 95 not upgraded.', 'Need to get 14.3 MB of archives.', 'After this operation, 62.7 MB of additional disk space will be used.', 'Err:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.14-0ubuntu0.20.04.2', ' Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out', 'Err:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.14-0ubuntu0.20.04.2', ' Unable to connect to archive.ubuntu.com:http:', 'Err:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Ign:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19', 'Err:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:', 'Err:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.19', ' Unable to connect to archive.ubuntu.com:http:'], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.14-0ubuntu0.20.04.2_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.152), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::23), connection timed out Could not connect to archive.ubuntu.com:80 (91.189.88.142), connection timed out Could not connect to archive.ubuntu.com:80 (2001:67c:1360:8001::24), connection timed out', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librbd1_15.2.14-0ubuntu0.20.04.2_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-block-extra_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-common_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-data_4.2-3ubuntu6.19_all.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-gui_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-system-x86_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/q/qemu/qemu-utils_4.2-3ubuntu6.19_amd64.deb Unable to connect to archive.ubuntu.com:http:', 'E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time')
  • flavor:
  • job_id: 6655264
  • kernel:
    • flavor: default
    • kdb: True
    • sha1: 4f95d4823dcdf7fd4db2b7538d1fdfa7513104e4
  • last_in_suite: False
  • machine_type: smithi
  • name: dis-2022-02-02_10:19:01-krbd-wip-krbd-rxbounce-option-testing-default-smithi
  • nuke_on_error: True
  • os_type:
  • os_version:
  • overrides:
    • admin_socket:
      • branch: wip-krbd-rxbounce-option
    • ceph:
      • conf:
        • client:
          • rbd default features: 37
          • rbd default map options: ms_mode=legacy
        • global:
          • mon client directed command retry: 5
          • ms die on skipped message: False
          • ms inject socket failures: 500
        • mgr:
          • debug mgr: 20
          • debug ms: 1
        • mon:
          • debug mon: 20
          • debug ms: 1
          • debug paxos: 20
        • osd:
          • bdev async discard: True
          • bdev enable discard: True
          • bluestore allocator: bitmap
          • bluestore block size: 96636764160
          • bluestore fsck on mount: True
          • debug bluefs: 1/20
          • debug bluestore: 1/20
          • debug ms: 1
          • debug osd: 20
          • debug rocksdb: 4/10
          • mon osd backfillfull_ratio: 0.85
          • mon osd full ratio: 0.9
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
          • osd objectstore: bluestore
          • osd shutdown pgref assert: True
      • flavor: default
      • fs: xfs
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • \(OSD_SLOW_PING_TIME
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • sha1: 6fc1c038150b5e8678c937d92256553681f5b7e4
    • ceph-deploy:
      • bluestore: True
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
        • osd:
          • bdev async discard: True
          • bdev enable discard: True
          • bluestore block size: 96636764160
          • bluestore fsck on mount: True
          • debug bluefs: 1/20
          • debug bluestore: 1/20
          • debug rocksdb: 4/10
          • mon osd backfillfull_ratio: 0.85
          • mon osd full ratio: 0.9
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
          • osd objectstore: bluestore
      • fs: xfs
    • install:
      • ceph:
        • flavor: default
        • sha1: 6fc1c038150b5e8678c937d92256553681f5b7e4
    • thrashosds:
      • bdev_inject_crash: 2
      • bdev_inject_crash_probability: 0.5
    • workunit:
      • branch: wip-krbd-rxbounce-option
      • sha1: 239f3b058ecd10b05894b75aab82b58c444cda60
  • owner: scheduled_dis@teuthology
  • pid:
  • roles:
    • ['mon.a', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
    • ['mon.b', 'mgr.y', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
    • ['client.0']
  • sentry_event: https://sentry.ceph.com/organizations/ceph/?query=459d38f14ca04e3d9354606871d3faa4
  • status: dead
  • success: False
  • branch: wip-krbd-rxbounce-option
  • seed:
  • sha1: 6fc1c038150b5e8678c937d92256553681f5b7e4
  • subset:
  • suite:
  • suite_branch: wip-krbd-rxbounce-option
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 239f3b058ecd10b05894b75aab82b58c444cda60
  • targets:
    • smithi083.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHXRIy3pS0e5WiirUg2dhRXVqAKBRu92Gt4XKYSqdatz76DtahfI6M2JFhaAVxnfzRll+j4+Vl89N7bldOgwSbuRUw0nX9Smf4KXiKnHSX1b3YXVY0egqRohYL5zSxDvBOFUWc+jGlM6AGyFzyAL+7Fg35lSIIklygQWgFSHHa6WrPILHbNQ6VV3q5ClVZmuQDoxgqjC3MjGjnzLQodLHH4BMCNI+B7OW9gsez2Sak+J54e82lIE1J51/fBKncFeKg19Fp6ZijgdbDbgL1dSX53mSNGVEPcVXO96ew5QDGmrVPb6KWfBdx3HU5xdWApsWo+Vlc/5jRH0OZxLfdV8QbYwElozpraxMoxDJfSf5Es9kow1mMD2Yk8awNp1iRcDM+ritXQtC1HMXGZDjSXwmb1xcQUlcO+c7ryv9+kunldUG75qXZiZM7NqiOx31jbFIYnv2/RYFA4+wbHCJfYD8HQW/QtRXbNczmqT+an5B2qoKqh/VA+S1fWflw8GSdy+s=
    • smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCssa9EOa80wnYqpyzDUWGEgWbyOAbhSNF7zxjFz/uQ8576TzacGEczqysRPmSX4kydKqHpQjrndxtFKbyOOXottO297uDoOwgmZmeKWWmdbRGUYH3iGGAfNqDNRbcLj56iu4GXfjMti8sW/xrzN19jq9hOPNq+8Ik7CJGyduyzSVQGW2/fiAngtl3oieXcTRzeWt1Xy5HGmmvVWA2o37A2ttDHknUbG5+Pwt379BfLbkkMhRYFxNLio18Kn1STlMr/h9sDtQRKl3HA1qxtiSN6NfTtL3V69iJJdMzzVrTDxVaabPRyHoSYb36aFAJxmvbz483iPBxAJj+2mIUndeRh5Fk76sf8yeNznsOhaObNBvDUM4n0sMWPhNns092YlbCOKIaYWaZv+nGrX0kbf6gRTqMgrK2jHXnY+F2gUbpWLdmkCwsg32hDB8S1V3AaVnQvaYgR1w2HKV5JcIDLY3xT02a+jalzBx/SUGAPIXK/8YAoAOUvB/QCm0UjfIQJps8=
    • smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPTmBDk0WjmjDDFq3SAiB4i672uxQBShdYFd/M96Nj5Sww9shfPKAVvQzSE0uSSNK1i++IldhMm1tDdYeeieL0n4UUAlP9qIfBxflD3kq4d5/uFYSQU7bckqv8x67yCU8yP04tub8pWzGFwBb0RcozZIq61sDHAvrkg6ByP+9SW1o7dHAdxt4LbdD1gln+MkIOiAnukz4DTgW6niY6XoAKCTLBhIlM/kKpNYqEuGyBhv6M9aj5zXagNPwbM+oaDRWJu+yfF/WLgfekgPtiIrvQHBFdPb+Lc5qFbn9g5gTavOpG/rieDS40uV4ZjY4KgR56ulMTS6kMsOpvclB/NGEnqKxRC/c0/dQbqbBvcApRt5iVckobKupWq4tPf0F7S90HtMoOas90cRJpgv6nqOWlJDoZrSEOxIxE/xtFe1rsu3tEATbkzFJbpUxcB4DJ8Ia0GKoJCelBe7hxknfMdqG1YCYAA6xfE8ZBvDZihWTsbXkns7/aIBvprko9cDRD4SU=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • flavor: default
      • kdb: True
      • sha1: 4f95d4823dcdf7fd4db2b7538d1fdfa7513104e4
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • install:
    • ceph:
    • rbd:
      • all:
        • image_size: 20480
    • workunit:
      • clients:
        • all:
          • suites/ffsb.sh
  • teuthology_branch: master
  • verbose: True
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2022-02-02 10:19:25
  • started: 2022-02-02 10:39:49
  • updated: 2022-02-02 11:08:29
  • status_class: danger
  • runtime: 0:28:40
  • wait_time: 0:13:11