Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi031.front.sepia.ceph.com smithi True True 2021-03-06 06:37:56.899785 scheduled_teuthology@teuthology ubuntu 16.04 x86_64 /home/teuthworker/archive/teuthology-2021-03-06_04:20:24-upgrade:mimic-x-nautilus-distro-basic-smithi/5941223
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
running 5941223 2021-03-06 04:21:03 2021-03-06 06:37:56 2021-03-06 07:31:18 0:54:11 smithi master ubuntu 16.04 upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04} 4
pass 5941209 2021-03-06 04:20:52 2021-03-06 05:38:13 2021-03-06 06:37:49 0:59:36 0:51:32 0:08:04 smithi master rhel 7.9 upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} 5
fail 5941194 2021-03-06 04:20:46 2021-03-06 05:12:41 2021-03-06 05:36:05 0:23:24 0:09:58 0:13:26 smithi master centos 7.8 upgrade:luminous-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/luminous 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-msgr2 6-nautilus 7-final-workload/{rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} 5
Failure Reason:

Command failed on smithi002 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5941188 2021-03-06 04:20:43 2021-03-06 04:59:07 2021-03-06 05:12:59 0:13:52 0:05:49 0:08:03 smithi master ubuntu 18.04 upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/luminous 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 7-final-workload objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} 5
Failure Reason:

Command failed on smithi002 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5941174 2021-03-06 04:20:37 2021-03-06 04:43:39 2021-03-06 04:59:34 0:15:55 0:06:14 0:09:41 smithi master ubuntu 18.04 upgrade:luminous-x/parallel/{0-cluster/{openstack start} 1-ceph-install/luminous 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest} 4
Failure Reason:

Command failed on smithi002 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5941153 2021-03-06 04:20:25 2021-03-06 04:21:29 2021-03-06 04:46:05 0:24:36 0:10:07 0:14:29 smithi master centos 7.8 upgrade:luminous-x/parallel/{0-cluster/{openstack start} 1-ceph-install/luminous 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/bluestore-bitmap supported-all-distro/centos_latest} 4
Failure Reason:

Command failed on smithi002 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

pass 5941047 2021-03-06 03:34:31 2021-03-06 03:40:37 2021-03-06 04:01:07 0:20:30 0:11:03 0:09:27 smithi py2 centos 7.6 upgrade:client-upgrade-mimic/mimic-client-x/rbd/{0-cluster/{openstack start} 1-install/mimic-client-x 2-features/defaults 3-workload/rbd_notification_tests supported/centos_7.6} 2
pass 5940653 2021-03-06 03:16:46 2021-03-06 03:59:52 2021-03-06 04:22:51 0:22:59 0:10:30 0:12:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 5940558 2021-03-05 23:59:27 2021-03-06 02:32:20 2021-03-06 02:54:14 0:21:54 0:15:13 0:06:41 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} 2
pass 5940524 2021-03-05 23:58:57 2021-03-06 02:15:39 2021-03-06 02:31:32 0:15:53 0:08:41 0:07:12 smithi master rados/verify/{ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/lockdep} 2
pass 5940470 2021-03-05 23:58:14 2021-03-06 01:54:43 2021-03-06 02:15:36 0:20:53 0:12:14 0:08:39 smithi master ubuntu rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zstd thrashers/default thrashosds-health workloads/rbd_fsx_nbd} 3
pass 5940408 2021-03-05 23:57:42 2021-03-06 01:24:17 2021-03-06 01:55:39 0:31:22 0:18:56 0:12:26 smithi master ubuntu 16.04 rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/copy-on-read msgr-failures/few objectstore/bluestore-comp-snappy pool/replicated-data-pool supported-random-distro$/{ubuntu_16.04} workloads/fsx} 3
pass 5940340 2021-03-05 23:57:11 2021-03-06 00:56:05 2021-03-06 01:24:25 0:28:20 0:20:07 0:08:13 smithi master centos 7.8 rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} tasks/rbd_python_api_tests_old_format} 1
pass 5940284 2021-03-05 23:56:50 2021-03-06 00:37:18 2021-03-06 00:56:03 0:18:45 0:13:43 0:05:02 smithi master rhel 7.9 rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-lz4 overrides rgw_pool_type/replicated supported-random-distro$/{rhel_7}} 2
fail 5939893 2021-03-05 21:11:53 2021-03-05 21:50:02 2021-03-05 22:15:24 0:25:22 0:12:38 0:12:44 smithi master centos 8.2 fs:workload/{begin clusters/1a5s-mds-1c-client-3node distro/{centos_8} mount objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} 5
Failure Reason:

Command failed on smithi118 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8b5bae19e5f93cfda58001b5b8f2affb03e2a736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6c0235ce-7dff-11eb-9069-001a4aab830c -- ceph orch daemon add mon smithi118:172.21.15.118=smithi118'

dead 5939878 2021-03-05 21:11:26 2021-03-05 21:34:51 2021-03-05 21:50:01 0:15:10 0:06:18 0:08:52 smithi master rhel 8.3 fs:workload/{begin clusters/1a5s-mds-1c-client-3node distro/{rhel_8} mount objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} 5
Failure Reason:

{'Failure object was': {'smithi131.front.sepia.ceph.com': {'cmd': ['/bin/pip3', 'install', '-U', 'git+https://github.com/sebastian-philipp/registries-conf-ctl'], 'msg': 'stdout: Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-l46hd3ts-build\\nCollecting toml (from registries-conf-ctl==1.0)\\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\\nCollecting docopt (from registries-conf-ctl==1.0)\\n\\n:stderr: WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\\n HTTP error 502 while getting https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz#sha256=49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491 (from https://pypi.org/simple/docopt/)\\n Could not install requirement docopt from https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz#sha256=49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491 (from registries-conf-ctl==1.0) because of error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\\nCould not install requirement docopt from https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz#sha256=49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491 (from registries-conf-ctl==1.0) because of HTTP error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz for URL https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz#sha256=49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491 (from https://pypi.org/simple/docopt/)\\n', 'invocation': {'module_args': {'name': ['git+https://github.com/sebastian-philipp/registries-conf-ctl'], 'state': 'latest', 'virtualenv_site_packages': False, 'virtualenv_command': 'virtualenv', 'editable': False, 'version': 'None', 'requirements': 'None', 'virtualenv': 'None', 'virtualenv_python': 'None', 'extra_args': 'None', 'chdir': 'None', 'executable': 'None', 'umask': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'cmd')"}

pass 5939849 2021-03-05 21:02:20 2021-03-05 21:14:02 2021-03-05 21:36:37 0:22:35 0:12:35 0:10:00 smithi master ubuntu rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} 2
pass 5939678 2021-03-05 14:55:21 2021-03-05 19:55:21 2021-03-05 20:27:17 0:31:56 0:24:25 0:07:31 smithi master rhel 8.3 rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} 2
pass 5939625 2021-03-05 14:54:38 2021-03-05 19:32:31 2021-03-05 19:56:01 0:23:30 0:12:18 0:11:12 smithi master centos 8.2 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados tasks/mon_recovery validater/lockdep} 2
pass 5939547 2021-03-05 14:53:39 2021-03-05 18:54:31 2021-03-05 19:32:40 0:38:09 0:26:30 0:11:39 smithi master centos 8.2 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} 2