Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi146.front.sepia.ceph.com smithi True True 2020-08-04 16:21:09.307345 scheduled_dang@sidious.arb.redhat.com ubuntu 18.04 x86_64 /home/teuthworker/archive/dang-2020-08-04_12:08:55-rgw-wip-dang-zipper-7-distro-basic-smithi/5286598
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
running 5286598 2020-08-04 16:09:16 2020-08-04 16:20:38 2020-08-04 16:42:38 0:23:21 smithi master ubuntu rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} 2
fail 5286573 2020-08-04 15:44:28 2020-08-04 15:54:23 2020-08-04 16:20:21 0:25:58 0:15:54 0:10:04 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
Failure Reason:

Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS)

fail 5286513 2020-08-04 13:02:49 2020-08-04 13:04:00 2020-08-04 13:20:00 0:16:00 0:08:24 0:07:36 smithi master centos 8.1 rgw:sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/{0-install webidentity_sample}} 2
Failure Reason:

name 'remote' is not defined

pass 5286402 2020-08-04 10:28:12 2020-08-04 11:56:47 2020-08-04 12:30:47 0:34:00 0:22:29 0:11:31 smithi master centos 8.0 rados:cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} 2
pass 5286342 2020-08-04 10:20:18 2020-08-04 11:04:50 2020-08-04 11:34:50 0:30:00 0:21:56 0:08:04 smithi master centos 8.1 rados:cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async start tasks/rados_python} 2
pass 5286332 2020-08-04 10:20:10 2020-08-04 10:48:55 2020-08-04 12:00:56 1:12:01 0:21:05 0:50:56 smithi master centos 8.1 rados:cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_python} 2
pass 5286295 2020-08-04 08:00:44 2020-08-04 08:15:09 2020-08-04 09:23:10 1:08:01 0:23:28 0:44:33 smithi master centos 8.1 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/rados_python} 3
dead 5286293 2020-08-04 08:00:43 2020-08-04 08:15:09 2020-08-04 08:43:08 0:27:59 0:03:17 0:24:42 smithi master ubuntu 18.04 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} 3
Failure Reason:

{'Failure object was': {'smithi065.front.sepia.ceph.com': {'results': [{'msg': 'Failed to download key at http://download.ceph.com/keys/autobuild.asc: Connection failure: timed out', 'failed': True, 'invocation': {'module_args': {'file': 'None', 'keyserver': 'None', 'url': 'http://download.ceph.com/keys/autobuild.asc', 'validate_certs': True, 'keyring': 'None', 'state': 'present', 'key': 'None', 'data': 'None', 'id': 'None'}}, '_ansible_no_log': False, 'changed': False, 'item': 'http://download.ceph.com/keys/autobuild.asc', 'ansible_loop_var': 'item', '_ansible_item_label': 'http://download.ceph.com/keys/autobuild.asc'}, {'invocation': {'module_args': {'file': 'None', 'keyserver': 'None', 'url': 'http://download.ceph.com/keys/release.asc', 'validate_certs': True, 'keyring': 'None', 'state': 'present', 'key': 'None', 'data': 'None', 'id': 'None'}}, 'changed': False, '_ansible_no_log': False, 'failed': False, 'item': 'http://download.ceph.com/keys/release.asc', 'ansible_loop_var': 'item', '_ansible_item_label': 'http://download.ceph.com/keys/release.asc'}], 'msg': 'All items completed', 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'http://download.ceph.com/keys/autobuild.asc')"}

pass 5285942 2020-08-04 05:00:51 2020-08-04 07:01:24 2020-08-04 08:27:26 1:26:02 0:20:06 1:05:56 smithi master ubuntu 18.04 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/rbd_workunit_suites_iozone} 3
pass 5285937 2020-08-04 05:00:46 2020-08-04 06:55:26 2020-08-04 07:57:27 1:02:01 0:33:57 0:28:04 smithi master rhel 8.1 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} 3
pass 5285698 2020-08-04 02:31:28 2020-08-04 03:12:44 2020-08-04 07:10:50 3:58:06 3:24:44 0:33:22 smithi master ubuntu 16.04 upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-msgr2 4-nautilus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check rgw_swift} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04} 4
pass 5285461 2020-08-04 01:19:56 2020-08-04 10:13:48 2020-08-04 11:07:48 0:54:00 0:25:46 0:28:14 smithi master rhel 7.8 kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_iozone} 3
fail 5285448 2020-08-04 01:19:42 2020-08-04 09:56:22 2020-08-04 15:32:30 5:36:08 0:28:07 5:08:01 smithi master ubuntu 16.04 kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_16.04}} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/strays} 6
Failure Reason:

"2020-08-04 15:18:41.476755 mds.a (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi168:0 (6780), after waiting 49.3747 seconds during MDS startup" in cluster log

pass 5285409 2020-08-04 01:17:56 2020-08-04 09:19:55 2020-08-04 10:33:56 1:14:01 1:04:50 0:09:11 smithi master rhel 7.8 multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} 3
pass 5285318 2020-08-04 01:14:19 2020-08-04 01:52:42 2020-08-04 03:36:43 1:44:01 1:32:26 0:11:35 smithi master centos 7.8 fs/snaps/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_latest} tasks/snaptests} 2
pass 5285255 2020-08-04 01:13:02 2020-08-04 01:17:38 2020-08-04 01:37:36 0:19:58 0:13:21 0:06:37 smithi master rhel 7.8 fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_latest} tasks/alternate-pool} 2
pass 5285061 2020-08-03 21:38:11 2020-08-04 00:54:44 2020-08-04 01:54:44 1:00:00 0:11:47 0:48:13 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5285058 2020-08-03 21:38:08 2020-08-04 00:54:04 2020-08-04 01:20:03 0:25:59 0:19:11 0:06:48 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} 1
pass 5284902 2020-08-03 21:35:52 2020-08-03 23:40:21 2020-08-04 00:54:22 1:14:01 1:07:42 0:06:19 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
fail 5284503 2020-08-03 19:42:15 2020-08-03 19:51:46 2020-08-03 22:37:49 2:46:03 2:19:03 0:27:00 smithi master ubuntu 18.04 upgrade/nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} 4
Failure Reason:

"2020-08-03T20:41:52.407600+0000 mds.a (mds.0) 5 : cluster [WRN] evicting unresponsive client smithi078:3 (55310), after 304.909 seconds" in cluster log