Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 6035532 2021-04-11 09:04:40 2021-04-11 09:06:24 2021-04-11 09:33:07 0:26:43 0:20:26 0:06:17 smithi master rhel 8.2 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-radosbench} 2
Failure Reason:

"2021-04-11T09:26:00.235518+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035533 2021-04-11 09:04:41 2021-04-11 09:06:24 2021-04-11 09:29:10 0:22:46 0:13:23 0:09:23 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

Command failed on smithi104 with status 22: "sudo cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4ed621d0-9aa7-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi104:[v1:172.21.15.104:6790]=c'"

fail 6035534 2021-04-11 09:04:43 2021-04-11 09:06:24 2021-04-11 09:31:33 0:25:09 0:19:10 0:05:59 smithi master rhel 8.2 rados/singleton/{all/deduptool msgr-failures/none msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest}} 1
Failure Reason:

"2021-04-11T09:25:14.584170+0000 mon.a (mon.0) 72 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035535 2021-04-11 09:04:44 2021-04-11 09:06:24 2021-04-11 09:27:55 0:21:31 0:11:45 0:09:46 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:21:01.958292+0000 mon.a (mon.0) 52 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035536 2021-04-11 09:04:45 2021-04-11 09:06:25 2021-04-11 09:33:35 0:27:10 0:20:22 0:06:48 smithi master rhel 8.2 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/redirect} 2
Failure Reason:

"2021-04-11T09:26:24.413202+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035537 2021-04-11 09:04:46 2021-04-11 09:06:25 2021-04-11 09:23:42 0:17:17 0:06:10 0:11:07 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} 2
Failure Reason:

Command failed on smithi006 with status 5: 'sudo systemctl stop ceph-4a210326-9aa7-11eb-81a3-001a4aab830c@mon.a'

fail 6035538 2021-04-11 09:04:47 2021-04-11 09:06:25 2021-04-11 09:25:17 0:18:52 0:11:15 0:07:37 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} 1
Failure Reason:

"2021-04-11T09:19:22.106260+0000 mon.a (mon.0) 64 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035539 2021-04-11 09:04:49 2021-04-11 09:06:26 2021-04-11 09:24:56 0:18:30 0:06:44 0:11:46 smithi master ubuntu 18.04 rados/multimon/{clusters/3 msgr-failures/few msgr/async no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} 2
Failure Reason:

"2021-04-11T09:23:28.858205+0000 mon.a (mon.0) 32 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035540 2021-04-11 09:04:50 2021-04-11 09:06:56 2021-04-11 09:35:02 0:28:06 0:20:14 0:07:52 smithi master rhel 8.2 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_latest} tasks/rados_stress_watch} 2
Failure Reason:

"2021-04-11T09:27:00.574746+0000 mgr.y (mgr.4112) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035541 2021-04-11 09:04:52 2021-04-11 09:06:57 2021-04-11 09:23:31 0:16:34 0:05:25 0:11:09 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} 2
Failure Reason:

Command failed on smithi063 with status 5: 'sudo systemctl stop ceph-73fd6806-9aa7-11eb-81a3-001a4aab830c@mon.smithi063'

fail 6035542 2021-04-11 09:04:53 2021-04-11 09:07:57 2021-04-11 09:28:57 0:21:00 0:12:26 0:08:34 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/version-number-sanity rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:21:09.128964+0000 mon.a (mon.0) 49 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035543 2021-04-11 09:04:54 2021-04-11 09:07:58 2021-04-11 09:29:52 0:21:54 0:12:35 0:09:19 smithi master centos 8.1 rados/singleton/{all/divergent_priors msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:23:47.490262+0000 mon.a (mon.0) 66 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035544 2021-04-11 09:04:56 2021-04-11 09:08:58 2021-04-11 09:33:52 0:24:54 0:15:08 0:09:46 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} 1
Failure Reason:

timeout expired in wait_until_healthy

fail 6035545 2021-04-11 09:04:57 2021-04-11 09:08:58 2021-04-11 09:33:03 0:24:05 0:13:15 0:10:50 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} 2
Failure Reason:

"2021-04-11T09:26:25.031287+0000 mgr.y (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035546 2021-04-11 09:04:59 2021-04-11 09:09:19 2021-04-11 09:37:02 0:27:43 0:14:38 0:13:05 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
Failure Reason:

"2021-04-11T09:29:59.748475+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035547 2021-04-11 09:05:00 2021-04-11 09:10:59 2021-04-11 09:32:57 0:21:58 0:12:34 0:09:24 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} 2
Failure Reason:

"2021-04-11T09:26:43.918303+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035548 2021-04-11 09:05:01 2021-04-11 09:11:00 2021-04-11 09:29:37 0:18:37 0:06:42 0:11:55 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} 2
Failure Reason:

Command failed on smithi197 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f50ed2d6-9aa7-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi197:[v2:172.21.15.197:3301,v1:172.21.15.197:6790]=c'"

fail 6035549 2021-04-11 09:05:03 2021-04-11 09:11:50 2021-04-11 09:37:32 0:25:42 0:14:21 0:11:21 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
Failure Reason:

"2021-04-11T09:30:15.232310+0000 mon.a (mon.0) 126 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035550 2021-04-11 09:05:04 2021-04-11 09:11:51 2021-04-11 09:37:02 0:25:11 0:13:43 0:11:28 smithi master centos 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
Failure Reason:

"2021-04-11T09:29:40.295084+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035551 2021-04-11 09:05:05 2021-04-11 09:13:21 2021-04-11 09:49:03 0:35:42 0:25:52 0:09:50 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

"2021-04-11T09:39:48.817543+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035552 2021-04-11 09:05:07 2021-04-11 09:13:21 2021-04-11 09:37:02 0:23:41 0:13:03 0:10:38 smithi master centos 8.1 rados/singleton/{all/divergent_priors2 msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:29:25.423390+0000 mon.a (mon.0) 71 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035553 2021-04-11 09:05:08 2021-04-11 09:13:22 2021-04-11 09:31:15 0:17:53 0:07:10 0:10:43 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2
Failure Reason:

Command failed on smithi151 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 22aa4c7a-9aa8-11eb-81a3-001a4aab830c -- ceph orch daemon add mon smithi151:172.21.15.151=smithi151'

fail 6035554 2021-04-11 09:05:10 2021-04-11 09:13:32 2021-04-11 09:43:23 0:29:51 0:12:19 0:17:32 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/pool-create-delete} 2
Failure Reason:

"2021-04-11T09:37:17.423330+0000 mon.a (mon.0) 77 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035555 2021-04-11 09:05:11 2021-04-11 09:15:03 2021-04-11 09:35:02 0:19:59 0:08:12 0:11:47 smithi master ubuntu 18.04 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/jewel backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} 3
Failure Reason:

Command failed on smithi100 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8abd8a02-9aa8-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi100:[v1:172.21.15.100:6789]=b'"

fail 6035556 2021-04-11 09:05:12 2021-04-11 09:15:03 2021-04-11 09:37:02 0:21:59 0:11:14 0:10:45 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} 1
Failure Reason:

"2021-04-11T09:29:46.165058+0000 mon.a (mon.0) 68 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035557 2021-04-11 09:05:13 2021-04-11 09:15:03 2021-04-11 09:43:02 0:27:59 0:20:28 0:07:31 smithi master rhel 8.2 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{rhel_latest} tasks/module_selftest} 2
Failure Reason:

"2021-04-11T09:35:31.300593+0000 mgr.z (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035558 2021-04-11 09:05:15 2021-04-11 09:15:03 2021-04-11 09:39:02 0:23:59 0:16:29 0:07:30 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

Command failed on smithi098 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2caee784-9aa9-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi098:[v2:172.21.15.98:3301,v1:172.21.15.98:6790]=c'"

fail 6035559 2021-04-11 09:05:16 2021-04-11 09:15:24 2021-04-11 09:41:03 0:25:39 0:13:18 0:12:21 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} 2
Failure Reason:

"2021-04-11T09:34:39.410006+0000 mgr.y (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035560 2021-04-11 09:05:18 2021-04-11 09:17:05 2021-04-11 09:47:04 0:29:59 0:19:36 0:10:23 smithi master rhel 8.2 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{rhel_latest} tasks/dashboard} 2
Failure Reason:

"2021-04-11T09:39:30.568039+0000 mgr.z (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035561 2021-04-11 09:05:19 2021-04-11 09:19:05 2021-04-11 09:45:51 0:26:46 0:20:15 0:06:31 smithi master rhel 8.2 rados/objectstore/{backends/alloc-hint supported-random-distro$/{rhel_latest}} 1
Failure Reason:

"2021-04-11T09:39:33.803720+0000 mon.a (mon.0) 64 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035562 2021-04-11 09:05:20 2021-04-11 09:19:05 2021-04-11 09:43:02 0:23:57 0:13:16 0:10:41 smithi master centos 8.1 rados/rest/{mgr-restful supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:34:50.484504+0000 mon.a (mon.0) 64 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035563 2021-04-11 09:05:21 2021-04-11 09:19:06 2021-04-11 09:39:51 0:20:45 0:13:02 0:07:43 smithi master centos 8.1 rados/singleton-nomsgr/{all/admin_socket_output rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:32:59.864546+0000 mon.a (mon.0) 53 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

pass 6035564 2021-04-11 09:05:22 2021-04-11 09:19:06 2021-04-11 09:37:55 0:18:49 0:11:20 0:07:29 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_latest} workloads/crush} 1
dead 6035565 2021-04-11 09:05:23 2021-04-11 09:19:06 2021-04-11 10:04:52 0:45:46 smithi master ubuntu 18.04 rados/upgrade/mimic-x-singleton/{0-cluster/{openstack start} 1-install/mimic 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} 4
fail 6035566 2021-04-11 09:05:25 2021-04-11 09:21:07 2021-04-11 09:52:14 0:31:07 0:22:29 0:08:38 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} 1
Failure Reason:

"2021-04-11T09:45:03.822575+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035567 2021-04-11 09:05:26 2021-04-11 09:21:07 2021-04-11 09:47:01 0:25:54 0:12:53 0:13:01 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} 2
Failure Reason:

"2021-04-11T09:39:24.293445+0000 mgr.y (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035568 2021-04-11 09:05:28 2021-04-11 09:23:18 2021-04-11 09:45:01 0:21:43 0:11:57 0:09:46 smithi master ubuntu 18.04 rados/singleton/{all/dump-stuck msgr-failures/none msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:37:48.127059+0000 mon.a (mon.0) 53 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035569 2021-04-11 09:05:30 2021-04-11 09:23:18 2021-04-11 09:45:57 0:22:39 0:13:28 0:09:11 smithi master centos 8.1 rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_latest}} 2
Failure Reason:

"2021-04-11T09:39:21.618902+0000 mgr.y (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035570 2021-04-11 09:05:31 2021-04-11 09:23:19 2021-04-11 09:47:02 0:23:43 0:14:08 0:09:35 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/rados_striper} 2
Failure Reason:

"2021-04-11T09:40:22.641627+0000 mon.a (mon.0) 116 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035571 2021-04-11 09:05:33 2021-04-11 09:23:39 2021-04-11 09:44:51 0:21:12 0:10:08 0:11:04 smithi master centos 7.6 rados/cephadm/smoke/{distro/centos_7 fixed-2 start} 2
Failure Reason:

Command failed on smithi063 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f1fce162-9aa9-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi063:[v2:172.21.15.63:3301,v1:172.21.15.63:6790]=c'"

fail 6035572 2021-04-11 09:05:34 2021-04-11 09:23:40 2021-04-11 09:45:57 0:22:17 0:10:32 0:11:45 smithi master ubuntu 18.04 rados/multimon/{clusters/6 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 2
Failure Reason:

"2021-04-11T09:40:14.292270+0000 mgr.y (mgr.4119) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035573 2021-04-11 09:05:35 2021-04-11 09:23:50 2021-04-11 09:48:36 0:24:46 0:10:27 0:14:19 smithi master centos 7.6 rados/cephadm/smoke-roleless/{distro/centos_7 start} 2
Failure Reason:

Command failed on smithi186 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7f3a1798-9aaa-11eb-81a3-001a4aab830c -- ceph orch daemon add mon smithi186:172.21.15.186=smithi186'

fail 6035574 2021-04-11 09:05:36 2021-04-11 09:25:06 2021-04-11 09:48:16 0:23:10 0:13:00 0:10:10 smithi master centos 8.1 rados/singleton/{all/ec-lost-unfound msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:41:05.642294+0000 mon.a (mon.0) 99 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035575 2021-04-11 09:05:37 2021-04-11 09:25:06 2021-04-11 09:45:45 0:20:39 0:11:37 0:09:02 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} 1
Failure Reason:

"2021-04-11T09:38:33.586896+0000 mon.a (mon.0) 64 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035576 2021-04-11 09:05:39 2021-04-11 09:25:07 2021-04-11 09:49:01 0:23:54 0:12:33 0:11:21 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunk-promote-flush} 2
Failure Reason:

"2021-04-11T09:41:09.220881+0000 mgr.y (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035577 2021-04-11 09:05:40 2021-04-11 09:25:17 2021-04-11 09:52:13 0:26:56 0:17:48 0:09:08 smithi master centos 8.2 rados/cephadm/upgrade/{1-start-distro/1-start-centos_8 2-repo_digest/defaut 3-start-upgrade 4-wait fixed-2} 2
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

fail 6035578 2021-04-11 09:05:42 2021-04-11 09:25:17 2021-04-11 09:56:03 0:30:46 0:22:03 0:08:43 smithi master rhel 8.2 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
Failure Reason:

"2021-04-11T09:48:42.778617+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

pass 6035579 2021-04-11 09:05:44 2021-04-11 09:26:58 2021-04-11 09:46:16 0:19:18 0:09:41 0:09:37 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_adoption} 1
fail 6035580 2021-04-11 09:05:46 2021-04-11 09:26:58 2021-04-11 09:48:08 0:21:10 0:11:53 0:09:17 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/balancer rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:40:55.790683+0000 mon.a (mon.0) 65 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

pass 6035581 2021-04-11 09:05:47 2021-04-11 09:26:59 2021-04-11 09:43:36 0:16:37 0:07:38 0:08:59 smithi master ubuntu 18.04 rados/singleton/{all/erasure-code-nonregression msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} 1
fail 6035582 2021-04-11 09:05:49 2021-04-11 09:26:59 2021-04-11 09:51:02 0:24:03 0:16:42 0:07:21 smithi master rhel 8.2 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
Failure Reason:

Command failed on smithi084 with status 22: "sudo cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e6020bc0-9aaa-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi084:[v2:172.21.15.84:3301,v1:172.21.15.84:6790]=c'"

fail 6035583 2021-04-11 09:05:51 2021-04-11 09:27:40 2021-04-11 09:53:14 0:25:34 0:13:31 0:12:03 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
Failure Reason:

"2021-04-11T09:45:13.209705+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035584 2021-04-11 09:05:52 2021-04-11 09:28:00 2021-04-11 09:55:41 0:27:41 0:21:21 0:06:20 smithi master rhel 8.2 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
Failure Reason:

"2021-04-11T09:49:32.705875+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035585 2021-04-11 09:05:54 2021-04-11 09:29:00 2021-04-11 09:51:32 0:22:32 0:13:56 0:08:36 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/lockdep} 2
Failure Reason:

"2021-04-11T09:44:58.130337+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035586 2021-04-11 09:05:55 2021-04-11 09:29:01 2021-04-11 09:52:37 0:23:36 0:12:46 0:10:50 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/set-chunks-read} 2
Failure Reason:

"2021-04-11T09:45:16.356034+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035587 2021-04-11 09:05:56 2021-04-11 09:29:11 2021-04-11 09:57:18 0:28:07 0:20:38 0:07:29 smithi master rhel 8.2 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest} thrashers/sync-many workloads/rados_5925} 2
Failure Reason:

"2021-04-11T09:50:20.594907+0000 mon.a (mon.0) 77 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035588 2021-04-11 09:05:58 2021-04-11 09:29:42 2021-04-11 09:49:27 0:19:45 0:08:18 0:11:27 smithi master ubuntu 18.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} 3
Failure Reason:

Command failed on smithi173 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c9cd69a-9aaa-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi173:[v1:172.21.15.173:6789]=b'"

fail 6035589 2021-04-11 09:05:59 2021-04-11 09:30:02 2021-04-11 09:58:11 0:28:09 0:20:14 0:07:55 smithi master rhel 8.2 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_latest} tasks/progress} 2
Failure Reason:

"2021-04-11T09:51:21.888016+0000 mgr.z (mgr.4115) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035590 2021-04-11 09:06:01 2021-04-11 09:31:12 2021-04-11 09:50:37 0:19:25 0:08:42 0:10:43 smithi master centos 8.0 rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} 2
Failure Reason:

Command failed on smithi144 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eabc02ce-9aaa-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi144:[v2:172.21.15.144:3301,v1:172.21.15.144:6790]=c'"

fail 6035591 2021-04-11 09:06:02 2021-04-11 09:31:23 2021-04-11 09:58:37 0:27:14 0:20:02 0:07:12 smithi master rhel 8.2 rados/singleton/{all/lost-unfound-delete msgr-failures/none msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest}} 1
Failure Reason:

"2021-04-11T09:51:11.062072+0000 mon.a (mon.0) 50 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035592 2021-04-11 09:06:04 2021-04-11 09:31:23 2021-04-11 09:52:40 0:21:17 0:11:14 0:10:03 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} 1
Failure Reason:

"2021-04-11T09:45:43.258796+0000 mon.a (mon.0) 68 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035593 2021-04-11 09:06:07 2021-04-11 09:31:23 2021-04-11 09:54:52 0:23:29 0:13:40 0:09:49 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
Failure Reason:

"2021-04-11T09:48:22.064926+0000 mgr.y (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035594 2021-04-11 09:06:08 2021-04-11 09:31:44 2021-04-11 09:57:18 0:25:34 0:12:57 0:12:37 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} 2
Failure Reason:

"2021-04-11T09:50:10.004013+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035595 2021-04-11 09:06:11 2021-04-11 09:33:04 2021-04-11 09:54:03 0:20:59 0:09:37 0:11:22 smithi master centos 8.0 rados/cephadm/smoke-roleless/{distro/centos_8.0 start} 2
Failure Reason:

Command failed on smithi067 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 303be54e-9aab-11eb-81a3-001a4aab830c -- ceph orch daemon add mon smithi067:172.21.15.67=smithi067'

fail 6035596 2021-04-11 09:06:12 2021-04-11 09:33:05 2021-04-11 09:56:37 0:23:32 0:12:33 0:10:59 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} 2
Failure Reason:

"2021-04-11T09:48:35.043615+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035597 2021-04-11 09:06:14 2021-04-11 09:33:15 2021-04-11 09:50:36 0:17:21 0:08:07 0:09:14 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5ed7075aceeb911c5eb13628a53ecca211402124 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 6035598 2021-04-11 09:06:15 2021-04-11 09:33:46 2021-04-11 10:00:48 0:27:02 0:20:20 0:06:42 smithi master rhel 8.2 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_latest} tasks/dashboard} 2
Failure Reason:

"2021-04-11T09:53:56.563425+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035599 2021-04-11 09:06:18 2021-04-11 09:33:56 2021-04-11 09:54:48 0:20:52 0:12:03 0:08:49 smithi master ubuntu 18.04 rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:47:36.792050+0000 mon.a (mon.0) 105 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035600 2021-04-11 09:06:20 2021-04-11 09:33:56 2021-04-11 10:01:31 0:27:35 0:20:59 0:06:36 smithi master rhel 8.2 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} 2
Failure Reason:

"2021-04-11T09:54:31.596753+0000 mgr.y (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: devicehealth" in cluster log

fail 6035601 2021-04-11 09:06:21 2021-04-11 09:33:57 2021-04-11 09:55:23 0:21:26 0:12:03 0:09:23 smithi master ubuntu 18.04 rados/singleton/{all/lost-unfound msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:48:46.342155+0000 mon.a (mon.0) 54 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035602 2021-04-11 09:06:22 2021-04-11 09:33:57 2021-04-11 09:56:44 0:22:47 0:13:47 0:09:00 smithi master centos 8.1 rados/singleton-nomsgr/{all/cache-fs-trunc rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:48:42.736090+0000 mon.a (mon.0) 69 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035603 2021-04-11 09:06:23 2021-04-11 09:33:57 2021-04-11 09:53:29 0:19:32 0:08:43 0:10:49 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

Command failed on smithi142 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f27b0ac-9aab-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi142:[v1:172.21.15.142:6790]=c'"

fail 6035604 2021-04-11 09:06:24 2021-04-11 09:34:08 2021-04-11 09:56:37 0:22:29 0:15:30 0:06:59 smithi master rhel 8.2 rados/multimon/{clusters/9 msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} tasks/mon_clock_no_skews} 3
Failure Reason:

"2021-04-11T09:55:57.257374+0000 mon.a (mon.0) 26 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035605 2021-04-11 09:06:27 2021-04-11 09:35:12 2021-04-11 09:52:11 0:16:59 0:07:20 0:09:39 smithi master centos 8.1 rados/cephadm/smoke/{distro/centos_latest fixed-2 start} 2
Failure Reason:

Command failed on smithi042 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0c9b6ef2-9aab-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi042:[v2:172.21.15.42:3301,v1:172.21.15.42:6790]=c'"

fail 6035606 2021-04-11 09:06:28 2021-04-11 09:35:12 2021-04-11 09:56:10 0:20:58 0:11:29 0:09:29 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} 1
Failure Reason:

"2021-04-11T09:49:14.857323+0000 mon.a (mon.0) 68 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

dead 6035607 2021-04-11 09:06:29 2021-04-11 09:35:13 2021-04-11 10:04:14 0:29:01 smithi master rhel 8.2 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
fail 6035608 2021-04-11 09:06:31 2021-04-11 09:37:14 2021-04-11 09:57:56 0:20:42 0:12:08 0:08:34 smithi master centos 8.1 rados/singleton/{all/max-pg-per-osd.from-mon msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:51:24.382016+0000 mon.a (mon.0) 59 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

dead 6035609 2021-04-11 09:06:32 2021-04-11 09:37:14 2021-04-11 10:04:38 0:27:24 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} 1
fail 6035610 2021-04-11 09:06:33 2021-04-11 09:37:15 2021-04-11 10:04:04 0:26:49 0:19:44 0:07:05 smithi master rhel 8.2 rados/singleton-bluestore/{all/cephtool msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest}} 1
Failure Reason:

"2021-04-11T09:56:21.249946+0000 mon.a (mon.0) 82 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

fail 6035611 2021-04-11 09:06:34 2021-04-11 09:37:15 2021-04-11 09:54:37 0:17:22 0:07:44 0:09:38 smithi master centos 8.1 rados/cephadm/smoke-roleless/{distro/centos_latest start} 2
Failure Reason:

Command failed on smithi154 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 764d78a4-9aab-11eb-81a3-001a4aab830c -- ceph orch daemon add mon smithi154:172.21.15.154=smithi154'

fail 6035612 2021-04-11 09:06:36 2021-04-11 09:37:15 2021-04-11 10:00:47 0:23:32 0:12:46 0:10:46 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} 2
Failure Reason:

"2021-04-11T09:53:23.882718+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

dead 6035613 2021-04-11 09:06:37 2021-04-11 09:37:16 2021-04-11 10:04:53 0:27:37 smithi master ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-start-upgrade 4-wait fixed-2} 2
fail 6035614 2021-04-11 09:06:38 2021-04-11 09:37:17 2021-04-11 09:58:48 0:21:31 0:11:28 0:10:03 smithi master ubuntu 18.04 rados/singleton/{all/max-pg-per-osd.from-primary msgr-failures/none msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2021-04-11T09:51:54.463552+0000 mon.a (mon.0) 76 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

dead 6035615 2021-04-11 09:06:39 2021-04-11 09:37:17 2021-04-11 10:04:44 0:27:27 smithi master rhel 8.2 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
dead 6035616 2021-04-11 09:06:41 2021-04-11 09:37:38 2021-04-11 10:04:06 0:26:28 smithi master rhel 8.2 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
dead 6035617 2021-04-11 09:06:42 2021-04-11 09:39:09 2021-04-11 10:04:54 0:25:45 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/mon_recovery validater/valgrind} 2
pass 6035618 2021-04-11 09:06:43 2021-04-11 09:39:59 2021-04-11 09:55:56 0:15:57 0:06:35 0:09:22 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} 1
dead 6035619 2021-04-11 09:06:44 2021-04-11 09:41:10 2021-04-11 10:04:54 0:23:44 smithi master rhel 8.2 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_latest} tasks/prometheus} 2
fail 6035620 2021-04-11 09:06:45 2021-04-11 09:41:10 2021-04-11 10:02:15 0:21:05 0:12:58 0:08:07 smithi master centos 8.1 rados/singleton-nomsgr/{all/ceph-kvstore-tool rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

"2021-04-11T09:56:10.395398+0000 mon.a (mon.0) 69 : cluster [WRN] Health check failed: Module 'devicehealth' has failed dependency: name 'Dict' is not defined (MGR_MODULE_DEPENDENCY)" in cluster log

dead 6035621 2021-04-11 09:06:47 2021-04-11 09:41:11 2021-04-11 10:04:28 0:23:17 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} 2
dead 6035622 2021-04-11 09:06:48 2021-04-11 09:43:06 2021-04-11 10:04:36 0:21:30 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_api_tests} 2
dead 6035623 2021-04-11 09:06:49 2021-04-11 09:43:06 2021-04-11 10:03:43 0:20:37 smithi master ubuntu 18.04 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} 3
fail 6035624 2021-04-11 09:06:50 2021-04-11 09:43:27 2021-04-11 10:00:50 0:17:23 0:07:35 0:09:48 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

Command failed on smithi051 with status 5: 'sudo systemctl stop ceph-aafe04f0-9aac-11eb-81a3-001a4aab830c@mon.a'

dead 6035625 2021-04-11 09:06:52 2021-04-11 09:43:27 2021-04-11 10:03:59 0:20:32 smithi master rhel 8.2 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/small-objects} 2
dead 6035626 2021-04-11 09:06:53 2021-04-11 09:43:38 2021-04-11 10:04:15 0:20:37 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_fio} 1
dead 6035627 2021-04-11 09:06:55 2021-04-11 09:44:59 2021-04-11 10:04:27 0:19:28 smithi master ubuntu 18.04 rados/singleton/{all/max-pg-per-osd.from-replica msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
fail 6035628 2021-04-11 09:06:56 2021-04-11 09:44:59 2021-04-11 10:00:22 0:15:23 0:08:25 0:06:58 smithi master rhel 7.7 rados/cephadm/smoke/{distro/rhel_7 fixed-2 start} 2
Failure Reason:

Command failed on smithi145 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 34b96e06-9aac-11eb-81a3-001a4aab830c -- ceph orch daemon add mon 'smithi145:[v2:172.21.15.145:3301,v1:172.21.15.145:6790]=c'"

dead 6035629 2021-04-11 09:06:57 2021-04-11 09:45:19 2021-04-11 10:04:19 0:19:00 smithi master rhel 8.2 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} 2
fail 6035630 2021-04-11 09:06:58 2021-04-11 09:45:20 2021-04-11 10:01:31 0:16:11 0:08:48 0:07:23 smithi master rhel 7.7 rados/cephadm/smoke-roleless/{distro/rhel_7 start} 2
Failure Reason:

Command failed on smithi092 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5ed7075aceeb911c5eb13628a53ecca211402124 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 491e2f80-9aac-11eb-81a3-001a4aab830c -- ceph orch daemon add mon smithi092:172.21.15.92=smithi092'

dead 6035631 2021-04-11 09:07:00 2021-04-11 09:46:00 2021-04-11 10:04:18 0:18:18 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} tasks/dashboard} 2
dead 6035632 2021-04-11 09:07:01 2021-04-11 09:46:01 2021-04-11 10:04:44 0:18:43 smithi master ubuntu 18.04 rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} 1
dead 6035633 2021-04-11 09:07:02 2021-04-11 09:46:01 2021-04-11 10:04:23 0:18:22 smithi master rhel 8.2 rados/singleton/{all/mon-auth-caps msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest}} 1
dead 6035634 2021-04-11 09:07:03 2021-04-11 09:46:01 2021-04-11 10:04:40 0:18:39 smithi master centos 8.1 rados/multimon/{clusters/21 msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} 3
fail 6035635 2021-04-11 09:07:04 2021-04-11 09:47:12 2021-04-11 10:02:09 0:14:57 0:06:01 0:08:56 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
Failure Reason:

Command failed on smithi132 with status 5: 'sudo systemctl stop ceph-d0b83292-9aac-11eb-81a3-001a4aab830c@mon.a'

dead 6035636 2021-04-11 09:07:05 2021-04-11 09:47:12 2021-04-11 10:04:22 0:17:10 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced} 2
dead 6035637 2021-04-11 09:07:07 2021-04-11 09:47:12 2021-04-11 10:03:40 0:16:28 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
dead 6035638 2021-04-11 09:07:08 2021-04-11 09:48:13 2021-04-11 10:04:52 0:16:39 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
dead 6035639 2021-04-11 09:07:09 2021-04-11 09:49:04 2021-04-11 10:03:51 0:14:47 smithi master centos 8.1 rados/singleton-nomsgr/{all/ceph-post-file rados supported-random-distro$/{centos_latest}} 1
dead 6035640 2021-04-11 09:07:10 2021-04-11 09:49:04 2021-04-11 10:03:58 0:14:54 smithi master rhel 8.0 rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} 2
dead 6035641 2021-04-11 09:07:11 2021-04-11 09:49:05 2021-04-11 10:04:52 0:15:47 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/sample_radosbench} 1
dead 6035642 2021-04-11 09:07:12 2021-04-11 09:49:35 2021-04-11 10:04:23 0:14:48 smithi master centos 8.1 rados/singleton/{all/mon-config-key-caps msgr-failures/none msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} 1
dead 6035643 2021-04-11 09:07:14 2021-04-11 09:49:35 2021-04-11 10:03:54 0:14:19 smithi master rhel 8.0 rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} 2
dead 6035644 2021-04-11 09:07:15 2021-04-11 09:50:46 2021-04-11 10:03:31 0:12:45 smithi master rhel 8.2 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} tasks/rados_workunit_loadgen_mostlyread} 2
dead 6035645 2021-04-11 09:07:16 2021-04-11 09:50:47 2021-04-11 10:04:02 0:13:15 smithi master rhel 8.2 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} 2
dead 6035646 2021-04-11 09:07:17 2021-04-11 09:51:07 2021-04-11 10:04:00 0:12:53 smithi master centos 8.0 rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root msgr/async-v2only start tasks/rados_api_tests} 2
dead 6035647 2021-04-11 09:07:18 2021-04-11 09:51:39 2021-04-11 10:03:40 0:12:01 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
dead 6035648 2021-04-11 09:07:19 2021-04-11 09:52:19 2021-04-11 10:03:37 0:11:18 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
dead 6035649 2021-04-11 09:07:21 2021-04-11 09:52:20 2021-04-11 10:03:40 0:11:20 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/lockdep} 2
dead 6035650 2021-04-11 09:07:22 2021-04-11 09:52:20 2021-04-11 10:03:57 0:11:37 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/workunits} 2
dead 6035651 2021-04-11 09:07:24 2021-04-11 09:52:40 2021-04-11 10:04:47 0:12:07 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} 2
dead 6035652 2021-04-11 09:07:26 2021-04-11 09:53:11 2021-04-11 10:04:40 0:11:29 smithi master ubuntu 18.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{ubuntu_18.04} msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} 3
dead 6035653 2021-04-11 09:07:27 2021-04-11 09:53:22 2021-04-11 10:04:05 0:10:43 smithi master rhel 8.2 rados/singleton/{all/mon-config-keys msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest}} 1
dead 6035654 2021-04-11 09:07:28 2021-04-11 09:53:22 2021-04-11 10:04:43 0:11:21 smithi master ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait fixed-2} 2
dead 6035655 2021-04-11 09:07:29 2021-04-11 09:53:32 2021-04-11 10:04:18 0:10:46 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr} 1
dead 6035656 2021-04-11 09:07:31 2021-04-11 09:53:33 2021-04-11 10:03:40 0:10:07 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} 1
dead 6035657 2021-04-11 09:07:32 2021-04-11 09:54:13 2021-04-11 10:04:51 0:10:38 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/export-after-evict rados supported-random-distro$/{ubuntu_latest}} 1
dead 6035658 2021-04-11 09:07:33 2021-04-11 09:54:13 2021-04-11 10:04:00 0:09:47 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} 1
dead 6035659 2021-04-11 09:07:34 2021-04-11 09:54:44 2021-04-11 10:04:06 0:09:22 smithi master rhel 8.2 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} 2
dead 6035660 2021-04-11 09:07:35 2021-04-11 09:54:54 2021-04-11 10:04:40 0:09:46 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects} 2
dead 6035661 2021-04-11 09:07:36 2021-04-11 09:54:54 2021-04-11 10:04:06 0:09:12 smithi master rhel 8.2 rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} 2
dead 6035662 2021-04-11 09:07:37 2021-04-11 09:55:45 2021-04-11 10:05:10 0:09:25 smithi master ubuntu 18.04 rados/singleton/{all/mon-config msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} 1
dead 6035663 2021-04-11 09:07:38 2021-04-11 09:55:45 2021-04-11 10:04:50 0:09:05 smithi master rhel 8.2 rados/cephadm/smoke-roleless/{distro/rhel_latest start} 2
dead 6035664 2021-04-11 09:07:39 2021-04-11 09:56:05 2021-04-11 10:05:22 0:09:17 smithi master ubuntu 18.04 rados/multimon/{clusters/3 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 2
dead 6035665 2021-04-11 09:07:40 2021-04-11 09:56:06 2021-04-11 10:03:40 0:07:34 smithi master rhel 8.2 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
dead 6035666 2021-04-11 09:07:42 2021-04-11 09:56:46 2021-04-11 10:05:23 0:08:37 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
dead 6035667 2021-04-11 09:07:43 2021-04-11 09:56:47 2021-04-11 10:04:00 0:07:13 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
dead 6035668 2021-04-11 09:07:44 2021-04-11 09:56:47 2021-04-11 10:04:32 0:07:45 smithi master rhel 8.2 rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_latest}} 1
dead 6035669 2021-04-11 09:07:45 2021-04-11 09:57:28 2021-04-11 10:04:08 0:06:40 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
dead 6035670 2021-04-11 09:07:46 2021-04-11 09:57:29 2021-04-11 10:04:49 0:07:20 smithi master ubuntu 18.04 rados/singleton/{all/osd-backfill msgr-failures/none msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
dead 6035671 2021-04-11 09:07:47 2021-04-11 09:57:29 2021-04-11 10:09:19 0:11:50 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed}
Failure Reason:

SSH connection to smithi039 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic'

dead 6035672 2021-04-11 09:07:48 2021-04-11 09:57:29 2021-04-11 10:04:38 0:07:09 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} 2
dead 6035673 2021-04-11 09:07:49 2021-04-11 09:57:30 2021-04-11 10:04:39 0:07:09 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} 1
dead 6035674 2021-04-11 09:07:50 2021-04-11 09:58:00 2021-04-11 10:05:07 0:07:07 smithi master centos 8.1 rados/singleton-nomsgr/{all/full-tiering rados supported-random-distro$/{centos_latest}} 1
fail 6035675 2021-04-11 09:07:51 2021-04-11 09:58:00 2021-04-11 10:05:41 0:07:41 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} 1
dead 6035676 2021-04-11 09:07:53 2021-04-11 09:58:00 2021-04-11 10:05:42 0:07:42 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} 2
Failure Reason:

SSH connection to smithi110 was lost: 'sudo apt-get update'

fail 6035677 2021-04-11 09:07:54 2021-04-11 09:58:21 2021-04-11 10:15:09 0:16:48 smithi master rhel 8.2 rados/singleton/{all/osd-recovery-incomplete msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest}}
Failure Reason:

machine smithi151.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035678 2021-04-11 09:07:55 2021-04-11 09:58:41 2021-04-11 10:13:08 0:14:27 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} tasks/crash}
Failure Reason:

machine smithi170.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035679 2021-04-11 09:07:56 2021-04-11 09:59:41 2021-04-11 10:13:26 0:13:45 smithi master rhel 8.2 rados/singleton-bluestore/{all/cephtool msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest}}
Failure Reason:

machine smithi091.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

dead 6035680 2021-04-11 09:07:57 2021-04-11 09:59:41 2021-04-11 10:12:16 0:12:35 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan smithi115.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6035681 2021-04-11 09:07:59 2021-04-11 10:00:52 2021-04-11 10:14:04 0:13:12 smithi master rhel 8.2 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
Failure Reason:

machine smithi040.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035682 2021-04-11 09:08:00 2021-04-11 10:00:52 2021-04-11 10:13:46 0:12:54 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/valgrind}
Failure Reason:

machine smithi094.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035683 2021-04-11 09:08:01 2021-04-11 10:00:52 2021-04-11 10:13:08 0:12:16 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} 1
Failure Reason:

machine smithi194.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035684 2021-04-11 09:08:02 2021-04-11 10:01:32 2021-04-11 10:15:09 0:13:37 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
Failure Reason:

machine smithi006.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

fail 6035685 2021-04-11 09:08:03 2021-04-11 10:01:33 2021-04-11 10:12:32 0:10:59 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_workunits}
Failure Reason:

machine smithi083.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

dead 6035686 2021-04-11 09:08:04 2021-04-11 10:01:33 2021-04-11 10:14:14 0:12:41 smithi master ubuntu 18.04 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan smithi045.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6035687 2021-04-11 09:08:05 2021-04-11 10:01:33 2021-04-11 10:15:19 0:13:46 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/fastclose msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests}
Failure Reason:

machine smithi066.front.sepia.ceph.com is locked by scheduled_teuthology@teuthology, not scheduled_kchai@teuthology

dead 6035688 2021-04-11 09:08:07 2021-04-11 10:02:24 2021-04-11 10:14:32 0:12:08 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2