Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7611971 2024-03-20 08:55:42 2024-03-20 08:59:41 2024-03-20 09:22:09 0:22:28 0:09:46 0:12:42 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-03-20T09:19:12.090973+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611972 2024-03-20 08:55:43 2024-03-20 08:59:41 2024-03-20 09:23:29 0:23:48 0:10:18 0:13:30 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

Command failed on smithi053 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9dc35a8-e69a-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4'

fail 7611973 2024-03-20 08:55:44 2024-03-20 09:00:12 2024-03-20 09:38:46 0:38:34 0:26:36 0:11:58 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi007 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1c61ce6-e69a-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7611974 2024-03-20 08:55:46 2024-03-20 09:01:42 2024-03-20 09:39:49 0:38:07 0:24:41 0:13:26 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Command failed on smithi039 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ab91b5dc-e69a-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7611975 2024-03-20 08:55:47 2024-03-20 09:01:43 2024-03-20 09:24:31 0:22:48 0:12:33 0:10:15 smithi main ubuntu 22.04 orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-03-20T09:22:50.387781+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611976 2024-03-20 08:55:48 2024-03-20 09:02:53 2024-03-20 09:22:43 0:19:50 0:08:59 0:10:51 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
Failure Reason:

"2024-03-20T09:21:51.745842+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7611977 2024-03-20 08:55:50 2024-03-20 09:02:53 2024-03-20 09:30:55 0:28:02 0:14:03 0:13:59 smithi main ubuntu 22.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
fail 7611978 2024-03-20 08:55:51 2024-03-20 09:07:14 2024-03-20 09:27:09 0:19:55 0:07:39 0:12:16 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-03-20T09:24:08.039800+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611979 2024-03-20 08:55:52 2024-03-20 09:08:15 2024-03-20 09:35:41 0:27:26 0:16:47 0:10:39 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-20T09:23:39.824469+0000 mon.smithi077 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611980 2024-03-20 08:55:54 2024-03-20 09:08:36 2024-03-20 09:32:19 0:23:43 0:12:08 0:11:35 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
Failure Reason:

"2024-03-20T09:30:10.508932+0000 mon.smithi076 (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611981 2024-03-20 08:55:55 2024-03-20 09:10:16 2024-03-20 09:32:46 0:22:30 0:09:16 0:13:14 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-20T09:29:21.968918+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611982 2024-03-20 08:55:56 2024-03-20 09:10:17 2024-03-20 09:33:22 0:23:05 0:09:51 0:13:14 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-20T09:32:01.399817+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611983 2024-03-20 08:55:58 2024-03-20 09:10:17 2024-03-20 10:01:49 0:51:32 0:39:46 0:11:46 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi050 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d5fb3dc-e69b-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.osd | length == 2\'"\'"\'\''

fail 7611984 2024-03-20 08:55:59 2024-03-20 09:10:37 2024-03-20 09:31:19 0:20:42 0:10:55 0:09:47 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-20T09:29:01.452224+0000 mon.a (mon.0) 103 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611985 2024-03-20 08:56:00 2024-03-20 09:10:38 2024-03-20 09:29:18 0:18:40 0:09:05 0:09:35 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-03-20T09:27:43.361385+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611986 2024-03-20 08:56:02 2024-03-20 09:10:38 2024-03-20 09:37:14 0:26:36 0:16:48 0:09:48 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-20T09:27:02.241238+0000 mon.smithi006 (mon.0) 117 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611987 2024-03-20 08:56:03 2024-03-20 09:10:39 2024-03-20 09:52:11 0:41:32 0:32:11 0:09:21 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-20T09:30:07.243107+0000 mon.smithi008 (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611988 2024-03-20 08:56:04 2024-03-20 09:10:39 2024-03-20 09:56:05 0:45:26 0:33:20 0:12:06 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-20T09:31:55.175888+0000 mon.smithi090 (mon.0) 112 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611989 2024-03-20 08:56:05 2024-03-20 09:11:19 2024-03-20 09:32:58 0:21:39 0:11:29 0:10:10 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

Command failed on smithi073 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 025bf4e4-e69c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1'

fail 7611990 2024-03-20 08:56:07 2024-03-20 09:11:20 2024-03-20 09:31:42 0:20:22 0:10:21 0:10:01 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-20T09:30:12.071021+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611991 2024-03-20 08:56:08 2024-03-20 09:12:40 2024-03-20 09:37:17 0:24:37 0:13:58 0:10:39 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-20T09:34:30.233110+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611992 2024-03-20 08:56:09 2024-03-20 09:13:01 2024-03-20 09:37:13 0:24:12 0:14:37 0:09:35 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

Command failed on smithi069 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b79285c6-e69c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4'

fail 7611993 2024-03-20 08:56:11 2024-03-20 09:13:21 2024-03-20 09:56:32 0:43:11 0:33:34 0:09:37 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi112 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 31d22dc4-e69c-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7611994 2024-03-20 08:56:12 2024-03-20 09:13:22 2024-03-20 09:41:00 0:27:38 0:16:50 0:10:48 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-20T09:30:00.358982+0000 mon.smithi133 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611995 2024-03-20 08:56:13 2024-03-20 09:13:42 2024-03-20 09:40:19 0:26:37 0:12:19 0:14:18 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-03-20T09:36:32.656518+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611996 2024-03-20 08:56:15 2024-03-20 09:16:33 2024-03-20 09:44:00 0:27:27 0:16:44 0:10:43 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-20T09:32:37.866653+0000 mon.smithi132 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611997 2024-03-20 08:56:16 2024-03-20 09:16:33 2024-03-20 09:35:50 0:19:17 0:09:39 0:09:38 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-20T09:33:55.911611+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7611998 2024-03-20 08:56:17 2024-03-20 09:16:54 2024-03-20 09:38:53 0:21:59 0:10:50 0:11:09 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-03-20T09:36:27.145777+0000 mon.a (mon.0) 103 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612001 2024-03-20 08:56:19 2024-03-20 09:17:44 2024-03-20 09:45:10 0:27:26 0:16:34 0:10:52 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-20T09:33:53.020568+0000 mon.smithi001 (mon.0) 117 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612004 2024-03-20 08:56:20 2024-03-20 09:18:25 2024-03-20 10:00:05 0:41:40 0:30:56 0:10:44 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-20T09:37:27.912359+0000 mon.smithi047 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612006 2024-03-20 08:56:21 2024-03-20 09:18:25 2024-03-20 09:42:19 0:23:54 0:13:27 0:10:27 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-20T09:38:49.594870+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612007 2024-03-20 08:56:23 2024-03-20 09:18:36 2024-03-20 09:56:02 0:37:26 0:26:33 0:10:53 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi086 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1bfe16b0-e69d-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612008 2024-03-20 08:56:24 2024-03-20 09:19:06 2024-03-20 09:47:34 0:28:28 0:17:06 0:11:22 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-20T09:36:11.385423+0000 mon.smithi072 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612009 2024-03-20 08:56:25 2024-03-20 09:19:57 2024-03-20 09:40:34 0:20:37 0:08:31 0:12:06 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

Command failed on smithi003 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 39ac65e0-e69d-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1'

dead 7612010 2024-03-20 08:56:26 2024-03-20 09:21:37 2024-03-20 21:32:43 12:11:06 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

hit max job timeout

fail 7612011 2024-03-20 08:56:28 2024-03-20 09:21:58 2024-03-20 09:42:47 0:20:49 0:10:51 0:09:58 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-20T09:40:56.121738+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612012 2024-03-20 08:56:29 2024-03-20 09:21:58 2024-03-20 09:44:58 0:23:00 0:10:41 0:12:19 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-03-20T09:43:08.062811+0000 mon.a (mon.0) 241 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7612013 2024-03-20 08:56:30 2024-03-20 09:23:49 2024-03-20 09:51:27 0:27:38 0:16:38 0:11:00 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-20T09:40:33.955303+0000 mon.smithi150 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7612014 2024-03-20 08:56:32 2024-03-20 09:23:49 2024-03-20 09:45:44 0:21:55 0:12:45 0:09:10 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
fail 7612015 2024-03-20 08:56:33 2024-03-20 09:24:30 2024-03-20 09:52:30 0:28:00 0:17:15 0:10:45 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-20T09:41:05.330307+0000 mon.smithi005 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612016 2024-03-20 08:56:34 2024-03-20 09:25:30 2024-03-20 09:42:40 0:17:10 0:07:20 0:09:50 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

Command failed on smithi053 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 90116e08-e69d-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1'

fail 7612017 2024-03-20 08:56:36 2024-03-20 09:25:51 2024-03-20 10:07:17 0:41:26 0:31:24 0:10:02 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-20T09:44:10.230214+0000 mon.smithi070 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612018 2024-03-20 08:56:37 2024-03-20 09:25:51 2024-03-20 09:50:11 0:24:20 0:13:15 0:11:05 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-20T09:47:28.414711+0000 mon.a (mon.0) 100 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612019 2024-03-20 08:56:38 2024-03-20 09:25:51 2024-03-20 09:44:28 0:18:37 0:10:14 0:08:23 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-20T09:43:02.965966+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612020 2024-03-20 08:56:40 2024-03-20 09:25:52 2024-03-20 09:44:37 0:18:45 0:10:04 0:08:41 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

"2024-03-20T09:43:03.800462+0000 mon.a (mon.0) 105 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612021 2024-03-20 08:56:41 2024-03-20 09:25:52 2024-03-20 10:08:59 0:43:07 0:32:39 0:10:28 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi136 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 24c389b4-e69e-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612022 2024-03-20 08:56:42 2024-03-20 09:25:52 2024-03-20 09:52:49 0:26:57 0:16:23 0:10:34 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-20T09:40:58.698489+0000 mon.smithi064 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612023 2024-03-20 08:56:44 2024-03-20 09:25:53 2024-03-20 09:49:23 0:23:30 0:13:55 0:09:35 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

"2024-03-20T09:46:27.795327+0000 mon.a (mon.0) 98 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612024 2024-03-20 08:56:45 2024-03-20 09:25:53 2024-03-20 09:53:25 0:27:32 0:17:33 0:09:59 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi195 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7612025 2024-03-20 08:56:46 2024-03-20 09:25:53 2024-03-20 09:53:31 0:27:38 0:16:52 0:10:46 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-20T09:42:05.537020+0000 mon.smithi038 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612026 2024-03-20 08:56:48 2024-03-20 09:26:04 2024-03-20 09:47:32 0:21:28 0:10:01 0:11:27 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-20T09:44:35.125110+0000 mon.a (mon.0) 104 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612027 2024-03-20 08:56:49 2024-03-20 09:26:04 2024-03-20 10:11:04 0:45:00 0:31:40 0:13:20 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-20T09:47:23.868787+0000 mon.smithi049 (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612028 2024-03-20 08:56:50 2024-03-20 09:26:04 2024-03-20 10:08:19 0:42:15 0:31:09 0:11:06 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-20T09:45:06.069024+0000 mon.smithi119 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612029 2024-03-20 08:56:52 2024-03-20 09:26:35 2024-03-20 09:52:13 0:25:38 0:10:04 0:15:34 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
Failure Reason:

"2024-03-20T09:49:44.627406+0000 mon.a (mon.0) 102 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7612030 2024-03-20 08:56:53 2024-03-20 09:30:56 2024-03-20 09:47:40 0:16:44 0:06:24 0:10:20 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7612031 2024-03-20 08:56:54 2024-03-20 09:30:56 2024-03-20 10:10:16 0:39:20 0:26:00 0:13:20 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi165 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a043bfe-e69f-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612032 2024-03-20 08:56:56 2024-03-20 09:32:57 2024-03-20 09:52:02 0:19:05 0:09:03 0:10:02 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

"2024-03-20T09:50:35.153409+0000 mon.a (mon.0) 108 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7612033 2024-03-20 08:56:57 2024-03-20 09:32:57 2024-03-20 10:06:14 0:33:17 0:14:30 0:18:47 smithi main ubuntu 22.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
fail 7612034 2024-03-20 08:56:58 2024-03-20 09:41:29 2024-03-20 10:14:20 0:32:51 0:19:43 0:13:08 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-20T10:03:00.591325+0000 mon.smithi037 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612035 2024-03-20 08:57:00 2024-03-20 09:41:29 2024-03-20 09:58:10 0:16:41 0:07:32 0:09:09 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
Failure Reason:

"2024-03-20T09:56:53.712457+0000 mon.smithi017 (mon.0) 120 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612036 2024-03-20 08:57:01 2024-03-20 09:41:29 2024-03-20 10:06:36 0:25:07 0:12:50 0:12:17 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

Command failed on smithi069 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 46393498-e6a0-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1'

fail 7612037 2024-03-20 08:57:02 2024-03-20 09:41:30 2024-03-20 10:36:05 0:54:35 0:42:54 0:11:41 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi043 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15fd193e-e6a0-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.osd | length == 2\'"\'"\'\''

fail 7612038 2024-03-20 08:57:04 2024-03-20 09:41:30 2024-03-20 10:16:48 0:35:18 0:21:23 0:13:55 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-20T10:05:07.885134+0000 mon.smithi066 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612039 2024-03-20 08:57:05 2024-03-20 09:41:30 2024-03-20 10:03:35 0:22:05 0:09:31 0:12:34 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-03-20T10:01:16.135422+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612040 2024-03-20 08:57:06 2024-03-20 09:41:31 2024-03-20 10:07:14 0:25:43 0:15:22 0:10:21 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-20T10:05:09.445444+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612041 2024-03-20 08:57:08 2024-03-20 09:41:31 2024-03-20 10:04:17 0:22:46 0:11:46 0:11:00 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-20T10:01:24.247838+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612042 2024-03-20 08:57:09 2024-03-20 09:41:32 2024-03-20 10:05:56 0:24:24 0:12:47 0:11:37 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

Command failed on smithi071 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65122f46-e6a0-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4'

fail 7612043 2024-03-20 08:57:10 2024-03-20 09:41:32 2024-03-20 10:27:50 0:46:18 0:34:57 0:11:21 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-20T10:04:52.816204+0000 mon.smithi042 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612044 2024-03-20 08:57:11 2024-03-20 09:41:32 2024-03-20 10:10:20 0:28:48 0:16:25 0:12:23 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-20T09:58:36.684691+0000 mon.smithi146 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612045 2024-03-20 08:57:13 2024-03-20 09:41:33 2024-03-20 10:30:21 0:48:48 0:38:02 0:10:46 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi003 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60ee69f2-e6a0-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612046 2024-03-20 08:57:14 2024-03-20 09:41:33 2024-03-20 10:10:05 0:28:32 0:16:30 0:12:02 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-20T09:58:55.644481+0000 mon.smithi176 (mon.0) 117 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612047 2024-03-20 08:57:15 2024-03-20 09:41:33 2024-03-20 10:08:17 0:26:44 0:16:07 0:10:37 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-03-20T10:05:20.054023+0000 mon.a (mon.0) 100 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612048 2024-03-20 08:57:17 2024-03-20 09:41:34 2024-03-20 10:09:45 0:28:11 0:16:22 0:11:49 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-03-20T10:05:20.545396+0000 mon.a (mon.0) 98 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612049 2024-03-20 08:57:18 2024-03-20 09:41:34 2024-03-20 10:05:56 0:24:22 0:12:24 0:11:58 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
Failure Reason:

"2024-03-20T10:03:51.417365+0000 mon.a (mon.0) 104 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612050 2024-03-20 08:57:19 2024-03-20 09:41:34 2024-03-20 10:09:52 0:28:18 0:16:57 0:11:21 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-20T09:58:00.341811+0000 mon.smithi032 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612051 2024-03-20 08:57:21 2024-03-20 09:41:35 2024-03-20 10:08:19 0:26:44 0:16:24 0:10:20 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-20T10:05:45.087739+0000 mon.a (mon.0) 100 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612052 2024-03-20 08:57:22 2024-03-20 09:41:35 2024-03-20 10:07:07 0:25:32 0:13:45 0:11:47 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-20T10:04:31.236256+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612053 2024-03-20 08:57:23 2024-03-20 09:41:35 2024-03-20 10:03:17 0:21:42 0:10:38 0:11:04 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

Command failed on smithi073 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3fe8abfa-e6a0-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4'

fail 7612054 2024-03-20 08:57:25 2024-03-20 09:41:36 2024-03-20 10:25:59 0:44:23 0:34:11 0:10:12 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-20T10:03:57.238044+0000 mon.smithi084 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612055 2024-03-20 08:57:26 2024-03-20 09:41:36 2024-03-20 10:09:18 0:27:42 0:16:31 0:11:11 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-20T09:57:37.313009+0000 mon.smithi012 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612056 2024-03-20 08:57:27 2024-03-20 09:41:46 2024-03-20 10:22:00 0:40:14 0:29:11 0:11:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 735aee08-e6a0-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612057 2024-03-20 08:57:29 2024-03-20 09:41:47 2024-03-20 10:16:36 0:34:49 0:21:12 0:13:37 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-20T10:04:04.166181+0000 mon.smithi029 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612058 2024-03-20 08:57:30 2024-03-20 09:41:57 2024-03-20 10:07:41 0:25:44 0:10:06 0:15:38 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-20T10:04:20.996499+0000 mon.a (mon.0) 103 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

dead 7612059 2024-03-20 08:57:31 2024-03-20 09:46:38 2024-03-20 21:59:27 12:12:49 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

hit max job timeout

fail 7612060 2024-03-20 08:57:33 2024-03-20 09:48:19 2024-03-20 10:10:33 0:22:14 0:11:32 0:10:42 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

Command failed on smithi033 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5467f71a-e6a1-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1'

fail 7612061 2024-03-20 08:57:34 2024-03-20 09:48:39 2024-03-20 10:10:42 0:22:03 0:10:14 0:11:49 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-20T10:09:25.241405+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612062 2024-03-20 08:57:35 2024-03-20 09:51:40 2024-03-20 10:17:52 0:26:12 0:10:40 0:15:32 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-20T10:15:40.501313+0000 mon.a (mon.0) 105 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612063 2024-03-20 08:57:37 2024-03-20 09:57:11 2024-03-20 10:20:54 0:23:43 0:11:30 0:12:13 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-03-20T10:17:46.978441+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

dead 7612064 2024-03-20 08:57:38 2024-03-20 09:57:11 2024-03-20 22:05:53 12:08:42 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

hit max job timeout

fail 7612065 2024-03-20 08:57:40 2024-03-20 09:57:12 2024-03-20 10:26:03 0:28:51 0:16:58 0:11:53 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-03-20T10:21:30.952387+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612066 2024-03-20 08:57:41 2024-03-20 09:57:12 2024-03-20 10:39:33 0:42:21 0:31:21 0:11:00 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-20T10:18:03.634251+0000 mon.smithi064 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612067 2024-03-20 08:57:42 2024-03-20 09:57:13 2024-03-20 10:25:16 0:28:03 0:14:39 0:13:24 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-20T10:21:33.315415+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612068 2024-03-20 08:57:44 2024-03-20 09:57:13 2024-03-20 10:24:57 0:27:44 0:15:15 0:12:29 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

"2024-03-20T10:22:58.170670+0000 mon.a (mon.0) 251 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7612069 2024-03-20 08:57:45 2024-03-20 09:57:13 2024-03-20 10:40:10 0:42:57 0:30:58 0:11:59 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-20T10:17:42.129618+0000 mon.smithi082 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612070 2024-03-20 08:57:47 2024-03-20 09:57:14 2024-03-20 10:40:12 0:42:58 0:33:30 0:09:28 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi045 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 67bd783e-e6a2-11ee-95c9-87774f69a715 -e sha1=98d972ce3416f5a9a175c9ef4cfd79e8a3c3acc2 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7612071 2024-03-20 08:57:48 2024-03-20 09:57:14 2024-03-20 10:24:01 0:26:47 0:17:02 0:09:45 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-20T10:12:31.737730+0000 mon.smithi097 (mon.0) 118 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612072 2024-03-20 08:57:49 2024-03-20 09:57:14 2024-03-20 10:20:01 0:22:47 0:10:50 0:11:57 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-20T10:17:25.123714+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612073 2024-03-20 08:57:51 2024-03-20 09:57:15 2024-03-20 10:27:11 0:29:56 0:16:44 0:13:12 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-20T10:14:59.579670+0000 mon.smithi008 (mon.0) 115 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612074 2024-03-20 08:57:52 2024-03-20 09:57:15 2024-03-20 10:18:01 0:20:46 0:10:01 0:10:45 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-20T10:15:23.913759+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612075 2024-03-20 08:57:54 2024-03-20 09:57:16 2024-03-20 10:19:38 0:22:22 0:10:43 0:11:39 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-03-20T10:17:34.101276+0000 mon.a (mon.0) 107 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7612076 2024-03-20 08:57:55 2024-03-20 09:57:16 2024-03-20 10:40:16 0:43:00 0:31:01 0:11:59 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-20T10:17:13.898890+0000 mon.smithi112 (mon.0) 113 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log