Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7555074 2024-02-09 19:46:34 2024-02-10 05:19:08 2024-02-10 05:37:31 0:18:23 0:07:29 0:10:54 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} 3
Failure Reason:

Command failed on smithi052 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7555075 2024-02-09 19:46:35 2024-02-10 05:19:38 2024-02-10 05:51:38 0:32:00 0:21:42 0:10:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-02-10T05:40:14.564606+0000 mon.smithi062 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555076 2024-02-09 19:46:36 2024-02-10 05:20:09 2024-02-10 05:44:27 0:24:18 0:12:45 0:11:33 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi080 with status 5: 'sudo systemctl stop ceph-a57d1fac-c7d6-11ee-95b8-87774f69a715@mon.a'

pass 7555077 2024-02-09 19:46:37 2024-02-10 05:20:09 2024-02-10 05:54:05 0:33:56 0:23:15 0:10:41 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
fail 7555078 2024-02-09 19:46:38 2024-02-10 05:20:40 2024-02-10 06:03:20 0:42:40 0:34:21 0:08:19 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-10T05:38:25.393513+0000 mon.smithi028 (mon.0) 503 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7555079 2024-02-09 19:46:39 2024-02-10 05:20:40 2024-02-10 06:16:17 0:55:37 0:45:44 0:09:53 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7555080 2024-02-09 19:46:40 2024-02-10 05:20:41 2024-02-10 05:44:45 0:24:04 0:13:57 0:10:07 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7555081 2024-02-09 19:46:40 2024-02-10 05:20:41 2024-02-10 05:51:32 0:30:51 0:22:34 0:08:17 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7555082 2024-02-09 19:46:41 2024-02-10 05:20:41 2024-02-10 05:56:33 0:35:52 0:26:34 0:09:18 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7555083 2024-02-09 19:46:42 2024-02-10 05:20:42 2024-02-10 05:48:25 0:27:43 0:16:50 0:10:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-02-10T05:37:49.894615+0000 mon.smithi040 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555084 2024-02-09 19:46:43 2024-02-10 05:21:22 2024-02-10 05:42:16 0:20:54 0:11:19 0:09:35 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
fail 7555085 2024-02-09 19:46:44 2024-02-10 05:21:23 2024-02-10 05:46:44 0:25:21 0:13:13 0:12:08 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-02-10T05:41:35.387173+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7555086 2024-02-09 19:46:45 2024-02-10 05:21:33 2024-02-10 05:47:16 0:25:43 0:15:26 0:10:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_cephadm} 1
fail 7555087 2024-02-09 19:46:46 2024-02-10 05:22:13 2024-02-10 05:50:08 0:27:55 0:16:09 0:11:46 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-02-10T05:39:52.588794+0000 mon.smithi039 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555088 2024-02-09 19:46:46 2024-02-10 05:22:24 2024-02-10 06:28:29 1:06:05 0:51:25 0:14:40 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi161 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c123fdac-c7d6-11ee-95b8-87774f69a715 -e sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -- bash -c 'ceph orch ps'"

fail 7555089 2024-02-09 19:46:47 2024-02-10 05:22:54 2024-02-10 05:52:39 0:29:45 0:20:16 0:09:29 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-02-10T05:44:00.411987+0000 mon.smithi107 (mon.0) 256 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555090 2024-02-09 19:46:48 2024-02-10 05:22:55 2024-02-10 06:39:08 1:16:13 1:06:48 0:09:25 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-10T06:00:00.000155+0000 mon.a (mon.0) 966 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7555091 2024-02-09 19:46:49 2024-02-10 05:23:25 2024-02-10 05:55:15 0:31:50 0:22:05 0:09:45 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-02-10T05:44:16.274007+0000 mon.smithi132 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555092 2024-02-09 19:46:50 2024-02-10 05:23:25 2024-02-10 05:39:49 0:16:24 0:06:00 0:10:24 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7555093 2024-02-09 19:46:51 2024-02-10 05:24:46 2024-02-10 05:49:59 0:25:13 0:15:10 0:10:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-02-10T05:42:24.468128+0000 mon.smithi089 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555094 2024-02-09 19:46:51 2024-02-10 05:24:46 2024-02-10 05:52:23 0:27:37 0:13:39 0:13:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-02-10T05:46:10.958976+0000 mon.smithi091 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555095 2024-02-09 19:46:52 2024-02-10 05:28:37 2024-02-10 06:00:57 0:32:20 0:18:55 0:13:25 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm_timeout.py'

fail 7555096 2024-02-09 19:46:53 2024-02-10 05:33:28 2024-02-10 06:00:45 0:27:17 0:14:42 0:12:35 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-02-10T05:53:09.142941+0000 mon.smithi023 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555097 2024-02-09 19:46:54 2024-02-10 05:34:49 2024-02-10 06:04:26 0:29:37 0:20:14 0:09:23 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-02-10T05:55:05.191164+0000 mon.smithi047 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555098 2024-02-09 19:46:55 2024-02-10 05:34:49 2024-02-10 06:36:50 1:02:01 0:52:03 0:09:58 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-10T06:10:00.194224+0000 mon.a (mon.0) 2885 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled" in cluster log

fail 7555099 2024-02-09 19:46:56 2024-02-10 05:35:30 2024-02-10 06:16:39 0:41:09 0:31:47 0:09:22 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-02-10T06:10:00.000147+0000 mon.smithi057 (mon.0) 515 : cluster 4 [ERR] MDS_ALL_DOWN: 1 filesystem is offline" in cluster log

fail 7555100 2024-02-09 19:46:57 2024-02-10 05:35:30 2024-02-10 05:59:00 0:23:30 0:14:31 0:08:59 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-02-10T05:51:11.758153+0000 mon.a (mon.0) 176 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555101 2024-02-09 19:46:58 2024-02-10 05:36:10 2024-02-10 06:06:49 0:30:39 0:18:06 0:12:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-02-10T05:56:18.132163+0000 mon.smithi164 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555102 2024-02-09 19:46:58 2024-02-10 05:39:31 2024-02-10 06:00:54 0:21:23 0:12:02 0:09:21 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
fail 7555103 2024-02-09 19:46:59 2024-02-10 05:39:52 2024-02-10 06:39:42 0:59:50 0:48:06 0:11:44 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-02-10T05:56:17.726978+0000 mon.a (mon.0) 168 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7555104 2024-02-09 19:47:00 2024-02-10 05:40:12 2024-02-10 06:04:31 0:24:19 0:13:07 0:11:12 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_extra_daemon_features} 2
fail 7555105 2024-02-09 19:47:01 2024-02-10 05:40:12 2024-02-10 06:26:27 0:46:15 0:32:00 0:14:15 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-10T06:10:00.000140+0000 mon.a (mon.0) 843 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7555106 2024-02-09 19:47:02 2024-02-10 05:44:53 2024-02-10 06:11:49 0:26:56 0:15:53 0:11:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-02-10T06:03:46.971503+0000 mon.smithi007 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555107 2024-02-09 19:47:03 2024-02-10 05:46:54 2024-02-10 06:12:37 0:25:43 0:15:44 0:09:59 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_host_drain} 3
fail 7555108 2024-02-09 19:47:04 2024-02-10 05:46:54 2024-02-10 06:14:30 0:27:36 0:13:16 0:14:20 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-02-10T06:07:57.409746+0000 mon.smithi081 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555109 2024-02-09 19:47:05 2024-02-10 05:51:35 2024-02-10 06:21:01 0:29:26 0:19:24 0:10:02 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-02-10T06:11:31.620105+0000 mon.smithi040 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555110 2024-02-09 19:47:06 2024-02-10 05:51:36 2024-02-10 06:14:40 0:23:04 0:13:09 0:09:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-02-10T06:08:10.769827+0000 mon.smithi062 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555111 2024-02-09 19:47:07 2024-02-10 05:51:36 2024-02-10 06:14:44 0:23:08 0:12:59 0:10:09 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7555112 2024-02-09 19:47:07 2024-02-10 05:51:36 2024-02-10 06:16:43 0:25:07 0:13:55 0:11:12 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-02-10T06:09:00.928679+0000 mon.smithi149 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555113 2024-02-09 19:47:08 2024-02-10 05:51:37 2024-02-10 06:54:35 1:02:58 0:51:30 0:11:28 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi089 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 84e5b1ba-c7da-11ee-95b8-87774f69a715 -e sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -- bash -c 'ceph orch ps'"

fail 7555114 2024-02-09 19:47:09 2024-02-10 05:51:37 2024-02-10 06:09:46 0:18:09 0:07:15 0:10:54 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi118 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7555115 2024-02-09 19:47:10 2024-02-10 05:51:37 2024-02-10 06:24:22 0:32:45 0:21:30 0:11:15 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-02-10T06:13:47.306682+0000 mon.smithi039 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555116 2024-02-09 19:47:11 2024-02-10 05:51:38 2024-02-10 06:20:55 0:29:17 0:20:27 0:08:50 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-02-10T06:11:35.725296+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555117 2024-02-09 19:47:12 2024-02-10 05:51:38 2024-02-10 06:23:25 0:31:47 0:20:49 0:10:58 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-02-10T06:12:59.347822+0000 mon.smithi052 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555118 2024-02-09 19:47:13 2024-02-10 05:51:38 2024-02-10 06:38:57 0:47:19 0:37:01 0:10:18 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-10T06:20:00.000238+0000 mon.a (mon.0) 1116 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7555119 2024-02-09 19:47:13 2024-02-10 05:51:39 2024-02-10 06:36:19 0:44:40 0:31:50 0:12:50 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-02-10T06:15:58.124955+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7555120 2024-02-09 19:47:14 2024-02-10 05:54:09 2024-02-10 06:16:18 0:22:09 0:11:22 0:10:47 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
pass 7555121 2024-02-09 19:47:15 2024-02-10 05:55:10 2024-02-10 06:20:12 0:25:02 0:15:48 0:09:14 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7555122 2024-02-09 19:47:16 2024-02-10 05:55:10 2024-02-10 06:22:31 0:27:21 0:14:28 0:12:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-02-10T06:14:34.398696+0000 mon.smithi152 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555123 2024-02-09 19:47:17 2024-02-10 05:56:41 2024-02-10 06:19:43 0:23:02 0:12:46 0:10:16 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
pass 7555124 2024-02-09 19:47:18 2024-02-10 05:56:41 2024-02-10 06:22:09 0:25:28 0:11:33 0:13:55 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
pass 7555125 2024-02-09 19:47:18 2024-02-10 06:01:02 2024-02-10 06:25:56 0:24:54 0:13:25 0:11:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_orch_cli} 1
fail 7555126 2024-02-09 19:47:19 2024-02-10 06:03:13 2024-02-10 06:29:16 0:26:03 0:15:18 0:10:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-02-10T06:20:51.900706+0000 mon.smithi122 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555127 2024-02-09 19:47:20 2024-02-10 06:04:13 2024-02-10 06:56:25 0:52:12 0:40:31 0:11:41 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-02-10T06:24:35.343064+0000 mon.a (mon.0) 430 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7555128 2024-02-09 19:47:21 2024-02-10 06:04:14 2024-02-10 06:36:18 0:32:04 0:22:10 0:09:54 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-02-10T06:25:23.883829+0000 mon.smithi067 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555129 2024-02-09 19:47:22 2024-02-10 06:04:34 2024-02-10 06:30:38 0:26:04 0:13:24 0:12:40 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-02-10T06:24:08.301638+0000 mon.smithi091 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555130 2024-02-09 19:47:22 2024-02-10 06:07:05 2024-02-10 06:55:51 0:48:46 0:37:31 0:11:15 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-02-10T06:31:57.930308+0000 mon.a (mon.0) 345 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,e" in cluster log

fail 7555131 2024-02-09 19:47:23 2024-02-10 06:07:05 2024-02-10 06:47:48 0:40:43 0:31:09 0:09:34 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1707546575.658418 mon.smithi019 (mon.0) 679 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7555132 2024-02-09 19:47:24 2024-02-10 06:07:06 2024-02-10 06:32:11 0:25:05 0:15:26 0:09:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-02-10T06:24:04.743508+0000 mon.smithi077 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555133 2024-02-09 19:47:25 2024-02-10 06:07:06 2024-02-10 06:52:02 0:44:56 0:35:28 0:09:28 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-10T06:40:00.000161+0000 mon.a (mon.0) 2926 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

fail 7555134 2024-02-09 19:47:26 2024-02-10 06:07:06 2024-02-10 06:33:52 0:26:46 0:17:01 0:09:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-02-10T06:23:30.844706+0000 mon.smithi047 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555135 2024-02-09 19:47:27 2024-02-10 06:07:07 2024-02-10 06:31:22 0:24:15 0:13:30 0:10:45 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-02-10T06:27:10.765744+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555136 2024-02-09 19:47:27 2024-02-10 06:07:07 2024-02-10 06:49:41 0:42:34 0:23:33 0:19:01 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-02-10T06:36:39.375945+0000 mon.smithi140 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555137 2024-02-09 19:47:28 2024-02-10 06:10:58 2024-02-10 06:44:45 0:33:47 0:23:18 0:10:29 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7555138 2024-02-09 19:47:29 2024-02-10 06:10:58 2024-02-10 06:36:20 0:25:22 0:13:22 0:12:00 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-02-10T06:29:56.906423+0000 mon.smithi006 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555139 2024-02-09 19:47:30 2024-02-10 06:12:39 2024-02-10 06:39:47 0:27:08 0:16:03 0:11:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-02-10T06:33:27.962549+0000 mon.a (mon.0) 246 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555140 2024-02-09 19:47:31 2024-02-10 06:14:39 2024-02-10 06:48:19 0:33:40 0:25:01 0:08:39 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-02-10T06:34:04.534954+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555141 2024-02-09 19:47:32 2024-02-10 06:14:40 2024-02-10 06:39:56 0:25:16 0:15:01 0:10:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

Command failed on smithi144 with status 123: "time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --"

fail 7555142 2024-02-09 19:47:33 2024-02-10 06:14:50 2024-02-10 06:37:20 0:22:30 0:11:10 0:11:20 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-02-10T06:31:47.422599+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555143 2024-02-09 19:47:33 2024-02-10 06:16:21 2024-02-10 06:43:48 0:27:27 0:14:46 0:12:41 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-02-10T06:35:19.289092+0000 mon.smithi072 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555144 2024-02-09 19:47:34 2024-02-10 06:18:11 2024-02-10 07:21:21 1:03:10 0:50:55 0:12:15 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi073 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c2b87a2-c7de-11ee-95b8-87774f69a715 -e sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -- bash -c 'ceph orch ps'"

pass 7555145 2024-02-09 19:47:35 2024-02-10 06:18:12 2024-02-10 06:39:33 0:21:21 0:11:52 0:09:29 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_adoption} 1
fail 7555146 2024-02-09 19:47:36 2024-02-10 06:18:12 2024-02-10 06:51:24 0:33:12 0:20:21 0:12:51 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-02-10T06:40:52.746375+0000 mon.smithi078 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555147 2024-02-09 19:47:37 2024-02-10 06:19:03 2024-02-10 07:42:06 1:23:03 1:10:56 0:12:07 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-10T06:42:00.868171+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555148 2024-02-09 19:47:38 2024-02-10 06:20:13 2024-02-10 06:54:01 0:33:48 0:22:03 0:11:45 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-02-10T06:43:44.126041+0000 mon.smithi003 (mon.0) 256 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555149 2024-02-09 19:47:39 2024-02-10 06:22:14 2024-02-10 06:47:12 0:24:58 0:14:02 0:10:56 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-02-10T06:39:24.532629+0000 mon.smithi005 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555150 2024-02-09 19:47:40 2024-02-10 06:22:14 2024-02-10 07:23:19 1:01:05 0:52:06 0:08:59 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-02-10T06:37:02.832956+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7555151 2024-02-09 19:47:40 2024-02-10 06:22:24 2024-02-10 06:45:19 0:22:55 0:12:19 0:10:36 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_ca_signed_key} 2
fail 7555152 2024-02-09 19:47:41 2024-02-10 06:22:25 2024-02-10 06:52:20 0:29:55 0:17:55 0:12:00 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-02-10T06:40:37.083230+0000 mon.smithi062 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555153 2024-02-09 19:47:42 2024-02-10 06:22:25 2024-02-10 06:45:12 0:22:47 0:14:10 0:08:37 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm} 1
fail 7555154 2024-02-09 19:47:43 2024-02-10 06:22:26 2024-02-10 06:40:07 0:17:41 0:07:16 0:10:25 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} 1
Failure Reason:

Command failed on smithi149 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7555155 2024-02-09 19:47:44 2024-02-10 06:22:26 2024-02-10 06:54:07 0:31:41 0:22:05 0:09:36 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-02-10T06:43:45.777013+0000 mon.smithi007 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555156 2024-02-09 19:47:44 2024-02-10 06:22:26 2024-02-10 07:07:14 0:44:48 0:35:01 0:09:47 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-10T06:50:00.000246+0000 mon.a (mon.0) 2217 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log

pass 7555157 2024-02-09 19:47:45 2024-02-10 06:22:27 2024-02-10 07:04:23 0:41:56 0:32:08 0:09:48 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7555158 2024-02-09 19:47:46 2024-02-10 06:22:27 2024-02-10 07:07:45 0:45:18 0:35:34 0:09:44 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-10T06:41:50.788539+0000 mon.smithi008 (mon.0) 494 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7555159 2024-02-09 19:47:47 2024-02-10 06:22:27 2024-02-10 07:05:05 0:42:38 0:33:29 0:09:09 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7555160 2024-02-09 19:47:48 2024-02-10 06:44:10 704 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
fail 7555161 2024-02-09 19:47:49 2024-02-10 06:22:28 2024-02-10 06:48:55 0:26:27 0:12:38 0:13:49 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-02-10T06:43:20.065015+0000 mon.smithi022 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555162 2024-02-09 19:47:49 2024-02-10 06:25:59 2024-02-10 07:12:05 0:46:06 0:22:45 0:23:21 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7555163 2024-02-09 19:47:50 2024-02-10 06:37:51 2024-02-10 07:02:38 0:24:47 0:14:31 0:10:16 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7555164 2024-02-09 19:47:51 2024-02-10 06:37:51 2024-02-10 07:00:43 0:22:52 0:13:29 0:09:23 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-02-10T06:55:12.506691+0000 mon.smithi050 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555165 2024-02-09 19:47:52 2024-02-10 06:37:52 2024-02-10 06:58:50 0:20:58 0:11:17 0:09:41 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
fail 7555166 2024-02-09 19:47:53 2024-02-10 06:37:52 2024-02-10 06:59:41 0:21:49 0:11:29 0:10:20 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-02-10T06:55:04.902897+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555167 2024-02-09 19:47:54 2024-02-10 06:37:52 2024-02-10 06:56:23 0:18:31 0:08:08 0:10:23 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi146 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7555168 2024-02-09 19:47:54 2024-02-10 06:37:53 2024-02-10 07:02:55 0:25:02 0:14:05 0:10:57 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-02-10T06:55:08.809921+0000 mon.smithi151 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555169 2024-02-09 19:47:55 2024-02-10 06:37:53 2024-02-10 07:22:40 0:44:47 0:34:36 0:10:11 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-10T07:10:00.000215+0000 mon.a (mon.0) 1205 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7555170 2024-02-09 19:47:56 2024-02-10 06:37:53 2024-02-10 07:10:29 0:32:36 0:20:32 0:12:04 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-02-10T06:59:53.701884+0000 mon.smithi006 (mon.0) 256 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555171 2024-02-09 19:47:57 2024-02-10 06:37:54 2024-02-10 07:01:59 0:24:05 0:13:23 0:10:42 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_timeout} 1
fail 7555172 2024-02-09 19:47:58 2024-02-10 06:37:54 2024-02-10 07:04:08 0:26:14 0:14:24 0:11:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-02-10T06:56:26.638365+0000 mon.smithi039 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555173 2024-02-09 19:47:58 2024-02-10 06:37:54 2024-02-10 07:00:47 0:22:53 0:13:37 0:09:16 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-02-10T06:55:07.573578+0000 mon.smithi105 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555174 2024-02-09 19:47:59 2024-02-10 06:37:55 2024-02-10 07:01:28 0:23:33 0:14:18 0:09:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-02-10T06:55:10.871697+0000 mon.smithi047 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555175 2024-02-09 19:48:00 2024-02-10 06:37:55 2024-02-10 07:01:52 0:23:57 0:13:40 0:10:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
fail 7555176 2024-02-09 19:48:01 2024-02-10 06:37:55 2024-02-10 10:26:52 3:48:57 3:36:05 0:12:52 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi132 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7555177 2024-02-09 19:48:02 2024-02-10 06:37:56 2024-02-10 07:26:40 0:48:44 0:37:38 0:11:06 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-02-10T06:56:08.195748+0000 mon.a (mon.0) 168 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7555178 2024-02-09 19:48:03 2024-02-10 06:37:56 2024-02-10 07:12:23 0:34:27 0:21:50 0:12:37 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-02-10T07:01:53.429518+0000 mon.smithi049 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555179 2024-02-09 19:48:03 2024-02-10 06:39:37 2024-02-10 07:28:22 0:48:45 0:32:32 0:16:13 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-02-10T07:06:29.091088+0000 mon.a (mon.0) 180 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555180 2024-02-09 19:48:04 2024-02-10 06:44:18 2024-02-10 07:19:35 0:35:17 0:23:04 0:12:13 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-02-10T07:08:30.929939+0000 mon.a (mon.0) 199 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555181 2024-02-09 19:48:05 2024-02-10 06:45:18 2024-02-10 07:34:33 0:49:15 0:36:43 0:12:32 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-10T07:20:00.000202+0000 mon.a (mon.0) 1478 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

pass 7555182 2024-02-09 19:48:06 2024-02-10 06:45:29 2024-02-10 07:18:57 0:33:28 0:15:30 0:17:58 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
dead 7555183 2024-02-09 19:48:07 2024-02-10 06:53:10 2024-02-10 07:13:05 0:19:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds

fail 7555184 2024-02-09 19:48:08 2024-02-10 06:53:10 2024-02-10 07:17:33 0:24:23 0:12:11 0:12:12 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-02-10T07:11:23.900252+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555185 2024-02-09 19:48:09 2024-02-10 06:53:11 2024-02-10 07:22:42 0:29:31 0:19:44 0:09:47 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-02-10T07:13:14.924223+0000 mon.smithi143 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555186 2024-02-09 19:48:09 2024-02-10 06:53:11 2024-02-10 07:24:09 0:30:58 0:15:42 0:15:16 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-02-10T07:15:35.720812+0000 mon.smithi019 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555187 2024-02-09 19:48:10 2024-02-10 06:53:12 2024-02-10 07:16:17 0:23:05 0:13:14 0:09:51 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7555188 2024-02-09 19:48:11 2024-02-10 06:53:12 2024-02-10 07:27:36 0:34:24 0:24:08 0:10:16 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-02-10T07:15:01.786867+0000 mon.smithi104 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555189 2024-02-09 19:48:12 2024-02-10 06:53:12 2024-02-10 07:36:46 0:43:34 0:33:59 0:09:35 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7555190 2024-02-09 19:48:13 2024-02-10 06:53:13 2024-02-10 07:20:18 0:27:05 0:16:56 0:10:09 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-02-10T07:09:51.912589+0000 mon.smithi169 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555191 2024-02-09 19:48:13 2024-02-10 06:53:13 2024-02-10 07:22:23 0:29:10 0:19:21 0:09:49 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-02-10T07:18:35.239404+0000 mon.a (mon.0) 581 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555192 2024-02-09 19:48:14 2024-02-10 06:53:13 2024-02-10 07:19:00 0:25:47 0:13:26 0:12:21 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-02-10T07:11:18.986994+0000 mon.smithi133 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555193 2024-02-09 19:48:15 2024-02-10 06:53:14 2024-02-10 07:19:09 0:25:55 0:15:12 0:10:43 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-02-10T07:11:11.227174+0000 mon.smithi078 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555194 2024-02-09 19:48:16 2024-02-10 06:53:14 2024-02-10 08:07:55 1:14:41 1:03:04 0:11:37 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-10T07:30:00.438885+0000 mon.a (mon.0) 1545 : cluster 3 [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets" in cluster log

fail 7555195 2024-02-09 19:48:17 2024-02-10 06:53:14 2024-02-10 07:12:22 0:19:08 0:07:29 0:11:39 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} 3
Failure Reason:

Command failed on smithi057 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7555196 2024-02-09 19:48:18 2024-02-10 06:53:15 2024-02-10 07:28:34 0:35:19 0:20:30 0:14:49 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli} 1
fail 7555197 2024-02-09 19:48:18 2024-02-10 06:58:56 2024-02-10 07:33:58 0:35:02 0:21:08 0:13:54 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-02-10T07:22:47.729781+0000 mon.smithi142 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555198 2024-02-09 19:48:19 2024-02-10 07:01:56 2024-02-10 07:35:52 0:33:56 0:23:41 0:10:15 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7555199 2024-02-09 19:48:20 2024-02-10 07:02:47 2024-02-10 07:26:09 0:23:22 0:11:36 0:11:46 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
fail 7555200 2024-02-09 19:48:21 2024-02-10 07:04:28 2024-02-10 07:40:38 0:36:10 0:25:12 0:10:58 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-02-10T07:25:02.620884+0000 mon.a (mon.0) 178 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7555201 2024-02-09 19:48:22 2024-02-10 07:05:08 2024-02-10 07:32:19 0:27:11 0:13:17 0:13:54 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-02-10T07:25:49.836742+0000 mon.smithi063 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555202 2024-02-09 19:48:22 2024-02-10 07:08:39 2024-02-10 07:32:54 0:24:15 0:13:07 0:11:08 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
fail 7555203 2024-02-09 19:48:23 2024-02-10 07:08:40 2024-02-10 07:31:25 0:22:45 0:11:47 0:10:58 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-02-10T07:25:57.547186+0000 mon.a (mon.0) 249 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555204 2024-02-09 19:48:24 2024-02-10 07:08:40 2024-02-10 07:47:47 0:39:07 0:28:05 0:11:02 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-02-10T07:26:33.709877+0000 mon.a (mon.0) 466 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7555205 2024-02-09 19:48:25 2024-02-10 07:08:40 2024-02-10 07:44:44 0:36:04 0:24:04 0:12:00 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_orch_cli_mon} 5
fail 7555206 2024-02-09 19:48:26 2024-02-10 07:08:41 2024-02-10 07:34:28 0:25:47 0:14:37 0:11:10 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-02-10T07:26:17.825144+0000 mon.smithi164 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555207 2024-02-09 19:48:26 2024-02-10 07:08:41 2024-02-10 07:33:30 0:24:49 0:14:04 0:10:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-02-10T07:26:20.408385+0000 mon.smithi007 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555208 2024-02-09 19:48:27 2024-02-10 07:08:41 2024-02-10 08:17:46 1:09:05 0:56:43 0:12:22 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi089 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 69b4ecac-c7e5-11ee-95b8-87774f69a715 -e sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''

fail 7555209 2024-02-09 19:48:28 2024-02-10 07:08:42 2024-02-10 07:44:59 0:36:17 0:25:27 0:10:50 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-02-10T07:30:10.846758+0000 mon.smithi067 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555210 2024-02-09 19:48:29 2024-02-10 07:08:42 2024-02-10 07:34:47 0:26:05 0:14:13 0:11:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_rgw_multisite} 3
fail 7555211 2024-02-09 19:48:29 2024-02-10 07:08:43 2024-02-10 07:35:45 0:27:02 0:15:18 0:11:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-02-10T07:27:22.898937+0000 mon.smithi105 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555212 2024-02-09 19:48:30 2024-02-10 07:08:43 2024-02-10 08:14:35 1:05:52 0:55:27 0:10:25 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-10T07:40:00.000146+0000 mon.a (mon.0) 1241 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7555213 2024-02-09 19:48:31 2024-02-10 07:08:43 2024-02-10 07:43:41 0:34:58 0:22:52 0:12:06 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-02-10T07:33:34.146923+0000 mon.a (mon.0) 199 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555214 2024-02-09 19:48:32 2024-02-10 07:08:44 2024-02-10 07:38:57 0:30:13 0:19:11 0:11:02 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-02-10T07:29:54.479102+0000 mon.smithi175 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555215 2024-02-09 19:48:33 2024-02-10 07:08:44 2024-02-10 07:35:50 0:27:06 0:13:22 0:13:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-02-10T07:29:24.211703+0000 mon.smithi083 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555216 2024-02-09 19:48:33 2024-02-10 07:12:15 2024-02-10 07:49:49 0:37:34 0:20:51 0:16:43 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-02-10T07:39:16.075571+0000 mon.smithi140 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555217 2024-02-09 19:48:34 2024-02-10 07:13:15 2024-02-10 08:17:31 1:04:16 0:52:04 0:12:12 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-10T07:50:00.000131+0000 mon.a (mon.0) 2853 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log

pass 7555218 2024-02-09 19:48:35 2024-02-10 07:37:44 557 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_adoption} 1
fail 7555219 2024-02-09 19:48:36 2024-02-10 07:18:37 2024-02-10 08:00:21 0:41:44 0:31:15 0:10:29 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1707550972.3561332 mon.smithi125 (mon.0) 688 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7555220 2024-02-09 19:48:36 2024-02-10 07:19:07 2024-02-10 07:47:54 0:28:47 0:15:33 0:13:14 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7555221 2024-02-09 19:48:37 2024-02-10 07:22:48 2024-02-10 07:46:44 0:23:56 0:13:26 0:10:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-02-10T07:39:52.674389+0000 mon.smithi037 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555222 2024-02-09 19:48:38 2024-02-10 07:23:59 2024-02-10 07:47:56 0:23:57 0:11:31 0:12:26 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-02-10T07:42:24.982688+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7555223 2024-02-09 19:48:39 2024-02-10 07:23:59 2024-02-10 07:49:05 0:25:06 0:14:34 0:10:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-02-10T07:40:35.423352+0000 mon.smithi062 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555224 2024-02-09 19:48:39 2024-02-10 07:23:59 2024-02-10 07:49:54 0:25:55 0:13:29 0:12:26 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-02-10T07:42:24.972571+0000 mon.smithi158 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555225 2024-02-09 19:48:40 2024-02-10 07:24:00 2024-02-10 07:44:43 0:20:43 0:11:40 0:09:03 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7555226 2024-02-09 19:48:41 2024-02-10 07:24:00 2024-02-10 08:34:01 1:10:01 1:00:34 0:09:27 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-02-10T07:41:22.246797+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7555227 2024-02-09 19:48:42 2024-02-10 08:05:06 1867 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-10T07:50:00.000206+0000 mon.a (mon.0) 986 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7555228 2024-02-09 19:48:43 2024-02-10 07:24:01 2024-02-10 07:53:36 0:29:35 0:20:32 0:09:03 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-02-10T07:44:03.244292+0000 mon.smithi077 (mon.0) 257 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7555229 2024-02-09 19:48:43 2024-02-10 07:24:01 2024-02-10 07:55:07 0:31:06 0:19:18 0:11:48 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
fail 7555230 2024-02-09 19:48:44 2024-02-10 07:24:01 2024-02-10 07:49:47 0:25:46 0:14:30 0:11:16 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-02-10T07:40:59.505084+0000 mon.smithi121 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555231 2024-02-09 19:48:45 2024-02-10 07:24:02 2024-02-10 07:50:45 0:26:43 0:15:51 0:10:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-02-10T07:42:51.793336+0000 mon.smithi006 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555232 2024-02-09 19:48:46 2024-02-10 07:24:02 2024-02-10 07:47:03 0:23:01 0:13:05 0:09:56 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-02-10T07:40:48.104018+0000 mon.smithi032 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7555233 2024-02-09 19:48:47 2024-02-10 07:24:02 2024-02-10 08:33:59 1:09:57 0:56:06 0:13:51 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi100 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a28ba10e-c7e7-11ee-95b8-87774f69a715 -e sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''