Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7634828 2024-04-01 21:08:43 2024-04-02 04:12:55 2024-04-02 05:24:38 1:11:43 1:01:50 0:09:53 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-02T04:34:39.259094+0000 mon.a (mon.0) 199 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634829 2024-04-01 21:08:44 2024-04-02 04:12:55 2024-04-02 04:31:39 0:18:44 0:09:23 0:09:21 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7634830 2024-04-01 21:08:45 2024-04-02 04:12:56 2024-04-02 04:47:38 0:34:42 0:23:30 0:11:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-02T04:35:00.877696+0000 mon.smithi106 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634831 2024-04-01 21:08:46 2024-04-02 04:13:46 2024-04-02 05:22:06 1:08:20 0:56:30 0:11:50 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-02T04:35:37.362303+0000 mon.a (mon.0) 478 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7634832 2024-04-01 21:08:48 2024-04-02 04:14:07 2024-04-02 04:44:51 0:30:44 0:15:15 0:15:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-02T04:36:44.312226+0000 mon.smithi049 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634833 2024-04-01 21:08:49 2024-04-02 04:14:07 2024-04-02 04:35:55 0:21:48 0:11:14 0:10:34 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7634834 2024-04-01 21:08:50 2024-04-02 04:14:37 2024-04-02 04:50:46 0:36:09 0:25:26 0:10:43 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7634835 2024-04-01 21:08:51 2024-04-02 04:14:38 2024-04-02 04:31:31 0:16:53 0:07:14 0:09:39 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7634836 2024-04-01 21:08:52 2024-04-02 04:14:38 2024-04-02 04:41:41 0:27:03 0:14:58 0:12:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-02T04:33:58.349695+0000 mon.smithi170 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634837 2024-04-01 21:08:53 2024-04-02 04:14:38 2024-04-02 05:12:26 0:57:48 0:47:42 0:10:06 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-02T04:36:04.308495+0000 mon.a (mon.0) 195 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634838 2024-04-01 21:08:54 2024-04-02 04:14:49 2024-04-02 05:02:58 0:48:09 0:37:46 0:10:23 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7634839 2024-04-01 21:08:55 2024-04-02 04:14:49 2024-04-02 05:04:50 0:50:01 0:38:00 0:12:01 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_orch_cli_mon} 5
fail 7634840 2024-04-01 21:08:56 2024-04-02 04:16:10 2024-04-02 04:46:19 0:30:09 0:20:01 0:10:08 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-02T04:37:17.063430+0000 mon.smithi169 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634841 2024-04-01 21:08:57 2024-04-02 04:16:30 2024-04-02 04:48:34 0:32:04 0:21:05 0:10:59 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-02T04:37:26.024468+0000 mon.smithi103 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634842 2024-04-01 21:08:58 2024-04-02 04:16:31 2024-04-02 04:46:32 0:30:01 0:19:31 0:10:30 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_extra_daemon_features} 2
fail 7634843 2024-04-01 21:08:59 2024-04-02 04:17:01 2024-04-02 04:42:36 0:25:35 0:14:42 0:10:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-04-02T04:34:54.321517+0000 mon.smithi017 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634844 2024-04-01 21:09:00 2024-04-02 04:17:42 2024-04-02 04:50:54 0:33:12 0:22:01 0:11:11 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} 2
fail 7634845 2024-04-01 21:09:01 2024-04-02 04:19:02 2024-04-02 04:48:22 0:29:20 0:18:36 0:10:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-04-02T04:35:59.316296+0000 mon.smithi071 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634846 2024-04-01 21:09:02 2024-04-02 04:19:13 2024-04-02 04:48:31 0:29:18 0:17:38 0:11:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_host_drain} 3
fail 7634847 2024-04-01 21:09:03 2024-04-02 04:21:13 2024-04-02 04:53:43 0:32:30 0:21:50 0:10:40 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-04-02T04:42:37.569690+0000 mon.smithi037 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634848 2024-04-01 21:09:04 2024-04-02 04:21:34 2024-04-02 05:11:10 0:49:36 0:38:49 0:10:47 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7634849 2024-04-01 21:09:05 2024-04-02 04:21:44 2024-04-02 05:04:54 0:43:10 0:33:54 0:09:16 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-02T04:46:49.936349+0000 mon.smithi053 (mon.0) 872 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7634850 2024-04-01 21:09:06 2024-04-02 04:21:55 2024-04-02 05:20:53 0:58:58 0:46:05 0:12:53 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7634851 2024-04-01 21:09:07 2024-04-02 04:24:05 2024-04-02 04:45:14 0:21:09 0:10:45 0:10:24 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_adoption} 1
pass 7634852 2024-04-01 21:09:08 2024-04-02 04:24:06 2024-04-02 04:45:24 0:21:18 0:12:13 0:09:05 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
fail 7634853 2024-04-01 21:09:09 2024-04-02 04:24:26 2024-04-02 04:47:40 0:23:14 0:12:53 0:10:21 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-02T04:40:49.096638+0000 mon.smithi144 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634854 2024-04-01 21:09:10 2024-04-02 04:24:47 2024-04-02 05:04:27 0:39:40 0:25:06 0:14:34 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-02T04:45:36.997362+0000 mon.a (mon.0) 209 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7634855 2024-04-01 21:09:11 2024-04-02 04:27:07 2024-04-02 04:53:49 0:26:42 0:14:38 0:12:04 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7634856 2024-04-01 21:09:12 2024-04-02 04:28:48 2024-04-02 04:52:32 0:23:44 0:13:28 0:10:16 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-04-02T04:46:13.663352+0000 mon.smithi045 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634857 2024-04-01 21:09:13 2024-04-02 04:29:38 2024-04-02 04:50:36 0:20:58 0:11:44 0:09:14 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
fail 7634858 2024-04-01 21:09:14 2024-04-02 04:29:59 2024-04-02 04:52:13 0:22:14 0:11:42 0:10:32 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-02T04:47:39.252164+0000 mon.a (mon.0) 210 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7634859 2024-04-01 21:09:15 2024-04-02 04:30:39 2024-04-02 05:27:39 0:57:00 0:46:48 0:10:12 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-02T04:58:27.647337+0000 mon.a (mon.0) 946 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7634860 2024-04-01 21:09:16 2024-04-02 04:30:40 2024-04-02 04:53:34 0:22:54 0:13:55 0:08:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-04-02T04:47:17.375525+0000 mon.smithi131 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634861 2024-04-01 21:09:17 2024-04-02 04:30:40 2024-04-02 04:57:40 0:27:00 0:18:02 0:08:58 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7634862 2024-04-01 21:09:18 2024-04-02 04:30:40 2024-04-02 05:01:07 0:30:27 0:20:20 0:10:07 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-02T04:51:52.512462+0000 mon.smithi079 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634863 2024-04-01 21:09:19 2024-04-02 04:31:21 2024-04-02 05:04:39 0:33:18 0:23:33 0:09:45 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7634864 2024-04-01 21:09:20 2024-04-02 04:31:31 2024-04-02 04:56:49 0:25:18 0:14:28 0:10:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-02T04:49:06.009490+0000 mon.smithi150 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634865 2024-04-01 21:09:21 2024-04-02 04:31:32 2024-04-02 05:07:51 0:36:19 0:25:24 0:10:55 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7634866 2024-04-01 21:09:22 2024-04-02 04:32:12 2024-04-02 04:58:20 0:26:08 0:14:06 0:12:02 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-02T04:50:50.604412+0000 mon.smithi023 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634867 2024-04-01 21:09:23 2024-04-02 04:32:23 2024-04-02 04:49:52 0:17:29 0:07:14 0:10:15 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi102 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

dead 7634868 2024-04-01 21:09:24 2024-04-02 04:32:23 2024-04-02 16:41:24 12:09:01 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

pass 7634869 2024-04-01 21:09:25 2024-04-02 04:32:23 2024-04-02 04:56:02 0:23:39 0:13:44 0:09:55 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_cephadm_timeout} 1
fail 7634870 2024-04-01 21:09:26 2024-04-02 04:33:04 2024-04-02 04:58:48 0:25:44 0:14:39 0:11:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-04-02T04:50:30.589995+0000 mon.smithi059 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634871 2024-04-01 21:09:27 2024-04-02 04:33:45 2024-04-02 05:14:31 0:40:46 0:32:05 0:08:41 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-02T05:00:00.000146+0000 mon.a (mon.0) 1051 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7634872 2024-04-01 21:09:28 2024-04-02 04:33:45 2024-04-02 05:05:51 0:32:06 0:21:25 0:10:41 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-04-02T05:00:00.000264+0000 mon.smithi003 (mon.0) 790 : cluster 4 [ERR] MDS_ALL_DOWN: 1 filesystem is offline" in cluster log

pass 7634873 2024-04-01 21:09:29 2024-04-02 04:34:16 2024-04-02 05:02:04 0:27:48 0:17:43 0:10:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7634874 2024-04-01 21:09:30 2024-04-02 04:34:46 2024-04-02 05:01:08 0:26:22 0:15:21 0:11:01 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7634875 2024-04-01 21:09:31 2024-04-02 04:35:27 2024-04-02 05:00:29 0:25:02 0:15:27 0:09:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-02T04:52:00.399168+0000 mon.smithi100 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634876 2024-04-01 21:09:32 2024-04-02 04:35:27 2024-04-02 04:59:10 0:23:43 0:12:13 0:11:30 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-02T04:52:29.496724+0000 mon.a (mon.0) 267 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7634877 2024-04-01 21:09:33 2024-04-02 04:35:57 2024-04-02 05:09:08 0:33:11 0:19:56 0:13:15 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-02T05:00:10.471865+0000 mon.smithi026 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

dead 7634878 2024-04-01 21:09:34 2024-04-02 04:39:38 2024-04-03 01:27:03 20:47:25 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

hit max job timeout

fail 7634879 2024-04-01 21:09:35 2024-04-02 04:44:49 2024-04-02 05:10:48 0:25:59 0:15:55 0:10:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-02T05:02:37.989488+0000 mon.smithi164 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634880 2024-04-01 21:09:36 2024-04-02 04:45:30 2024-04-02 05:57:33 1:12:03 1:01:01 0:11:02 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-02T05:07:44.265958+0000 mon.a (mon.0) 198 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7634881 2024-04-01 21:09:37 2024-04-02 04:46:10 2024-04-02 05:11:56 0:25:46 0:15:19 0:10:27 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7634882 2024-04-01 21:09:38 2024-04-02 04:46:11 2024-04-02 05:37:08 0:50:57 0:41:46 0:09:11 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7634883 2024-04-01 21:09:39 2024-04-02 04:46:11 2024-04-02 05:15:33 0:29:22 0:19:33 0:09:49 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_orch_cli} 1
fail 7634884 2024-04-01 21:09:40 2024-04-02 04:46:11 2024-04-02 05:20:06 0:33:55 0:23:35 0:10:20 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-02T05:06:26.935906+0000 mon.smithi170 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634885 2024-04-01 21:09:41 2024-04-02 04:46:12 2024-04-02 05:45:50 0:59:38 0:49:32 0:10:06 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-02T05:07:53.666500+0000 mon.a (mon.0) 198 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634886 2024-04-01 21:09:42 2024-04-02 04:46:12 2024-04-02 05:11:07 0:24:55 0:16:13 0:08:42 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-02T05:02:05.060118+0000 mon.smithi138 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634887 2024-04-01 21:09:43 2024-04-02 04:46:13 2024-04-02 05:12:35 0:26:22 0:11:33 0:14:49 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_ca_signed_key} 2
fail 7634888 2024-04-01 21:09:44 2024-04-02 04:46:13 2024-04-02 05:08:55 0:22:42 0:12:58 0:09:44 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-02T05:02:19.527575+0000 mon.smithi031 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634889 2024-04-01 21:09:45 2024-04-02 04:46:23 2024-04-02 05:13:52 0:27:29 0:14:58 0:12:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-02T05:05:33.881163+0000 mon.smithi033 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634890 2024-04-01 21:09:46 2024-04-02 04:48:34 2024-04-02 05:22:04 0:33:30 0:21:15 0:12:15 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-02T05:10:50.014102+0000 mon.smithi012 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634891 2024-04-01 21:09:47 2024-04-02 04:50:45 2024-04-02 05:12:32 0:21:47 0:11:49 0:09:58 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

pass 7634892 2024-04-01 21:09:48 2024-04-02 04:50:55 2024-04-02 05:26:54 0:35:59 0:24:58 0:11:01 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7634893 2024-04-01 21:09:49 2024-04-02 04:50:55 2024-04-02 05:26:23 0:35:28 0:22:45 0:12:43 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-04-02T05:20:00.000234+0000 mon.a (mon.0) 1081 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

fail 7634894 2024-04-01 21:09:50 2024-04-02 04:53:26 2024-04-02 08:47:22 3:53:56 3:40:26 0:13:30 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi123 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7634895 2024-04-01 21:09:51 2024-04-02 04:53:56 2024-04-02 05:37:41 0:43:45 0:24:46 0:18:59 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli_mon} 5
pass 7634896 2024-04-01 21:09:52 2024-04-02 05:01:18 2024-04-02 05:29:10 0:27:52 0:17:12 0:10:40 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
fail 7634897 2024-04-01 21:09:53 2024-04-02 05:01:48 2024-04-02 05:27:58 0:26:10 0:14:19 0:11:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-02T05:19:59.717998+0000 mon.smithi019 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634898 2024-04-01 21:09:54 2024-04-02 05:01:49 2024-04-02 05:24:25 0:22:36 0:13:10 0:09:26 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
dead 7634899 2024-04-01 21:09:55 2024-04-02 05:01:49 2024-04-02 17:11:00 12:09:11 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

hit max job timeout

fail 7634900 2024-04-01 21:09:56 2024-04-02 05:01:49 2024-04-02 05:27:36 0:25:47 0:14:29 0:11:18 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-02T05:19:22.297539+0000 mon.smithi153 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634901 2024-04-01 21:09:57 2024-04-02 05:01:50 2024-04-02 06:01:33 0:59:43 0:49:58 0:09:45 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-02T05:23:42.844120+0000 mon.a (mon.0) 203 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

dead 7634902 2024-04-01 21:09:58 2024-04-02 05:01:50 2024-04-02 17:10:54 12:09:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

hit max job timeout

pass 7634903 2024-04-01 21:09:59 2024-04-02 05:01:50 2024-04-02 05:19:56 0:18:06 0:06:11 0:11:55 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_cephadm_repos} 1
fail 7634904 2024-04-01 21:10:00 2024-04-02 05:01:51 2024-04-02 05:55:57 0:54:06 0:42:25 0:11:41 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi037 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a781000-f0b0-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

fail 7634905 2024-04-01 21:10:01 2024-04-02 05:01:51 2024-04-02 05:40:28 0:38:37 0:26:12 0:12:25 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-04-02T05:24:46.343542+0000 mon.smithi023 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634906 2024-04-01 21:10:02 2024-04-02 05:01:51 2024-04-02 05:26:48 0:24:57 0:15:25 0:09:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-04-02T05:18:07.532693+0000 mon.smithi059 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634907 2024-04-01 21:10:03 2024-04-02 05:01:52 2024-04-02 05:24:49 0:22:57 0:13:17 0:09:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_extra_daemon_features} 2
fail 7634908 2024-04-01 21:10:04 2024-04-02 05:01:52 2024-04-02 06:06:40 1:04:48 0:54:41 0:10:07 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7634909 2024-04-01 21:10:05 2024-04-02 05:01:53 2024-04-02 05:19:13 0:17:20 0:07:25 0:09:55 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/calico rook/1.7.2} 1
Failure Reason:

Command failed on smithi181 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7634910 2024-04-01 21:10:06 2024-04-02 05:01:53 2024-04-02 05:21:13 0:19:20 0:10:00 0:09:20 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_adoption} 1
fail 7634911 2024-04-01 21:10:07 2024-04-02 05:01:53 2024-04-02 05:32:21 0:30:28 0:18:45 0:11:43 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-02T05:23:33.258564+0000 mon.smithi131 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634912 2024-04-01 21:10:08 2024-04-02 05:01:54 2024-04-02 05:25:49 0:23:55 0:12:54 0:11:01 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-04-02T05:20:29.488022+0000 mon.smithi148 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634913 2024-04-01 21:10:09 2024-04-02 05:01:54 2024-04-02 05:37:48 0:35:54 0:23:20 0:12:34 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
fail 7634914 2024-04-01 21:10:10 2024-04-02 05:04:45 2024-04-02 05:34:44 0:29:59 0:19:59 0:10:00 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-04-02T05:25:28.131722+0000 mon.smithi089 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634915 2024-04-01 21:10:11 2024-04-02 05:04:56 2024-04-02 06:28:43 1:23:47 1:11:57 0:11:50 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-02T05:40:00.000175+0000 mon.a (mon.0) 1378 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

pass 7634916 2024-04-01 21:10:12 2024-04-02 05:04:56 2024-04-02 05:41:31 0:36:35 0:23:59 0:12:36 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_host_drain} 3
pass 7634917 2024-04-01 21:10:13 2024-04-02 05:07:57 2024-04-02 05:31:05 0:23:08 0:14:34 0:08:34 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7634918 2024-04-01 21:10:14 2024-04-02 05:07:57 2024-04-02 05:37:07 0:29:10 0:14:09 0:15:01 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-02T05:28:59.668079+0000 mon.smithi169 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634919 2024-04-01 21:10:15 2024-04-02 05:11:58 2024-04-02 05:37:18 0:25:20 0:11:20 0:14:00 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-02T05:31:12.383784+0000 mon.a (mon.0) 210 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7634920 2024-04-01 21:10:16 2024-04-02 05:12:39 2024-04-02 05:38:31 0:25:52 0:14:25 0:11:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-02T05:30:07.782029+0000 mon.smithi064 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634921 2024-04-01 21:10:17 2024-04-02 05:12:49 2024-04-02 05:42:48 0:29:59 0:13:50 0:16:09 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-02T05:36:40.911921+0000 mon.smithi098 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634922 2024-04-01 21:10:18 2024-04-02 05:17:20 2024-04-02 05:40:05 0:22:45 0:13:27 0:09:18 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7634923 2024-04-01 21:10:19 2024-04-02 05:17:20 2024-04-02 09:07:18 3:49:58 3:36:56 0:13:02 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi203 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7634924 2024-04-01 21:10:20 2024-04-02 05:17:21 2024-04-02 05:46:28 0:29:07 0:19:36 0:09:31 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_cephadm_timeout} 1
fail 7634925 2024-04-01 21:10:21 2024-04-02 05:17:21 2024-04-02 05:48:54 0:31:33 0:20:35 0:10:58 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-04-02T05:38:01.068377+0000 mon.smithi053 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634926 2024-04-01 21:10:22 2024-04-02 05:17:21 2024-04-02 06:05:57 0:48:36 0:37:48 0:10:48 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-02T05:35:34.930035+0000 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7634927 2024-04-01 21:10:23 2024-04-02 05:17:22 2024-04-02 05:42:09 0:24:47 0:14:31 0:10:16 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-04-02T05:33:48.802082+0000 mon.smithi016 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634928 2024-04-01 21:10:24 2024-04-02 05:17:22 2024-04-02 06:03:44 0:46:22 0:30:26 0:15:56 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-02T05:50:00.000120+0000 mon.a (mon.0) 1085 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7634929 2024-04-01 21:10:25 2024-04-02 05:17:22 2024-04-02 06:19:15 1:01:53 0:51:30 0:10:23 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-02T05:50:00.000228+0000 mon.a (mon.0) 2567 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

pass 7634930 2024-04-01 21:10:26 2024-04-02 05:17:23 2024-04-02 05:46:09 0:28:46 0:18:59 0:09:47 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7634931 2024-04-01 21:10:27 2024-04-02 05:17:23 2024-04-02 05:43:24 0:26:01 0:15:08 0:10:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-02T05:35:19.902761+0000 mon.smithi177 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634932 2024-04-01 21:10:28 2024-04-02 05:17:24 2024-04-02 05:41:01 0:23:37 0:13:07 0:10:30 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-02T05:34:07.558070+0000 mon.smithi083 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634933 2024-04-01 21:10:29 2024-04-02 05:17:24 2024-04-02 05:48:56 0:31:32 0:22:04 0:09:28 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-02T05:37:23.565915+0000 mon.smithi082 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634934 2024-04-01 21:10:30 2024-04-02 05:17:24 2024-04-02 05:42:49 0:25:25 0:15:06 0:10:19 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7634935 2024-04-01 21:10:31 2024-04-02 05:17:25 2024-04-02 05:47:42 0:30:17 0:20:20 0:09:57 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7634936 2024-04-01 21:10:32 2024-04-02 05:17:25 2024-04-02 06:37:52 1:20:27 1:07:38 0:12:49 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7634937 2024-04-01 21:10:33 2024-04-02 05:20:06 2024-04-02 06:06:42 0:46:36 0:35:33 0:11:03 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-02T05:48:04.661925+0000 mon.smithi072 (mon.0) 869 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7634938 2024-04-01 21:10:34 2024-04-02 05:21:16 2024-04-02 06:07:12 0:45:56 0:33:42 0:12:14 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7634939 2024-04-01 21:10:35 2024-04-02 05:24:27 2024-04-02 05:49:59 0:25:32 0:15:00 0:10:32 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
pass 7634940 2024-04-01 21:10:36 2024-04-02 05:24:58 2024-04-02 05:50:56 0:25:58 0:13:54 0:12:04 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7634941 2024-04-01 21:10:37 2024-04-02 05:26:58 2024-04-02 06:04:20 0:37:22 0:24:24 0:12:58 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
fail 7634942 2024-04-01 21:10:38 2024-04-02 05:29:19 2024-04-02 05:58:27 0:29:08 0:16:50 0:12:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-02T05:47:31.038583+0000 mon.smithi106 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634943 2024-04-01 21:10:39 2024-04-02 05:31:09 2024-04-02 05:53:37 0:22:28 0:11:24 0:11:04 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
fail 7634944 2024-04-01 21:10:40 2024-04-02 05:32:40 2024-04-02 05:53:47 0:21:07 0:11:25 0:09:42 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-02T05:49:03.429231+0000 mon.a (mon.0) 212 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7634945 2024-04-01 21:10:41 2024-04-02 05:32:50 2024-04-02 06:00:47 0:27:57 0:17:20 0:10:37 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-02T05:51:04.012322+0000 mon.smithi066 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634946 2024-04-01 21:10:42 2024-04-02 05:32:51 2024-04-02 06:02:43 0:29:52 0:20:08 0:09:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7634947 2024-04-01 21:10:43 2024-04-02 05:32:51 2024-04-02 06:02:28 0:29:37 0:19:19 0:10:18 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-02T05:53:35.947427+0000 mon.smithi059 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634948 2024-04-01 21:10:44 2024-04-02 05:32:52 2024-04-02 06:34:31 1:01:39 0:51:28 0:10:11 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-02T05:54:26.126290+0000 mon.a (mon.0) 201 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634949 2024-04-01 21:10:45 2024-04-02 05:32:52 2024-04-02 06:04:30 0:31:38 0:21:20 0:10:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-02T05:53:16.751277+0000 mon.smithi032 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634950 2024-04-01 21:10:46 2024-04-02 05:32:52 2024-04-02 06:14:28 0:41:36 0:32:24 0:09:12 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

"2024-04-02T05:54:12.121538+0000 mon.a (mon.0) 201 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634951 2024-04-01 21:10:47 2024-04-02 05:32:53 2024-04-02 05:51:43 0:18:50 0:07:54 0:10:56 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/flannel rook/master} 3
Failure Reason:

Command failed on smithi029 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7634952 2024-04-01 21:10:48 2024-04-02 05:32:53 2024-04-02 05:57:50 0:24:57 0:15:06 0:09:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-02T05:49:30.916646+0000 mon.smithi132 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634953 2024-04-01 21:10:49 2024-04-02 05:32:53 2024-04-02 06:21:42 0:48:49 0:37:39 0:11:10 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-02T05:51:47.465545+0000 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7634954 2024-04-01 21:10:50 2024-04-02 05:32:54 2024-04-02 05:58:40 0:25:46 0:15:06 0:10:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_ca_signed_key} 2
fail 7634955 2024-04-01 21:10:51 2024-04-02 05:32:54 2024-04-02 05:51:16 0:18:22 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi189 with status 1: 'sudo yum install -y kernel'

fail 7634956 2024-04-01 21:10:52 2024-04-02 05:37:45 2024-04-02 05:57:28 0:19:43 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

Failed to reconnect to smithi189

fail 7634957 2024-04-01 21:10:53 2024-04-02 05:37:46 2024-04-02 06:00:59 0:23:13 0:14:07 0:09:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-02T05:54:26.120066+0000 mon.smithi084 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634958 2024-04-01 21:10:54 2024-04-02 05:37:56 2024-04-02 06:08:35 0:30:39 0:14:04 0:16:35 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-02T06:01:27.866661+0000 mon.smithi050 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634959 2024-04-01 21:10:55 2024-04-02 06:03:51 2024-04-02 06:33:53 0:30:02 0:19:43 0:10:19 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-04-02T06:24:46.926291+0000 mon.smithi006 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634960 2024-04-01 21:10:56 2024-04-02 06:03:51 2024-04-02 06:40:38 0:36:47 0:24:42 0:12:05 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7634961 2024-04-01 21:10:57 2024-04-02 06:06:42 2024-04-02 06:25:42 0:19:00 0:10:12 0:08:48 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7634962 2024-04-01 21:10:58 2024-04-02 06:06:42 2024-04-02 06:58:04 0:51:22 0:37:19 0:14:03 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-02T06:40:00.070124+0000 mon.a (mon.0) 2057 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7634963 2024-04-01 21:10:59 2024-04-02 06:11:03 2024-04-02 06:38:32 0:27:29 0:17:54 0:09:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-04-02T06:28:05.763457+0000 mon.smithi003 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634964 2024-04-01 21:11:00 2024-04-02 06:11:23 2024-04-02 06:40:40 0:29:17 0:12:44 0:16:33 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-02T06:35:02.498004+0000 mon.a (mon.0) 267 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7634965 2024-04-01 21:11:02 2024-04-02 06:17:25 2024-04-02 06:53:47 0:36:22 0:24:16 0:12:06 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rotate-keys} 2
fail 7634966 2024-04-01 21:11:03 2024-04-02 06:19:05 2024-04-02 06:43:57 0:24:52 0:16:10 0:08:42 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-04-02T06:35:25.489184+0000 mon.smithi062 (mon.0) 274 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634967 2024-04-01 21:11:04 2024-04-02 06:19:06 2024-04-02 06:39:50 0:20:44 0:09:13 0:11:31 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi027 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6cec793d9073f0594e2364b9b687ecaf22424cca TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

pass 7634968 2024-04-01 21:11:05 2024-04-02 06:19:06 2024-04-02 07:02:29 0:43:23 0:33:49 0:09:34 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7634969 2024-04-01 21:11:06 2024-04-02 06:19:06 2024-04-02 06:40:37 0:21:31 0:12:16 0:09:15 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
fail 7634970 2024-04-01 21:11:07 2024-04-02 06:19:07 2024-04-02 06:43:49 0:24:42 0:12:48 0:11:54 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-02T06:37:34.361889+0000 mon.smithi012 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634971 2024-04-01 21:11:08 2024-04-02 06:19:07 2024-04-02 06:48:32 0:29:25 0:19:15 0:10:10 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-04-02T06:39:00.286175+0000 mon.smithi112 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634972 2024-04-01 21:11:09 2024-04-02 06:19:07 2024-04-02 06:42:32 0:23:25 0:14:33 0:08:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-04-02T06:36:35.540563+0000 mon.smithi070 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634973 2024-04-01 21:11:10 2024-04-02 06:19:18 2024-04-02 06:45:47 0:26:29 0:14:17 0:12:12 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_extra_daemon_features} 2
fail 7634974 2024-04-01 21:11:11 2024-04-02 06:19:18 2024-04-02 06:45:25 0:26:07 0:14:46 0:11:21 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-02T06:37:19.369155+0000 mon.smithi033 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634975 2024-04-01 21:11:12 2024-04-02 06:19:19 2024-04-02 07:23:11 1:03:52 0:54:06 0:09:46 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-02T07:00:00.000142+0000 mon.a (mon.0) 1476 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7634976 2024-04-01 21:11:13 2024-04-02 06:19:19 2024-04-02 07:14:56 0:55:37 0:43:43 0:11:54 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi016 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fcace7fa-f0ba-11ee-b647-cb9ed24678a4 -e sha1=6cec793d9073f0594e2364b9b687ecaf22424cca -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7634977 2024-04-01 21:11:14 2024-04-02 06:19:19 2024-04-02 06:47:33 0:28:14 0:15:44 0:12:30 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7634978 2024-04-01 21:11:15 2024-04-02 06:19:20 2024-04-02 06:53:40 0:34:20 0:23:19 0:11:01 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-02T06:41:43.977293+0000 mon.smithi176 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634979 2024-04-01 21:11:16 2024-04-02 06:19:20 2024-04-02 06:48:36 0:29:16 0:19:23 0:09:53 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-02T06:39:04.212840+0000 mon.smithi082 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634980 2024-04-01 21:11:17 2024-04-02 06:19:20 2024-04-02 07:21:00 1:01:40 0:51:34 0:10:06 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-02T06:50:00.000167+0000 mon.a (mon.0) 2411 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

pass 7634981 2024-04-01 21:11:18 2024-04-02 06:19:21 2024-04-02 06:47:22 0:28:01 0:17:13 0:10:48 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_host_drain} 3
fail 7634982 2024-04-01 21:11:19 2024-04-02 06:19:21 2024-04-02 07:08:51 0:49:30 0:39:22 0:10:08 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7634983 2024-04-01 21:11:20 2024-04-02 06:19:21 2024-04-02 06:45:22 0:26:01 0:15:07 0:10:54 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
pass 7634984 2024-04-01 21:11:21 2024-04-02 06:19:22 2024-04-02 06:44:20 0:24:58 0:12:42 0:12:16 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
fail 7634985 2024-04-01 21:11:22 2024-04-02 06:19:22 2024-04-02 06:45:32 0:26:10 0:14:28 0:11:42 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-04-02T06:36:54.648458+0000 mon.smithi129 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634986 2024-04-01 21:11:23 2024-04-02 06:20:23 2024-04-02 06:50:38 0:30:15 0:21:07 0:09:08 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
fail 7634987 2024-04-01 21:11:24 2024-04-02 06:21:23 2024-04-02 06:45:01 0:23:38 0:12:27 0:11:11 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-02T06:39:00.454956+0000 mon.a (mon.0) 268 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7634988 2024-04-01 21:11:25 2024-04-02 06:21:34 2024-04-02 06:50:45 0:29:11 0:15:10 0:14:01 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-04-02T06:42:24.944060+0000 mon.smithi042 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634989 2024-04-01 21:11:26 2024-04-02 06:25:54 2024-04-02 06:49:34 0:23:40 0:13:34 0:10:06 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7634990 2024-04-01 21:11:27 2024-04-02 06:26:35 2024-04-02 07:01:22 0:34:47 0:21:58 0:12:49 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-02T06:51:00.660072+0000 mon.smithi037 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634991 2024-04-01 21:11:28 2024-04-02 06:29:56 2024-04-02 06:56:41 0:26:45 0:13:47 0:12:58 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-02T06:48:41.978186+0000 mon.smithi098 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7634992 2024-04-01 21:11:29 2024-04-02 06:31:26 2024-04-02 07:23:45 0:52:19 0:42:04 0:10:15 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-02T06:53:27.506017+0000 mon.a (mon.0) 201 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7634993 2024-04-01 21:11:31 2024-04-02 06:32:07 2024-04-02 06:51:29 0:19:22 0:07:20 0:12:02 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/host rook/1.7.2} 1
Failure Reason:

Command failed on smithi105 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7634994 2024-04-01 21:11:32 2024-04-02 06:33:47 2024-04-02 06:59:57 0:26:10 0:15:17 0:10:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-02T06:51:38.991261+0000 mon.smithi143 (mon.0) 271 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7634995 2024-04-01 21:11:33 2024-04-02 06:34:38 2024-04-02 07:04:10 0:29:32 0:19:47 0:09:45 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_monitoring_stack_basic} 3
pass 7634996 2024-04-01 21:11:34 2024-04-02 06:34:38 2024-04-02 07:19:32 0:44:54 0:36:16 0:08:38 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7634997 2024-04-01 21:11:35 2024-04-02 06:34:38 2024-04-02 06:57:38 0:23:00 0:13:24 0:09:36 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
fail 7634998 2024-04-01 21:11:36 2024-04-02 06:34:39 2024-04-02 07:01:29 0:26:50 0:17:26 0:09:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-02T06:51:40.018381+0000 mon.smithi046 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log