Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7617845 2024-03-22 21:08:37 2024-03-24 03:54:34 2024-03-24 04:29:52 0:35:18 0:22:10 0:13:08 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-24T04:17:02.995366+0000 mon.smithi016 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617846 2024-03-22 21:08:38 2024-03-24 03:56:25 2024-03-24 04:40:36 0:44:11 0:33:17 0:10:54 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711254112.382655 mon.smithi044 (mon.0) 685 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7617847 2024-03-22 21:08:39 2024-03-24 03:56:35 2024-03-24 04:43:11 0:46:36 0:33:40 0:12:56 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-24T04:30:00.000163+0000 mon.smithi022 (mon.0) 543 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

pass 7617848 2024-03-22 21:08:40 2024-03-24 03:58:06 2024-03-24 04:53:25 0:55:19 0:44:44 0:10:35 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7617849 2024-03-22 21:08:41 2024-03-24 03:58:56 2024-03-24 04:35:52 0:36:56 0:14:11 0:22:45 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
fail 7617850 2024-03-22 21:08:42 2024-03-24 04:09:48 2024-03-24 05:00:15 0:50:27 0:12:19 0:38:08 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-24T04:55:05.356418+0000 mon.smithi023 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617851 2024-03-22 21:08:43 2024-03-24 04:35:59 2024-03-24 05:24:38 0:48:39 0:33:30 0:15:09 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7617852 2024-03-22 21:08:44 2024-03-24 04:39:30 2024-03-24 05:21:35 0:42:05 0:17:40 0:24:25 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7617853 2024-03-22 21:08:45 2024-03-24 04:54:33 2024-03-24 05:17:32 0:22:59 0:12:41 0:10:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-24T05:11:04.359747+0000 mon.smithi044 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617854 2024-03-22 21:08:46 2024-03-24 04:54:33 2024-03-24 05:22:00 0:27:27 0:18:21 0:09:06 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
fail 7617855 2024-03-22 21:08:47 2024-03-24 04:54:33 2024-03-24 05:34:10 0:39:37 0:11:08 0:28:29 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-24T05:28:10.713557+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617856 2024-03-22 21:08:48 2024-03-24 10:47:04 2024-03-24 11:36:48 0:49:44 0:36:52 0:12:52 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-24T11:20:00.000125+0000 mon.a (mon.0) 1444 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7617857 2024-03-22 21:08:49 2024-03-24 10:49:05 2024-03-24 11:57:06 1:08:01 0:57:01 0:11:00 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-24T11:06:44.476659+0000 mon.a (mon.0) 167 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7617858 2024-03-22 21:08:50 2024-03-24 10:49:05 2024-03-24 11:08:22 0:19:17 0:09:52 0:09:25 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_adoption} 1
fail 7617859 2024-03-22 21:08:51 2024-03-24 10:49:05 2024-03-24 11:14:13 0:25:08 0:13:33 0:11:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-24T11:07:27.608834+0000 mon.smithi145 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617860 2024-03-22 21:08:52 2024-03-24 10:50:56 2024-03-24 11:12:18 0:21:22 0:11:46 0:09:36 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7617861 2024-03-22 21:08:53 2024-03-24 10:50:56 2024-03-24 11:21:55 0:30:59 0:20:08 0:10:51 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-24T11:12:31.068639+0000 mon.smithi164 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617862 2024-03-22 21:08:54 2024-03-24 10:52:07 2024-03-24 11:25:22 0:33:15 0:23:27 0:09:48 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
dead 7617863 2024-03-22 21:08:55 2024-03-24 10:52:07 2024-03-24 23:00:34 12:08:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

hit max job timeout

fail 7617864 2024-03-22 21:08:55 2024-03-24 10:52:18 2024-03-24 11:15:27 0:23:09 0:13:42 0:09:27 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-24T11:08:31.726922+0000 mon.smithi132 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617865 2024-03-22 21:08:56 2024-03-24 10:52:18 2024-03-24 11:22:19 0:30:01 0:19:43 0:10:18 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
fail 7617866 2024-03-22 21:08:57 2024-03-24 10:52:48 2024-03-24 11:10:18 0:17:30 0:07:30 0:10:00 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} 1
Failure Reason:

Command failed on smithi059 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7617867 2024-03-22 21:08:58 2024-03-24 10:52:49 2024-03-24 11:18:22 0:25:33 0:14:02 0:11:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-24T11:10:20.940766+0000 mon.smithi190 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617868 2024-03-22 21:08:59 2024-03-24 10:53:19 2024-03-24 12:09:21 1:16:02 1:04:32 0:11:30 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-24T11:30:00.388832+0000 mon.a (mon.0) 1343 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7617869 2024-03-22 21:09:01 2024-03-24 10:53:19 2024-03-24 12:02:50 1:09:31 0:56:49 0:12:42 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi100 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e2f9ef6c-e9ce-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''

pass 7617870 2024-03-22 21:09:02 2024-03-24 10:53:40 2024-03-24 11:08:48 0:15:08 0:06:08 0:09:00 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7617871 2024-03-22 21:09:03 2024-03-24 10:53:40 2024-03-24 11:25:35 0:31:55 0:21:41 0:10:14 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-24T11:14:53.532510+0000 mon.smithi110 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617872 2024-03-22 21:09:04 2024-03-24 10:53:40 2024-03-24 11:18:50 0:25:10 0:15:11 0:09:59 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7617873 2024-03-22 21:09:05 2024-03-24 10:53:41 2024-03-24 11:19:04 0:25:23 0:15:51 0:09:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-24T11:10:57.223642+0000 mon.smithi174 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617874 2024-03-22 21:09:06 2024-03-24 10:54:11 2024-03-24 11:17:39 0:23:28 0:12:03 0:11:25 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-24T11:11:00.740864+0000 mon.a (mon.0) 243 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617875 2024-03-22 21:09:07 2024-03-24 10:54:12 2024-03-24 11:23:53 0:29:41 0:20:03 0:09:38 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-24T11:14:52.345004+0000 mon.smithi134 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617876 2024-03-22 21:09:08 2024-03-24 10:54:22 2024-03-24 11:17:27 0:23:05 0:14:08 0:08:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f73f1dfd96849300c4322bd7259a3ab37f94c200 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm_timeout.py'

fail 7617877 2024-03-22 21:09:09 2024-03-24 10:54:22 2024-03-24 11:38:58 0:44:36 0:32:21 0:12:15 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-03-24T11:12:11.895983+0000 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7617878 2024-03-22 21:09:10 2024-03-24 10:56:03 2024-03-24 11:20:57 0:24:54 0:15:51 0:09:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-24T11:12:47.371974+0000 mon.smithi016 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617879 2024-03-22 21:09:11 2024-03-24 10:56:03 2024-03-24 11:52:14 0:56:11 0:45:23 0:10:48 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-24T11:18:41.877547+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7617880 2024-03-22 21:09:12 2024-03-24 10:56:14 2024-03-24 11:30:32 0:34:18 0:23:11 0:11:07 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-24T11:17:31.171561+0000 mon.smithi163 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617881 2024-03-22 21:09:13 2024-03-24 10:56:14 2024-03-24 11:24:44 0:28:30 0:19:01 0:09:29 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_extra_daemon_features} 2
fail 7617882 2024-03-22 21:09:14 2024-03-24 10:56:55 2024-03-24 12:02:01 1:05:06 0:55:30 0:09:36 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-24T11:30:00.000137+0000 mon.a (mon.0) 1187 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7617883 2024-03-22 21:09:15 2024-03-24 10:56:55 2024-03-24 11:44:42 0:47:47 0:33:52 0:13:55 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711279518.5996983 mon.smithi019 (mon.0) 686 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7617884 2024-03-22 21:09:16 2024-03-24 10:58:36 2024-03-24 11:23:08 0:24:32 0:16:24 0:08:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-24T11:14:49.370402+0000 mon.smithi028 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617885 2024-03-22 21:09:17 2024-03-24 10:58:36 2024-03-24 11:22:39 0:24:03 0:13:42 0:10:21 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-24T11:16:41.170404+0000 mon.smithi023 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617886 2024-03-22 21:09:18 2024-03-24 10:58:36 2024-03-24 11:25:38 0:27:02 0:16:01 0:11:01 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
fail 7617887 2024-03-22 21:09:19 2024-03-24 10:58:37 2024-03-24 11:23:58 0:25:21 0:15:47 0:09:34 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-24T11:16:09.725501+0000 mon.smithi029 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617888 2024-03-22 21:09:20 2024-03-24 10:58:37 2024-03-24 11:29:54 0:31:17 0:21:11 0:10:06 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-24T11:18:40.436345+0000 mon.smithi047 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617889 2024-03-22 21:09:21 2024-03-24 10:58:37 2024-03-24 11:21:44 0:23:07 0:13:10 0:09:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7617890 2024-03-22 21:09:22 2024-03-24 10:58:48 2024-03-24 11:34:40 0:35:52 0:25:52 0:10:00 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7617891 2024-03-22 21:09:23 2024-03-24 10:58:48 2024-03-24 11:35:17 0:36:29 0:23:52 0:12:37 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-24T11:30:00.000237+0000 mon.a (mon.0) 1157 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log

dead 7617892 2024-03-22 21:09:24 2024-03-24 10:58:49 2024-03-24 11:03:48 0:04:59 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
Failure Reason:

Error reimaging machines: Expected smithi115's OS to be ubuntu 22.04 but found centos 9

fail 7617893 2024-03-22 21:09:25 2024-03-24 10:58:49 2024-03-24 11:05:52 0:07:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

[Errno None] Unable to connect to port 22 on 172.21.15.115

pass 7617894 2024-03-22 21:09:26 2024-03-24 10:58:49 2024-03-24 11:26:20 0:27:31 0:14:14 0:13:17 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
fail 7617895 2024-03-22 21:09:27 2024-03-24 11:01:20 2024-03-24 11:24:39 0:23:19 0:12:32 0:10:47 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-24T11:19:10.634390+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617896 2024-03-22 21:09:28 2024-03-24 11:01:20 2024-03-24 11:28:59 0:27:39 0:14:31 0:13:08 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-24T11:20:38.375202+0000 mon.smithi064 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617897 2024-03-22 21:09:29 2024-03-24 11:03:51 2024-03-24 11:48:47 0:44:56 0:31:41 0:13:15 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-24T11:40:00.000104+0000 mon.a (mon.0) 1856 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7617898 2024-03-22 21:09:30 2024-03-24 11:04:02 2024-03-24 11:28:47 0:24:45 0:14:13 0:10:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-24T11:22:04.760756+0000 mon.smithi122 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617899 2024-03-22 21:09:31 2024-03-24 11:05:42 2024-03-24 12:12:49 1:07:07 0:54:24 0:12:43 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi121 with status 110: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a55b8ae2-e9d0-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr'"

fail 7617900 2024-03-22 21:09:32 2024-03-24 11:05:43 2024-03-24 11:42:11 0:36:28 0:25:18 0:11:10 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-03-24T11:30:00.000251+0000 mon.a (mon.0) 445 : cluster 3 [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm" in cluster log

fail 7617901 2024-03-22 21:09:33 2024-03-24 11:05:53 2024-03-24 12:18:07 1:12:14 1:02:47 0:09:27 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-03-24T11:23:32.703434+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7617902 2024-03-22 21:09:34 2024-03-24 11:05:54 2024-03-24 11:43:37 0:37:43 0:25:25 0:12:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-24T11:28:57.549524+0000 mon.smithi203 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617903 2024-03-22 21:09:35 2024-03-24 11:07:34 2024-03-24 11:30:37 0:23:03 0:12:55 0:10:08 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_orch_cli} 1
fail 7617904 2024-03-22 21:09:36 2024-03-24 11:07:34 2024-03-24 11:32:59 0:25:25 0:15:54 0:09:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-24T11:24:54.682502+0000 mon.smithi153 (mon.0) 258 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617905 2024-03-22 21:09:37 2024-03-24 11:07:45 2024-03-24 11:37:39 0:29:54 0:18:56 0:10:58 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-24T11:28:59.046192+0000 mon.smithi159 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617906 2024-03-22 21:09:38 2024-03-24 11:07:45 2024-03-24 11:26:53 0:19:08 0:07:29 0:11:39 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} 3
Failure Reason:

Command failed on smithi102 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7617907 2024-03-22 21:09:39 2024-03-24 11:08:56 2024-03-24 11:35:23 0:26:27 0:13:44 0:12:43 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-24T11:28:12.753591+0000 mon.smithi090 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617908 2024-03-22 21:09:40 2024-03-24 11:09:26 2024-03-24 11:50:22 0:40:56 0:27:40 0:13:16 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-03-24T11:43:58.309927+0000 mon.a (mon.0) 1939 : cluster 3 [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm" in cluster log

pass 7617909 2024-03-22 21:09:41 2024-03-24 11:10:57 2024-03-24 11:45:46 0:34:49 0:23:23 0:11:26 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
fail 7617910 2024-03-22 21:09:42 2024-03-24 11:12:28 2024-03-24 11:45:26 0:32:58 0:21:38 0:11:20 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-24T11:35:54.761442+0000 mon.smithi018 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617911 2024-03-22 21:09:43 2024-03-24 11:13:58 2024-03-24 12:11:37 0:57:39 0:48:17 0:09:22 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-24T11:50:00.000146+0000 mon.a (mon.0) 1128 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7617912 2024-03-22 21:09:44 2024-03-24 11:13:59 2024-03-24 11:58:38 0:44:39 0:35:05 0:09:34 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711280304.410507 mon.smithi050 (mon.0) 717 : cluster [WRN] Replacing daemon mds.cephfs.smithi050.wkauvy as rank 0 with standby daemon mds.cephfs.smithi050.jegtzl" in cluster log

pass 7617913 2024-03-22 21:09:45 2024-03-24 11:13:59 2024-03-24 11:38:43 0:24:44 0:15:04 0:09:40 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
dead 7617914 2024-03-22 21:09:46 2024-03-24 11:13:59 2024-03-24 23:23:07 12:09:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

hit max job timeout

pass 7617915 2024-03-22 21:09:47 2024-03-24 11:14:00 2024-03-24 11:37:46 0:23:46 0:13:47 0:09:59 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7617916 2024-03-22 21:09:47 2024-03-24 11:14:00 2024-03-24 11:43:37 0:29:37 0:20:14 0:09:23 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7617917 2024-03-22 21:09:48 2024-03-24 11:14:01 2024-03-24 11:39:41 0:25:40 0:15:28 0:10:12 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-24T11:32:14.824811+0000 mon.smithi006 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617918 2024-03-22 21:09:49 2024-03-24 11:14:01 2024-03-24 11:40:15 0:26:14 0:14:59 0:11:15 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-24T11:34:31.007605+0000 mon.smithi039 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617919 2024-03-22 21:09:50 2024-03-24 11:14:01 2024-03-24 11:40:59 0:26:58 0:17:14 0:09:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7617920 2024-03-22 21:09:51 2024-03-24 11:14:02 2024-03-24 11:45:31 0:31:29 0:21:28 0:10:01 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-24T11:35:12.841829+0000 mon.smithi005 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617921 2024-03-22 21:09:52 2024-03-24 11:14:02 2024-03-24 11:53:44 0:39:42 0:28:21 0:11:21 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-03-24T11:33:05.824924+0000 mon.a (mon.0) 477 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7617922 2024-03-22 21:09:53 2024-03-24 11:14:12 2024-03-24 11:41:44 0:27:32 0:16:26 0:11:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-24T11:34:35.996183+0000 mon.smithi057 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617923 2024-03-22 21:09:54 2024-03-24 11:14:13 2024-03-24 12:19:08 1:04:55 0:50:55 0:14:00 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-24T11:50:00.000144+0000 mon.a (mon.0) 2750 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7617924 2024-03-22 21:09:55 2024-03-24 11:17:24 2024-03-24 12:20:36 1:03:12 0:52:16 0:10:56 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-24T11:50:00.000145+0000 mon.a (mon.0) 2677 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log

pass 7617925 2024-03-22 21:09:56 2024-03-24 11:17:24 2024-03-24 11:36:17 0:18:53 0:10:05 0:08:48 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_adoption} 1
fail 7617926 2024-03-22 21:09:58 2024-03-24 11:17:24 2024-03-24 15:08:21 3:50:57 3:36:03 0:14:54 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi162 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f73f1dfd96849300c4322bd7259a3ab37f94c200 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7617927 2024-03-22 21:09:59 2024-03-24 11:46:48 905 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-24T11:38:13.407679+0000 mon.smithi081 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617928 2024-03-22 21:10:00 2024-03-24 11:21:36 2024-03-24 11:44:37 0:23:01 0:13:58 0:09:03 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-24T11:38:14.585657+0000 mon.smithi073 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617929 2024-03-22 21:10:01 2024-03-24 11:21:36 2024-03-24 11:44:00 0:22:24 0:11:16 0:11:08 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi070 with status 5: 'sudo systemctl stop ceph-36ace750-e9d3-11ee-95cd-87774f69a715@mon.a'

fail 7617930 2024-03-22 21:10:02 2024-03-24 11:22:27 2024-03-24 11:56:26 0:33:59 0:21:50 0:12:09 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-24T11:44:58.501885+0000 mon.smithi138 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617931 2024-03-22 21:10:03 2024-03-24 11:24:47 2024-03-24 11:50:45 0:25:58 0:14:45 0:11:13 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7617932 2024-03-22 21:10:04 2024-03-24 11:25:28 2024-03-24 12:10:59 0:45:31 0:34:22 0:11:09 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-24T11:44:15.457063+0000 mon.smithi032 (mon.0) 514 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 7617933 2024-03-22 21:10:04 2024-03-24 11:25:49 2024-03-24 12:19:21 0:53:32 0:44:46 0:08:46 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7617934 2024-03-22 21:10:05 2024-03-24 11:25:49 2024-03-24 11:52:23 0:26:34 0:13:43 0:12:51 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
fail 7617935 2024-03-22 21:10:06 2024-03-24 11:28:40 2024-03-24 12:05:51 0:37:11 0:25:28 0:11:43 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-03-24T11:47:57.209301+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617936 2024-03-22 21:10:07 2024-03-24 11:29:21 2024-03-24 11:56:26 0:27:05 0:17:10 0:09:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-24T11:46:42.182276+0000 mon.smithi060 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617937 2024-03-22 21:10:08 2024-03-24 11:29:21 2024-03-24 11:52:16 0:22:55 0:11:57 0:10:58 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
fail 7617938 2024-03-22 21:10:09 2024-03-24 11:29:21 2024-03-24 11:53:42 0:24:21 0:11:34 0:12:47 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-24T11:47:46.653387+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7617939 2024-03-22 21:10:10 2024-03-24 11:29:32 2024-03-24 11:54:17 0:24:45 0:14:31 0:10:14 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm} 1
fail 7617940 2024-03-22 21:10:11 2024-03-24 11:29:32 2024-03-24 11:56:23 0:26:51 0:16:36 0:10:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-24T11:45:46.486664+0000 mon.smithi069 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617941 2024-03-22 21:10:12 2024-03-24 11:29:32 2024-03-24 11:58:58 0:29:26 0:19:24 0:10:02 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-24T11:50:32.959038+0000 mon.smithi028 (mon.0) 250 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617942 2024-03-22 21:10:13 2024-03-24 11:29:33 2024-03-24 12:35:12 1:05:39 0:54:16 0:11:23 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-24T12:00:00.000113+0000 mon.a (mon.0) 1176 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7617943 2024-03-22 21:10:14 2024-03-24 11:29:33 2024-03-24 12:03:07 0:33:34 0:21:17 0:12:17 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-24T11:52:19.287411+0000 mon.smithi023 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617944 2024-03-22 21:10:15 2024-03-24 11:29:33 2024-03-24 12:11:53 0:42:20 0:32:34 0:09:46 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
pass 7617945 2024-03-22 21:10:16 2024-03-24 11:29:34 2024-03-24 12:14:04 0:44:30 0:33:06 0:11:24 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7617946 2024-03-22 21:10:17 2024-03-24 11:29:34 2024-03-24 11:47:50 0:18:16 0:07:15 0:11:01 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/1.7.2} 1
Failure Reason:

Command failed on smithi184 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7617947 2024-03-22 21:10:18 2024-03-24 11:29:35 2024-03-24 11:44:55 0:15:20 0:06:12 0:09:08 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm_repos} 1
fail 7617948 2024-03-22 21:10:19 2024-03-24 11:29:35 2024-03-24 11:55:36 0:26:01 0:14:06 0:11:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-24T11:49:17.596865+0000 mon.smithi176 (mon.0) 258 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617949 2024-03-22 21:10:20 2024-03-24 11:29:35 2024-03-24 12:20:46 0:51:11 0:39:08 0:12:03 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-24T11:48:18.989137+0000 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7617950 2024-03-22 21:10:21 2024-03-24 11:29:36 2024-03-24 12:00:03 0:30:27 0:19:08 0:11:19 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_cephadm_timeout} 1
fail 7617951 2024-03-22 21:10:22 2024-03-24 11:29:36 2024-03-24 11:55:53 0:26:17 0:15:21 0:10:56 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-24T11:48:31.159562+0000 mon.smithi029 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617952 2024-03-22 21:10:23 2024-03-24 11:29:36 2024-03-24 11:55:19 0:25:43 0:14:14 0:11:29 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-24T11:47:17.683833+0000 mon.smithi052 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617953 2024-03-22 21:10:24 2024-03-24 11:29:37 2024-03-24 12:00:14 0:30:37 0:20:49 0:09:48 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-24T11:51:18.839261+0000 mon.smithi102 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617954 2024-03-22 21:10:25 2024-03-24 11:29:37 2024-03-24 12:04:58 0:35:21 0:25:24 0:09:57 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7617955 2024-03-22 21:10:26 2024-03-24 11:29:38 2024-03-24 11:53:21 0:23:43 0:14:00 0:09:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-03-24T11:51:26.647664+0000 mon.a (mon.0) 8 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log

fail 7617956 2024-03-22 21:10:27 2024-03-24 11:29:38 2024-03-24 12:24:41 0:55:03 0:44:15 0:10:48 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-24T12:10:00.000184+0000 mon.a (mon.0) 1341 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7617957 2024-03-22 21:10:28 2024-03-24 11:29:38 2024-03-24 11:57:19 0:27:41 0:17:53 0:09:48 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-24T11:47:01.772837+0000 mon.smithi080 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617958 2024-03-22 21:10:29 2024-03-24 11:29:39 2024-03-24 11:56:14 0:26:35 0:11:46 0:14:49 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-24T11:51:05.516987+0000 mon.a (mon.0) 248 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

dead 7617959 2024-03-22 21:10:30 2024-03-24 11:34:50 2024-03-24 23:47:13 12:12:23 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

pass 7617960 2024-03-22 21:10:31 2024-03-24 11:36:21 2024-03-24 12:10:45 0:34:24 0:23:13 0:11:11 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
fail 7617961 2024-03-22 21:10:32 2024-03-24 11:37:51 2024-03-24 12:03:54 0:26:03 0:15:01 0:11:02 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-24T11:55:28.296842+0000 mon.smithi043 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617962 2024-03-22 21:10:33 2024-03-24 11:38:52 2024-03-24 12:06:09 0:27:17 0:15:34 0:11:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_host_drain} 3
fail 7617963 2024-03-22 21:10:34 2024-03-24 11:41:02 2024-03-24 12:07:13 0:26:11 0:13:22 0:12:49 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-24T12:01:01.533638+0000 mon.smithi042 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617964 2024-03-22 21:10:35 2024-03-24 11:43:43 2024-03-24 12:13:28 0:29:45 0:19:49 0:09:56 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-24T12:05:12.836230+0000 mon.smithi012 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617965 2024-03-22 21:10:36 2024-03-24 11:43:44 2024-03-24 12:07:45 0:24:01 0:14:03 0:09:58 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7617966 2024-03-22 21:10:36 2024-03-24 11:45:04 2024-03-24 12:07:52 0:22:48 0:13:18 0:09:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-24T12:01:24.494658+0000 mon.smithi062 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617967 2024-03-22 21:10:37 2024-03-24 11:45:04 2024-03-24 12:09:00 0:23:56 0:14:06 0:09:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-24T12:02:37.271975+0000 mon.smithi018 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617968 2024-03-22 21:10:38 2024-03-24 11:45:05 2024-03-24 12:30:07 0:45:02 0:35:44 0:09:18 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
fail 7617969 2024-03-22 21:10:39 2024-03-24 11:45:05 2024-03-24 12:39:31 0:54:26 0:43:32 0:10:54 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi019 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fd883814-e9d5-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7617970 2024-03-22 21:10:40 2024-03-24 11:45:06 2024-03-24 12:15:02 0:29:56 0:19:12 0:10:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7617971 2024-03-22 21:10:41 2024-03-24 11:45:06 2024-03-24 12:29:13 0:44:07 0:31:43 0:12:24 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711282222.5988038 mon.smithi163 (mon.0) 681 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7617972 2024-03-22 21:10:42 2024-03-24 11:45:07 2024-03-24 12:12:00 0:26:53 0:15:33 0:11:20 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7617973 2024-03-22 21:10:43 2024-03-24 11:45:07 2024-03-24 12:17:31 0:32:24 0:21:24 0:11:00 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-24T12:05:53.227118+0000 mon.smithi089 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617974 2024-03-22 21:10:44 2024-03-24 11:45:07 2024-03-24 12:17:49 0:32:42 0:20:21 0:12:21 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-24T12:07:32.585527+0000 mon.smithi118 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617975 2024-03-22 21:10:45 2024-03-24 11:45:08 2024-03-24 12:49:19 1:04:11 0:52:18 0:11:53 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-24T12:20:00.000168+0000 mon.a (mon.0) 2567 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled" in cluster log

pass 7617976 2024-03-22 21:10:46 2024-03-24 11:45:08 2024-03-24 12:08:10 0:23:02 0:11:38 0:11:24 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
fail 7617977 2024-03-22 21:10:47 2024-03-24 11:45:08 2024-03-24 12:10:37 0:25:29 0:15:30 0:09:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-24T12:03:31.497095+0000 mon.smithi194 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617978 2024-03-22 21:10:48 2024-03-24 11:45:09 2024-03-24 12:09:22 0:24:13 0:12:50 0:11:23 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7617979 2024-03-22 21:10:49 2024-03-24 11:45:09 2024-03-24 12:08:56 0:23:47 0:13:10 0:10:37 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-24T12:03:25.337307+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617980 2024-03-22 21:10:50 2024-03-24 11:45:10 2024-03-24 12:08:40 0:23:30 0:13:50 0:09:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

Test failure: test_cephfs_mirror (tasks.cephadm_cases.test_cli.TestCephadmCLI)

fail 7617981 2024-03-22 21:10:51 2024-03-24 11:45:10 2024-03-24 12:11:15 0:26:05 0:15:47 0:10:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-24T12:03:41.923197+0000 mon.smithi131 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617982 2024-03-22 21:10:52 2024-03-24 11:45:10 2024-03-24 12:35:11 0:50:01 0:36:56 0:13:05 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_orch_cli_mon} 5
fail 7617983 2024-03-22 21:10:53 2024-03-24 11:45:51 2024-03-24 12:22:49 0:36:58 0:22:00 0:14:58 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-24T12:12:12.058443+0000 mon.smithi170 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617984 2024-03-22 21:10:54 2024-03-24 11:50:52 2024-03-24 12:15:41 0:24:49 0:13:53 0:10:56 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-24T12:09:10.353743+0000 mon.smithi104 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617985 2024-03-22 21:10:55 2024-03-24 11:52:33 2024-03-24 12:55:53 1:03:20 0:51:50 0:11:30 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-24T12:20:00.000155+0000 mon.a (mon.0) 891 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7617986 2024-03-22 21:10:56 2024-03-24 11:54:23 2024-03-24 12:19:04 0:24:41 0:07:35 0:17:06 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/master} 3
Failure Reason:

Command failed on smithi053 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7617987 2024-03-22 21:10:57 2024-03-24 12:00:34 2024-03-24 12:26:21 0:25:47 0:15:41 0:10:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-24T12:18:20.660677+0000 mon.smithi130 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617988 2024-03-22 21:10:58 2024-03-24 12:00:35 2024-03-24 15:49:48 3:49:13 3:36:18 0:12:55 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi174 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f73f1dfd96849300c4322bd7259a3ab37f94c200 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7617989 2024-03-22 21:10:59 2024-03-24 12:00:35 2024-03-24 12:29:08 0:28:33 0:16:27 0:12:06 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_rgw_multisite} 3
fail 7617990 2024-03-22 21:11:00 2024-03-24 12:00:36 2024-03-24 12:29:08 0:28:32 0:16:30 0:12:02 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-24T12:18:46.148806+0000 mon.smithi083 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617991 2024-03-22 21:11:01 2024-03-24 12:00:36 2024-03-24 12:45:06 0:44:30 0:32:33 0:11:57 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-03-24T12:23:18.157258+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7617992 2024-03-22 21:11:02 2024-03-24 12:00:36 2024-03-24 12:25:32 0:24:56 0:15:25 0:09:31 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7617993 2024-03-22 21:11:03 2024-03-24 12:00:37 2024-03-24 12:34:11 0:33:34 0:24:01 0:09:33 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-24T12:21:38.567232+0000 mon.smithi028 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617994 2024-03-22 21:11:03 2024-03-24 12:00:37 2024-03-24 12:31:13 0:30:36 0:16:51 0:13:45 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7617995 2024-03-22 21:11:04 2024-03-24 12:00:37 2024-03-24 12:54:08 0:53:31 0:41:23 0:12:08 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-03-24T12:19:39.626802+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7617996 2024-03-22 21:11:05 2024-03-24 12:00:38 2024-03-24 12:25:04 0:24:26 0:13:11 0:11:15 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-24T12:19:45.337692+0000 mon.smithi067 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617997 2024-03-22 21:11:06 2024-03-24 12:00:38 2024-03-24 12:25:21 0:24:43 0:14:56 0:09:47 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-24T12:17:41.575498+0000 mon.smithi060 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7617998 2024-03-22 21:11:07 2024-03-24 12:00:39 2024-03-24 12:22:19 0:21:40 0:11:34 0:10:06 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-24T12:17:59.192185+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7617999 2024-03-22 21:11:08 2024-03-24 12:00:39 2024-03-24 13:20:52 1:20:13 1:09:24 0:10:49 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-24T12:30:00.000318+0000 mon.a (mon.0) 922 : cluster 3 [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull" in cluster log

pass 7618000 2024-03-22 21:11:09 2024-03-24 12:00:39 2024-03-24 12:22:47 0:22:08 0:12:51 0:09:17 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_adoption} 1
fail 7618001 2024-03-22 21:11:10 2024-03-24 12:00:40 2024-03-24 12:25:20 0:24:40 0:14:01 0:10:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-24T12:18:58.719668+0000 mon.smithi138 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7618002 2024-03-22 21:11:11 2024-03-24 12:00:40 2024-03-24 12:44:19 0:43:39 0:32:06 0:11:33 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711283098.1223984 mon.smithi047 (mon.0) 689 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7618003 2024-03-22 21:11:12 2024-03-24 12:00:41 2024-03-24 12:31:08 0:30:27 0:20:06 0:10:21 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-24T12:22:39.796493+0000 mon.smithi046 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7618004 2024-03-22 21:11:13 2024-03-24 12:00:41 2024-03-24 12:26:58 0:26:17 0:13:56 0:12:21 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7618005 2024-03-22 21:11:14 2024-03-24 12:00:41 2024-03-24 12:33:07 0:32:26 0:21:20 0:11:06 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-24T12:22:43.729121+0000 mon.smithi107 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7618006 2024-03-22 21:11:15 2024-03-24 12:00:42 2024-03-24 12:27:10 0:26:28 0:14:23 0:12:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-24T12:19:25.994510+0000 mon.smithi080 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7618007 2024-03-22 21:11:15 2024-03-24 12:00:42 2024-03-24 12:39:56 0:39:14 0:23:48 0:15:26 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-24T12:23:48.677630+0000 mon.a (mon.0) 174 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7618008 2024-03-22 21:11:16 2024-03-24 12:05:03 2024-03-24 12:29:15 0:24:12 0:14:20 0:09:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm} 1
fail 7618009 2024-03-22 21:11:17 2024-03-24 12:06:14 2024-03-24 12:35:17 0:29:03 0:18:02 0:11:01 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-24T12:24:35.605806+0000 mon.smithi082 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7618010 2024-03-22 21:11:18 2024-03-24 12:08:14 2024-03-24 12:44:26 0:36:12 0:24:48 0:11:24 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2