Pulpito

teuthology-2024-04-08_21:08:03-orch-squid-distro-default-smithi

User Scheduled Started Updated Runtime Suite Branch Machine Type Revision Pass Fail Dead
teuthology 2024-04-08 21:08:03 2024-04-10 07:30:58 2024-04-10 09:18:34 1:47:36 orch squid smithi b6f3e46 117 52 3

Average wait time: 0:11:06

See other runs of suite 'orch' on branch 'squid'?

See other runs on branch 'squid'?

See other runs scheduled on 2024-04-08?

See detail view?

Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7646166 2024-04-08 21:08:40 2024-04-08 21:14:16 2024-04-08 21:43:55 0:29:39 0:18:28 0:11:11 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7646167 2024-04-08 21:08:41 2024-04-08 21:14:16 2024-04-08 21:39:26 0:25:10 0:16:03 0:09:07 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_host_drain} 3
pass 7646168 2024-04-08 21:08:42 2024-04-08 21:14:17 2024-04-08 21:46:42 0:32:25 0:22:08 0:10:17 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7646169 2024-04-08 21:08:43 2024-04-08 21:14:17 2024-04-08 22:01:11 0:46:54 0:38:36 0:08:18 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646170 2024-04-08 21:08:44 2024-04-08 21:14:18 2024-04-08 21:59:19 0:45:01 0:34:55 0:10:06 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-08T21:39:47.645798+0000 mon.smithi102 (mon.0) 870 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7646171 2024-04-08 21:08:45 2024-04-08 21:14:18 2024-04-08 22:07:42 0:53:24 0:45:00 0:08:24 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7646172 2024-04-08 21:08:46 2024-04-08 21:14:18 2024-04-08 21:35:27 0:21:09 0:10:34 0:10:35 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_adoption} 1
pass 7646173 2024-04-08 21:08:47 2024-04-08 21:14:19 2024-04-08 21:40:58 0:26:39 0:14:13 0:12:26 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7646174 2024-04-08 21:08:48 2024-04-08 22:16:54 2024-04-08 22:40:11 0:23:17 0:13:31 0:09:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7646175 2024-04-08 21:08:49 2024-04-08 22:17:14 2024-04-08 22:53:44 0:36:30 0:25:38 0:10:52 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-08T22:35:09.552246+0000 mon.a (mon.0) 208 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646176 2024-04-08 21:08:50 2024-04-08 22:17:15 2024-04-08 22:42:19 0:25:04 0:15:29 0:09:35 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7646177 2024-04-08 21:08:51 2024-04-08 22:17:15 2024-04-08 22:39:38 0:22:23 0:13:19 0:09:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7646178 2024-04-08 21:08:52 2024-04-08 22:17:15 2024-04-08 22:37:53 0:20:38 0:11:52 0:08:46 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
fail 7646179 2024-04-08 21:08:53 2024-04-08 22:17:16 2024-04-08 22:40:37 0:23:21 0:12:23 0:10:58 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-08T22:35:04.073844+0000 mon.a (mon.0) 208 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7646180 2024-04-08 21:08:54 2024-04-08 22:17:16 2024-04-08 23:30:19 1:13:03 1:03:22 0:09:41 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-08T22:36:14.134517+0000 mon.a (mon.0) 198 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7646181 2024-04-08 21:08:55 2024-04-08 22:17:16 2024-04-08 23:22:33 1:05:17 0:56:31 0:08:46 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-08T22:51:07.971223+0000 mon.a (mon.0) 1022 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

dead 7646182 2024-04-08 21:08:56 2024-04-08 22:17:17 2024-04-09 10:25:44 12:08:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

hit max job timeout

pass 7646183 2024-04-08 21:08:57 2024-04-08 22:17:17 2024-04-08 22:41:08 0:23:51 0:14:31 0:09:20 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7646184 2024-04-08 21:08:58 2024-04-08 22:17:17 2024-04-08 22:49:31 0:32:14 0:21:19 0:10:55 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7646185 2024-04-08 21:08:59 2024-04-08 22:17:18 2024-04-08 22:54:47 0:37:29 0:27:18 0:10:11 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7646186 2024-04-08 21:09:00 2024-04-08 22:17:18 2024-04-08 22:42:41 0:25:23 0:14:56 0:10:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7646187 2024-04-08 21:09:01 2024-04-08 22:17:18 2024-04-08 22:55:47 0:38:29 0:27:52 0:10:37 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_monitoring_stack_basic} 3
pass 7646188 2024-04-08 21:09:02 2024-04-08 22:17:19 2024-04-08 22:41:13 0:23:54 0:14:09 0:09:45 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
fail 7646189 2024-04-08 21:09:03 2024-04-08 22:17:19 2024-04-08 22:36:16 0:18:57 0:07:53 0:11:04 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/master} 3
Failure Reason:

Command failed on smithi023 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

dead 7646190 2024-04-08 21:09:04 2024-04-08 22:17:19 2024-04-09 10:26:30 12:09:11 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

pass 7646191 2024-04-08 21:09:05 2024-04-08 22:17:20 2024-04-08 22:42:58 0:25:38 0:14:39 0:10:59 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_cephadm_timeout} 1
pass 7646192 2024-04-08 21:09:06 2024-04-08 22:17:20 2024-04-08 22:44:32 0:27:12 0:15:05 0:12:07 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7646193 2024-04-08 21:09:07 2024-04-08 22:17:20 2024-04-08 23:03:19 0:45:59 0:35:07 0:10:52 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7646194 2024-04-08 21:09:08 2024-04-08 22:17:21 2024-04-08 22:50:20 0:32:59 0:22:16 0:10:43 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7646195 2024-04-08 21:09:09 2024-04-08 22:17:21 2024-04-08 22:44:27 0:27:06 0:15:54 0:11:12 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_rgw_multisite} 3
pass 7646196 2024-04-08 21:09:10 2024-04-08 22:17:22 2024-04-08 22:46:25 0:29:03 0:16:29 0:12:34 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7646197 2024-04-08 21:09:11 2024-04-08 22:17:22 2024-04-08 22:49:08 0:31:46 0:18:06 0:13:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7646198 2024-04-08 21:09:12 2024-04-08 22:17:32 2024-04-08 22:47:53 0:30:21 0:16:38 0:13:43 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-08T22:41:44.542237+0000 mon.a (mon.0) 268 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646199 2024-04-08 21:09:13 2024-04-08 22:17:33 2024-04-08 22:50:28 0:32:55 0:20:20 0:12:35 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7646200 2024-04-08 21:09:14 2024-04-08 22:17:33 2024-04-08 23:02:33 0:45:00 0:32:37 0:12:23 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-04-08T22:36:18.030469+0000 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7646201 2024-04-08 21:09:15 2024-04-08 22:17:33 2024-04-08 22:49:29 0:31:56 0:18:42 0:13:14 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
fail 7646202 2024-04-08 21:09:16 2024-04-08 22:17:34 2024-04-08 23:18:32 1:00:58 0:47:57 0:13:01 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-08T22:44:05.998219+0000 mon.a (mon.0) 201 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7646203 2024-04-08 21:09:17 2024-04-08 22:17:34 2024-04-08 22:49:16 0:31:42 0:18:42 0:13:00 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_set_mon_crush_locations} 3
fail 7646204 2024-04-08 21:09:18 2024-04-08 22:17:35 2024-04-08 23:13:07 0:55:32 0:42:57 0:12:35 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7646205 2024-04-08 21:09:19 2024-04-08 22:17:35 2024-04-08 22:49:54 0:32:19 0:19:55 0:12:24 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_orch_cli} 1
pass 7646206 2024-04-08 21:09:21 2024-04-08 22:17:35 2024-04-08 22:54:03 0:36:28 0:24:23 0:12:05 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
fail 7646207 2024-04-08 21:09:22 2024-04-08 22:17:36 2024-04-08 23:22:33 1:04:57 0:50:40 0:14:17 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-08T22:43:00.083325+0000 mon.a (mon.0) 198 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7646208 2024-04-08 21:09:23 2024-04-08 22:17:36 2024-04-08 22:49:25 0:31:49 0:18:50 0:12:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7646209 2024-04-08 21:09:24 2024-04-08 22:17:36 2024-04-08 22:44:47 0:27:11 0:15:05 0:12:06 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_ca_signed_key} 2
pass 7646210 2024-04-08 21:09:25 2024-04-08 22:17:37 2024-04-08 22:50:12 0:32:35 0:16:44 0:15:51 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7646211 2024-04-08 21:09:26 2024-04-08 22:17:37 2024-04-08 22:47:32 0:29:55 0:17:22 0:12:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7646212 2024-04-08 21:09:27 2024-04-08 22:17:37 2024-04-08 22:53:32 0:35:55 0:23:03 0:12:52 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
fail 7646213 2024-04-08 21:09:28 2024-04-08 22:17:38 2024-04-08 22:50:08 0:32:30 0:16:49 0:15:41 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi018 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

pass 7646214 2024-04-08 21:09:29 2024-04-08 22:22:39 2024-04-08 22:58:28 0:35:49 0:25:01 0:10:48 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7646215 2024-04-08 21:09:30 2024-04-08 22:22:39 2024-04-08 23:05:24 0:42:45 0:22:29 0:20:16 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-04-08T23:00:00.000126+0000 mon.a (mon.0) 1219 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

fail 7646216 2024-04-08 21:09:31 2024-04-08 22:31:41 2024-04-09 02:23:51 3:52:10 3:36:44 0:15:26 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi120 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7646217 2024-04-08 21:09:32 2024-04-08 22:34:21 2024-04-08 23:11:34 0:37:13 0:24:18 0:12:55 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli_mon} 5
pass 7646218 2024-04-08 21:09:33 2024-04-08 22:38:22 2024-04-08 23:03:02 0:24:40 0:14:28 0:10:12 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7646219 2024-04-08 21:09:34 2024-04-08 22:39:43 2024-04-08 23:03:16 0:23:33 0:14:03 0:09:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7646220 2024-04-08 21:09:35 2024-04-08 22:40:14 2024-04-08 23:04:46 0:24:32 0:13:07 0:11:25 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7646221 2024-04-08 21:09:36 2024-04-08 22:41:14 2024-04-08 23:05:36 0:24:22 0:12:05 0:12:17 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-08T22:59:05.001191+0000 mon.a (mon.0) 267 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646222 2024-04-08 21:09:37 2024-04-08 22:42:25 2024-04-08 23:08:22 0:25:57 0:14:52 0:11:05 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
fail 7646223 2024-04-08 21:09:38 2024-04-08 22:42:45 2024-04-08 23:58:44 1:15:59 1:04:48 0:11:11 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi050 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

pass 7646224 2024-04-08 21:09:39 2024-04-08 22:43:06 2024-04-08 23:10:21 0:27:15 0:14:25 0:12:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7646225 2024-04-08 21:09:40 2024-04-08 22:44:36 2024-04-08 23:01:20 0:16:44 0:06:38 0:10:06 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_cephadm_repos} 1
fail 7646226 2024-04-08 21:09:41 2024-04-08 22:44:36 2024-04-08 23:47:13 1:02:37 0:50:28 0:12:09 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi118 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa01dac4-f5fb-11ee-b649-cb9ed24678a4 -e sha1=b6f3e469c8285b60ad2826d6f524be2e6014ed05 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7646227 2024-04-08 21:09:42 2024-04-08 22:44:37 2024-04-08 23:20:26 0:35:49 0:25:30 0:10:19 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7646228 2024-04-08 21:09:43 2024-04-08 22:44:57 2024-04-08 23:11:47 0:26:50 0:15:42 0:11:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7646229 2024-04-08 21:09:44 2024-04-08 22:46:18 2024-04-08 23:09:04 0:22:46 0:13:54 0:08:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_extra_daemon_features} 2
fail 7646230 2024-04-08 21:09:45 2024-04-08 22:46:18 2024-04-09 00:08:21 1:22:03 1:09:43 0:12:20 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646231 2024-04-08 21:09:46 2024-04-08 22:46:19 2024-04-08 23:06:00 0:19:41 0:07:35 0:12:06 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/calico rook/1.7.2} 3
Failure Reason:

Command failed on smithi090 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646232 2024-04-08 21:09:47 2024-04-08 22:47:39 2024-04-08 23:08:47 0:21:08 0:10:31 0:10:37 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_adoption} 1
pass 7646233 2024-04-08 21:09:48 2024-04-08 22:47:39 2024-04-08 23:16:54 0:29:15 0:18:37 0:10:38 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7646234 2024-04-08 21:09:49 2024-04-08 22:49:10 2024-04-08 23:12:33 0:23:23 0:13:48 0:09:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7646235 2024-04-08 21:09:50 2024-04-08 22:49:20 2024-04-08 23:22:54 0:33:34 0:23:56 0:09:38 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7646236 2024-04-08 21:09:51 2024-04-08 22:49:31 2024-04-08 23:19:35 0:30:04 0:20:12 0:09:52 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7646237 2024-04-08 21:09:52 2024-04-08 22:49:31 2024-04-09 00:00:44 1:11:13 1:01:06 0:10:07 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-08T23:20:00.000151+0000 mon.a (mon.0) 1193 : cluster 3 [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull" in cluster log

fail 7646238 2024-04-08 21:09:53 2024-04-08 22:49:31 2024-04-08 23:22:04 0:32:33 0:22:43 0:09:50 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-08T23:20:00.000211+0000 mon.a (mon.0) 645 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

pass 7646239 2024-04-08 21:09:54 2024-04-08 22:50:22 2024-04-08 23:15:40 0:25:18 0:15:17 0:10:01 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7646240 2024-04-08 21:09:55 2024-04-08 22:50:22 2024-04-08 23:15:41 0:25:19 0:14:39 0:10:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
fail 7646241 2024-04-08 21:09:56 2024-04-08 22:50:33 2024-04-08 23:13:27 0:22:54 0:11:50 0:11:04 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-08T23:07:21.876894+0000 mon.a (mon.0) 209 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646242 2024-04-08 21:09:57 2024-04-08 22:50:33 2024-04-08 23:18:43 0:28:10 0:14:55 0:13:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7646243 2024-04-08 21:09:58 2024-04-08 22:53:34 2024-04-08 23:16:52 0:23:18 0:13:54 0:09:24 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7646244 2024-04-08 21:09:59 2024-04-08 22:53:34 2024-04-08 23:20:13 0:26:39 0:14:00 0:12:39 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7646245 2024-04-08 21:10:00 2024-04-08 22:53:35 2024-04-09 02:43:10 3:49:35 3:36:32 0:13:03 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi171 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7646246 2024-04-08 21:10:01 2024-04-08 22:53:35 2024-04-08 23:23:50 0:30:15 0:19:11 0:11:04 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_cephadm_timeout} 1
pass 7646247 2024-04-08 21:10:02 2024-04-08 22:53:55 2024-04-08 23:23:55 0:30:00 0:19:56 0:10:04 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7646248 2024-04-08 21:10:03 2024-04-08 22:54:06 2024-04-08 23:33:21 0:39:15 0:28:34 0:10:41 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-08T23:12:18.586458+0000 mon.a (mon.0) 483 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7646249 2024-04-08 21:10:04 2024-04-08 22:54:26 2024-04-08 23:20:23 0:25:57 0:14:53 0:11:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
fail 7646250 2024-04-08 21:10:05 2024-04-08 22:54:26 2024-04-08 23:36:13 0:41:47 0:30:43 0:11:04 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-08T23:20:00.000125+0000 mon.a (mon.0) 868 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7646251 2024-04-08 21:10:07 2024-04-08 22:54:57 2024-04-08 23:59:06 1:04:09 0:52:23 0:11:46 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-08T23:30:00.000230+0000 mon.a (mon.0) 2695 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log

pass 7646252 2024-04-08 21:10:08 2024-04-08 22:55:57 2024-04-08 23:27:50 0:31:53 0:19:24 0:12:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_monitoring_stack_basic} 3
pass 7646253 2024-04-08 21:10:09 2024-04-08 22:58:28 2024-04-08 23:27:07 0:28:39 0:16:17 0:12:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7646254 2024-04-08 21:10:10 2024-04-08 22:59:59 2024-04-08 23:25:06 0:25:07 0:14:02 0:11:05 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7646255 2024-04-08 21:10:11 2024-04-08 22:59:59 2024-04-08 23:33:26 0:33:27 0:21:33 0:11:54 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7646256 2024-04-08 21:10:12 2024-04-08 23:01:50 2024-04-08 23:26:26 0:24:36 0:15:55 0:08:41 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7646258 2024-04-08 21:10:13 2024-04-08 23:01:50 2024-04-08 23:31:24 0:29:34 0:20:33 0:09:01 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_rgw_multisite} 3
fail 7646261 2024-04-08 21:10:14 2024-04-08 23:01:51 2024-04-09 00:18:50 1:16:59 1:07:50 0:09:09 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646263 2024-04-08 21:10:15 2024-04-08 23:01:51 2024-04-08 23:47:52 0:46:01 0:34:31 0:11:30 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-08T23:21:17.507538+0000 mon.smithi033 (mon.0) 505 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7646265 2024-04-08 21:10:16 2024-04-08 23:03:12 2024-04-08 23:45:34 0:42:22 0:34:55 0:07:27 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7646266 2024-04-08 21:10:17 2024-04-08 23:03:12 2024-04-08 23:26:06 0:22:54 0:13:29 0:09:25 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
pass 7646267 2024-04-08 21:10:18 2024-04-08 23:03:22 2024-04-08 23:24:35 0:21:13 0:12:20 0:08:53 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
fail 7646268 2024-04-08 21:10:19 2024-04-08 23:03:23 2024-04-08 23:43:56 0:40:33 0:25:18 0:15:15 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-08T23:25:39.394430+0000 mon.a (mon.0) 210 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646269 2024-04-08 21:10:20 2024-04-08 23:08:24 2024-04-08 23:36:03 0:27:39 0:17:05 0:10:34 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7646270 2024-04-08 21:10:21 2024-04-08 23:08:54 2024-04-08 23:29:57 0:21:03 0:11:34 0:09:29 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
pass 7646271 2024-04-08 21:10:22 2024-04-08 23:09:05 2024-04-08 23:31:57 0:22:52 0:11:45 0:11:07 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7646272 2024-04-08 21:10:23 2024-04-08 23:10:25 2024-04-08 23:37:29 0:27:04 0:16:38 0:10:26 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7646273 2024-04-08 21:10:24 2024-04-08 23:10:36 2024-04-08 23:36:45 0:26:09 0:15:40 0:10:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7646274 2024-04-08 21:10:25 2024-04-08 23:11:36 2024-04-08 23:41:30 0:29:54 0:19:17 0:10:37 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7646275 2024-04-08 21:10:26 2024-04-08 23:11:36 2024-04-09 00:14:12 1:02:36 0:50:57 0:11:39 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-08T23:50:00.000164+0000 mon.a (mon.0) 1242 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

pass 7646276 2024-04-08 21:10:27 2024-04-08 23:11:57 2024-04-08 23:44:24 0:32:27 0:21:52 0:10:35 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
fail 7646277 2024-04-08 21:10:28 2024-04-08 23:12:37 2024-04-08 23:59:35 0:46:58 0:32:43 0:14:15 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

"2024-04-08T23:37:36.746462+0000 mon.a (mon.0) 200 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7646278 2024-04-08 21:10:29 2024-04-10 07:30:57 2024-04-10 07:48:26 0:17:29 0:07:17 0:10:12 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/1-node k8s/1.21 net/flannel rook/master} 1
Failure Reason:

Command failed on smithi096 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646279 2024-04-08 21:10:30 2024-04-10 07:30:58 2024-04-10 07:54:06 0:23:08 0:13:48 0:09:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
fail 7646280 2024-04-08 21:10:31 2024-04-10 07:30:58 2024-04-10 08:10:33 0:39:35 0:28:53 0:10:42 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-10T07:50:38.914922+0000 mon.a (mon.0) 469 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7646281 2024-04-08 21:10:32 2024-04-10 07:30:58 2024-04-10 07:52:20 0:21:22 0:11:33 0:09:49 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7646282 2024-04-08 21:10:33 2024-04-10 07:30:59 2024-04-10 08:36:53 1:05:54 0:52:14 0:13:40 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi121 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a0fdd25c-f70e-11ee-b64a-cb9ed24678a4 -e sha1=b6f3e469c8285b60ad2826d6f524be2e6014ed05 -- bash -c 'ceph orch ps'"

pass 7646283 2024-04-08 21:10:34 2024-04-10 07:31:39 2024-04-10 08:08:31 0:36:52 0:24:36 0:12:16 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
pass 7646284 2024-04-08 21:10:35 2024-04-10 07:33:20 2024-04-10 08:03:13 0:29:53 0:14:26 0:15:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7646285 2024-04-08 21:10:36 2024-04-10 07:38:11 2024-04-10 08:03:20 0:25:09 0:14:37 0:10:32 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7646286 2024-04-08 21:10:37 2024-04-10 07:38:11 2024-04-10 08:09:46 0:31:35 0:20:33 0:11:02 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7646287 2024-04-08 21:10:38 2024-04-10 07:40:02 2024-04-10 08:16:12 0:36:10 0:25:18 0:10:52 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7646288 2024-04-08 21:10:39 2024-04-10 07:40:32 2024-04-10 08:03:41 0:23:09 0:13:26 0:09:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi134 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7646289 2024-04-08 21:10:40 2024-04-10 07:40:33 2024-04-10 08:27:57 0:47:24 0:37:58 0:09:26 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-10T08:10:00.000140+0000 mon.a (mon.0) 2265 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

pass 7646290 2024-04-08 21:10:41 2024-04-10 07:40:53 2024-04-10 08:09:46 0:28:53 0:17:22 0:11:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7646291 2024-04-08 21:10:42 2024-04-10 07:42:14 2024-04-10 08:05:22 0:23:08 0:12:48 0:10:20 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-10T07:59:05.560684+0000 mon.a (mon.0) 267 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646292 2024-04-08 21:10:43 2024-04-10 07:42:14 2024-04-10 08:18:16 0:36:02 0:24:28 0:11:34 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rotate-keys} 2
pass 7646293 2024-04-08 21:10:44 2024-04-10 07:43:25 2024-04-10 08:08:43 0:25:18 0:15:55 0:09:23 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7646294 2024-04-08 21:10:45 2024-04-10 07:43:25 2024-04-10 08:03:46 0:20:21 0:09:04 0:11:17 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi156 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4870d5a890743a871e99c190e2baea5ccf0df349 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

pass 7646295 2024-04-08 21:10:46 2024-04-10 07:44:05 2024-04-10 08:27:02 0:42:57 0:33:50 0:09:07 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7646296 2024-04-08 21:10:48 2024-04-10 07:44:06 2024-04-10 08:08:19 0:24:13 0:12:19 0:11:54 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
pass 7646297 2024-04-08 21:10:49 2024-04-10 07:44:06 2024-04-10 08:08:50 0:24:44 0:13:54 0:10:50 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7646298 2024-04-08 21:10:50 2024-04-10 07:45:57 2024-04-10 08:13:40 0:27:43 0:18:36 0:09:07 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7646299 2024-04-08 21:10:51 2024-04-10 07:45:57 2024-04-10 08:11:47 0:25:50 0:13:52 0:11:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7646300 2024-04-08 21:10:53 2024-04-10 07:48:39 2024-04-10 08:12:58 0:24:19 0:14:43 0:09:36 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
fail 7646301 2024-04-08 21:10:54 2024-04-10 07:49:49 2024-04-10 08:53:55 1:04:06 0:53:25 0:10:41 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-10T08:20:00.000127+0000 mon.a (mon.0) 1067 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7646302 2024-04-08 21:10:55 2024-04-10 07:50:10 2024-04-10 08:50:01 0:59:51 0:49:37 0:10:14 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi079 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30c3d2ae-f711-11ee-b64a-cb9ed24678a4 -e sha1=b6f3e469c8285b60ad2826d6f524be2e6014ed05 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

fail 7646303 2024-04-08 21:10:56 2024-04-10 07:50:10 2024-04-10 08:15:19 0:25:09 0:15:51 0:09:18 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-10T08:10:00.003182+0000 mon.a (mon.0) 834 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log

pass 7646304 2024-04-08 21:10:57 2024-04-10 07:50:10 2024-04-10 08:22:14 0:32:04 0:22:13 0:09:51 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7646305 2024-04-08 21:10:58 2024-04-10 07:50:41 2024-04-10 08:20:29 0:29:48 0:19:32 0:10:16 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
fail 7646306 2024-04-08 21:10:59 2024-04-10 00:24:41 2024-04-10 01:25:57 1:01:16 0:51:44 0:09:32 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-10T00:50:00.161024+0000 mon.a (mon.0) 1254 : cluster 3 [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets" in cluster log

pass 7646307 2024-04-08 21:11:00 2024-04-10 07:50:41 2024-04-10 08:19:40 0:28:59 0:16:15 0:12:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_host_drain} 3
fail 7646308 2024-04-08 21:11:01 2024-04-10 07:52:22 2024-04-10 08:41:57 0:49:35 0:37:56 0:11:39 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7646309 2024-04-08 21:11:02 2024-04-10 07:53:03 2024-04-10 08:18:12 0:25:09 0:14:13 0:10:56 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
pass 7646310 2024-04-08 21:11:03 2024-04-10 07:53:03 2024-04-10 08:16:31 0:23:28 0:12:48 0:10:40 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7646311 2024-04-08 21:11:04 2024-04-10 07:53:23 2024-04-10 08:18:56 0:25:33 0:14:30 0:11:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7646312 2024-04-08 21:11:05 2024-04-10 07:53:34 2024-04-10 08:17:20 0:23:46 0:12:49 0:10:57 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7646313 2024-04-08 21:11:06 2024-04-10 07:53:34 2024-04-10 08:20:34 0:27:00 0:11:51 0:15:09 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-10T08:14:07.707564+0000 mon.a (mon.0) 267 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7646314 2024-04-08 21:11:07 2024-04-10 07:57:25 2024-04-10 08:24:34 0:27:09 0:15:39 0:11:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7646315 2024-04-08 21:11:08 2024-04-10 07:57:25 2024-04-10 08:23:58 0:26:33 0:14:53 0:11:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7646316 2024-04-08 21:11:09 2024-04-10 07:58:56 2024-04-10 08:31:09 0:32:13 0:21:30 0:10:43 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7646317 2024-04-08 21:11:10 2024-04-10 07:58:56 2024-04-10 08:22:21 0:23:25 0:13:49 0:09:36 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7646318 2024-04-08 21:11:11 2024-04-10 07:59:27 2024-04-10 08:52:48 0:53:21 0:43:19 0:10:02 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-10T08:40:00.000145+0000 mon.a (mon.0) 1535 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7646319 2024-04-08 21:11:12 2024-04-10 07:59:27 2024-04-10 08:18:20 0:18:53 0:07:24 0:11:29 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/host rook/1.7.2} 3
Failure Reason:

Command failed on smithi122 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646320 2024-04-08 21:11:13 2024-04-10 08:00:38 2024-04-10 08:26:27 0:25:49 0:15:40 0:10:09 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7646321 2024-04-08 21:11:14 2024-04-10 08:01:38 2024-04-10 08:34:28 0:32:50 0:20:49 0:12:01 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_monitoring_stack_basic} 3
pass 7646322 2024-04-08 21:11:15 2024-04-10 08:03:19 2024-04-10 08:46:39 0:43:20 0:33:23 0:09:57 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7646323 2024-04-08 21:11:16 2024-04-10 08:03:29 2024-04-10 08:29:32 0:26:03 0:14:08 0:11:55 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
pass 7646324 2024-04-08 21:11:17 2024-04-10 08:18:31 2024-04-10 08:46:19 0:27:48 0:17:22 0:10:26 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
fail 7646325 2024-04-08 21:11:18 2024-04-10 08:19:02 2024-04-10 09:03:12 0:44:10 0:32:36 0:11:34 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-04-10T08:41:12.728046+0000 mon.a (mon.0) 201 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7646326 2024-04-08 21:11:19 2024-04-10 08:19:42 2024-04-10 08:45:41 0:25:59 0:15:14 0:10:45 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7646327 2024-04-08 21:11:20 2024-04-10 08:20:33 2024-04-10 08:56:03 0:35:30 0:24:04 0:11:26 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7646328 2024-04-08 21:11:21 2024-04-10 08:22:23 2024-04-10 09:05:32 0:43:09 0:31:41 0:11:28 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-04-10T08:38:59.079227+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7646329 2024-04-08 21:11:22 2024-04-08 22:15:32 2024-04-08 22:38:21 0:22:49 0:13:45 0:09:04 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7646330 2024-04-08 21:11:23 2024-04-10 08:22:24 2024-04-10 08:49:47 0:27:23 0:14:12 0:13:11 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7646331 2024-04-08 21:11:24 2024-04-10 08:24:24 2024-04-10 08:49:49 0:25:25 0:14:58 0:10:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7646332 2024-04-08 21:11:25 2024-04-10 08:24:35 2024-04-10 08:48:22 0:23:47 0:11:49 0:11:58 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
fail 7646333 2024-04-08 21:11:26 2024-04-10 08:26:35 2024-04-10 09:14:10 0:47:35 0:37:09 0:10:26 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-10T09:00:00.000193+0000 mon.a (mon.0) 1660 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

pass 7646334 2024-04-08 21:11:27 2024-04-10 08:27:06 2024-04-10 08:55:03 0:27:57 0:14:37 0:13:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7646335 2024-04-08 21:11:28 2024-04-10 08:29:37 2024-04-10 09:01:47 0:32:10 0:21:59 0:10:11 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7646336 2024-04-08 21:11:29 2024-04-10 08:30:17 2024-04-10 09:17:20 0:47:03 0:36:44 0:10:19 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7646337 2024-04-08 21:11:30 2024-04-10 08:30:17 2024-04-10 09:18:34 0:48:17 0:37:07 0:11:10 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7646338 2024-04-08 21:11:31 2024-04-10 08:30:18 2024-04-10 09:00:10 0:29:52 0:19:34 0:10:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7646339 2024-04-08 21:11:32 2024-04-10 08:30:18 2024-04-10 08:59:45 0:29:27 0:20:54 0:08:33 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7646340 2024-04-08 21:11:33 2024-04-10 08:30:19 2024-04-10 08:55:29 0:25:10 0:14:45 0:10:25 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
dead 7646341 2024-04-08 21:11:34 2024-04-10 08:30:19 2024-04-10 08:31:43 0:01:24 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi174

fail 7646342 2024-04-08 21:11:35 2024-04-10 08:30:39 2024-04-10 08:53:10 0:22:31 0:12:01 0:10:30 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi008 with status 5: 'sudo systemctl stop ceph-3e16b038-f717-11ee-b64a-cb9ed24678a4@mon.a'