Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7657244 2024-04-15 20:10:00 2024-04-15 20:11:10 2024-04-15 21:06:52 0:55:42 0:44:46 0:10:56 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7657245 2024-04-15 20:10:01 2024-04-15 20:11:10 2024-04-15 20:48:50 0:37:40 0:28:15 0:09:25 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7657246 2024-04-15 20:10:02 2024-04-15 20:11:11 2024-04-15 20:48:59 0:37:48 0:24:48 0:13:00 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7657247 2024-04-15 20:10:03 2024-04-15 20:11:11 2024-04-15 20:50:22 0:39:11 0:24:02 0:15:09 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-15T20:41:34.335855+0000 mon.a (mon.0) 439 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657248 2024-04-15 20:10:03 2024-04-15 20:11:12 2024-04-15 20:33:20 0:22:08 0:08:38 0:13:30 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/master} 1
Failure Reason:

Command failed on smithi083 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7657249 2024-04-15 20:10:04 2024-04-15 20:11:12 2024-04-15 20:37:26 0:26:14 0:14:52 0:11:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
fail 7657250 2024-04-15 20:10:05 2024-04-15 20:11:12 2024-04-15 20:42:30 0:31:18 0:17:08 0:14:10 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-15T20:38:11.915144+0000 mon.smithi062 (mon.0) 824 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657251 2024-04-15 20:10:06 2024-04-15 20:11:13 2024-04-15 20:40:15 0:29:02 0:18:29 0:10:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7657252 2024-04-15 20:10:07 2024-04-15 20:11:13 2024-04-15 20:55:01 0:43:48 0:27:14 0:16:34 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7657253 2024-04-15 20:10:08 2024-04-15 20:11:13 2024-04-15 20:41:43 0:30:30 0:18:25 0:12:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7657254 2024-04-15 20:10:09 2024-04-15 20:11:14 2024-04-15 21:19:55 1:08:41 0:52:04 0:16:37 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
pass 7657255 2024-04-15 20:10:10 2024-04-15 20:11:14 2024-04-15 20:54:27 0:43:13 0:29:49 0:13:24 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7657256 2024-04-15 20:10:11 2024-04-15 20:11:14 2024-04-15 20:51:50 0:40:36 0:24:55 0:15:41 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
pass 7657257 2024-04-15 20:10:12 2024-04-15 20:11:15 2024-04-15 21:04:46 0:53:31 0:39:58 0:13:33 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7657258 2024-04-15 20:10:12 2024-04-15 20:11:15 2024-04-15 20:57:19 0:46:04 0:35:15 0:10:49 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
fail 7657259 2024-04-15 20:10:13 2024-04-15 20:11:16 2024-04-15 21:14:29 1:03:13 0:48:53 0:14:20 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-04-15T20:43:25.637243+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

pass 7657260 2024-04-15 20:10:14 2024-04-15 20:11:16 2024-04-15 20:41:34 0:30:18 0:18:36 0:11:42 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_adoption} 1
pass 7657261 2024-04-15 20:10:15 2024-04-15 20:11:16 2024-04-15 20:37:46 0:26:30 0:13:11 0:13:19 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
pass 7657262 2024-04-15 20:10:16 2024-04-15 20:11:17 2024-04-15 20:35:32 0:24:15 0:13:29 0:10:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/deploy-raw} 2
fail 7657263 2024-04-15 20:10:17 2024-04-15 20:11:17 2024-04-15 20:47:33 0:36:16 0:26:06 0:10:10 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-15T20:29:24.220858+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7657264 2024-04-15 20:10:18 2024-04-15 20:11:18 2024-04-15 20:44:55 0:33:37 0:20:31 0:13:06 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
pass 7657265 2024-04-15 20:10:19 2024-04-15 20:11:18 2024-04-15 20:48:09 0:36:51 0:24:00 0:12:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7657266 2024-04-15 20:10:20 2024-04-15 20:11:19 2024-04-15 20:34:59 0:23:40 0:11:57 0:11:43 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
pass 7657267 2024-04-15 20:10:20 2024-04-15 20:11:19 2024-04-15 20:45:39 0:34:20 0:21:36 0:12:44 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7657268 2024-04-15 20:10:21 2024-04-15 20:11:19 2024-04-15 21:04:15 0:52:56 0:39:37 0:13:19 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
fail 7657269 2024-04-15 20:10:22 2024-04-15 20:11:20 2024-04-15 20:50:23 0:39:03 0:28:13 0:10:50 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-15T20:41:24.553766+0000 mon.a (mon.0) 400 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657270 2024-04-15 20:10:23 2024-04-15 20:11:20 2024-04-15 20:50:30 0:39:10 0:25:31 0:13:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7657271 2024-04-15 20:10:24 2024-04-15 20:11:21 2024-04-15 20:54:17 0:42:56 0:30:54 0:12:02 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7657272 2024-04-15 20:10:25 2024-04-15 20:11:21 2024-04-15 20:49:28 0:38:07 0:22:29 0:15:38 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7657273 2024-04-15 20:10:26 2024-04-15 20:11:21 2024-04-15 20:52:39 0:41:18 0:25:02 0:16:16 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7657274 2024-04-15 20:10:27 2024-04-15 20:11:32 2024-04-15 20:49:16 0:37:44 0:23:49 0:13:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-15T20:45:55.303837+0000 mon.smithi038 (mon.0) 832 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657275 2024-04-15 20:10:28 2024-04-15 20:11:32 2024-04-15 21:03:52 0:52:20 0:40:08 0:12:12 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
fail 7657276 2024-04-15 20:10:28 2024-04-15 20:11:32 2024-04-15 21:01:28 0:49:56 0:39:12 0:10:44 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7657277 2024-04-15 20:10:29 2024-04-15 20:11:33 2024-04-15 20:43:36 0:32:03 0:16:30 0:15:33 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-15T20:40:39.519928+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7657278 2024-04-15 20:10:30 2024-04-15 20:11:33 2024-04-15 20:47:19 0:35:46 0:20:16 0:15:30 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7657279 2024-04-15 20:10:31 2024-04-15 20:11:34 2024-04-15 20:42:58 0:31:24 0:18:09 0:13:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7657280 2024-04-15 20:10:32 2024-04-15 20:11:34 2024-04-15 20:56:58 0:45:24 0:31:15 0:14:09 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
fail 7657281 2024-04-15 20:10:33 2024-04-15 20:11:34 2024-04-15 20:51:28 0:39:54 0:24:07 0:15:47 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-15T20:41:58.543496+0000 mon.a (mon.0) 257 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657282 2024-04-15 20:10:34 2024-04-15 20:11:35 2024-04-15 20:49:42 0:38:07 0:23:16 0:14:51 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7657283 2024-04-15 20:10:35 2024-04-15 20:11:35 2024-04-15 20:43:52 0:32:17 0:22:04 0:10:13 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7657284 2024-04-15 20:10:36 2024-04-15 20:11:36 2024-04-15 20:48:25 0:36:49 0:21:58 0:14:51 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7657285 2024-04-15 20:10:37 2024-04-15 20:11:36 2024-04-15 20:48:36 0:37:00 0:22:54 0:14:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7657286 2024-04-15 20:10:38 2024-04-15 20:11:37 2024-04-15 20:47:54 0:36:17 0:24:03 0:12:14 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-15T20:43:40.228634+0000 mon.a (mon.0) 449 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657287 2024-04-15 20:10:39 2024-04-15 20:11:37 2024-04-15 21:05:50 0:54:13 0:39:18 0:14:55 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
fail 7657288 2024-04-15 20:10:40 2024-04-15 20:11:37 2024-04-15 20:40:06 0:28:29 0:17:32 0:10:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

"2024-04-15T20:37:25.710137+0000 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657289 2024-04-15 20:10:41 2024-04-15 20:11:38 2024-04-15 20:43:34 0:31:56 0:17:58 0:13:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7657290 2024-04-15 20:10:42 2024-04-15 20:11:38 2024-04-15 21:18:41 1:07:03 0:55:15 0:11:48 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
fail 7657291 2024-04-15 20:10:43 2024-04-15 20:11:39 2024-04-15 21:05:30 0:53:51 0:39:46 0:14:05 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-04-15T21:00:00.000108+0000 mon.smithi047 (mon.0) 775 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is degraded" in cluster log

fail 7657292 2024-04-15 20:10:44 2024-04-15 20:11:39 2024-04-15 20:50:20 0:38:41 0:28:16 0:10:25 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-15T20:48:27.069016+0000 mon.a (mon.0) 531 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7657293 2024-04-15 20:10:45 2024-04-15 20:11:39 2024-04-15 20:47:51 0:36:12 0:21:48 0:14:24 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7657294 2024-04-15 20:10:45 2024-04-15 20:11:40 2024-04-15 20:34:40 0:23:00 0:08:51 0:14:09 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/1.7.2} 3
Failure Reason:

Command failed on smithi112 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7657295 2024-04-15 20:10:46 2024-04-15 20:11:40 2024-04-15 20:49:33 0:37:53 0:26:37 0:11:16 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
fail 7657296 2024-04-15 20:10:47 2024-04-15 20:11:40 2024-04-15 20:56:59 0:45:19 0:31:52 0:13:27 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-15T20:49:02.284881+0000 mon.smithi132 (mon.0) 869 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7657297 2024-04-15 20:10:48 2024-04-15 20:11:41 2024-04-15 21:17:09 1:05:28 0:51:02 0:14:26 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-15T20:55:51.501729+0000 mon.a (mon.0) 1067 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657298 2024-04-15 20:10:49 2024-04-15 20:11:41 2024-04-15 20:44:31 0:32:50 0:22:30 0:10:20 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_cephadm} 1
fail 7657299 2024-04-15 20:10:50 2024-04-15 20:11:41 2024-04-15 20:43:38 0:31:57 0:18:51 0:13:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-15T20:38:50.441417+0000 mon.smithi131 (mon.0) 1026 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657300 2024-04-15 20:10:51 2024-04-15 20:11:42 2024-04-15 20:51:27 0:39:45 0:24:23 0:15:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-15T20:46:35.383663+0000 mon.smithi107 (mon.0) 868 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657301 2024-04-15 20:10:52 2024-04-15 20:11:42 2024-04-15 20:51:45 0:40:03 0:24:04 0:15:59 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7657302 2024-04-15 20:10:53 2024-04-15 20:11:43 2024-04-15 20:47:19 0:35:36 0:21:31 0:14:05 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7657303 2024-04-15 20:10:54 2024-04-15 20:11:43 2024-04-15 20:36:53 0:25:10 0:10:25 0:14:45 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3f7db60e5ba4c4d053b594b7ec58703d2833c55b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7657304 2024-04-15 20:10:55 2024-04-15 20:11:43 2024-04-15 20:48:57 0:37:14 0:25:52 0:11:22 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-15T20:43:30.090655+0000 mon.smithi106 (mon.0) 756 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657305 2024-04-15 20:10:56 2024-04-15 20:11:44 2024-04-15 21:08:45 0:57:01 0:37:40 0:19:21 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
fail 7657306 2024-04-15 20:10:57 2024-04-15 20:21:36 2024-04-15 21:22:54 1:01:18 0:37:52 0:23:26 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7657307 2024-04-15 20:10:58 2024-04-15 20:35:08 2024-04-15 21:10:59 0:35:51 0:23:31 0:12:20 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli_mon} 5
pass 7657308 2024-04-15 20:10:59 2024-04-15 20:37:49 2024-04-15 21:04:17 0:26:28 0:14:07 0:12:21 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7657309 2024-04-15 20:10:59 2024-04-15 20:40:19 2024-04-15 21:02:56 0:22:37 0:11:09 0:11:28 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7657310 2024-04-15 20:11:00 2024-04-15 20:41:40 2024-04-15 21:06:42 0:25:02 0:13:42 0:11:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7657311 2024-04-15 20:11:01 2024-04-15 20:43:53 2024-04-15 21:13:19 0:29:26 0:19:29 0:09:57 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
dead 7657312 2024-04-15 20:11:02 2024-04-15 20:43:53 2024-04-16 08:52:37 12:08:44 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

hit max job timeout

pass 7657313 2024-04-15 20:11:03 2024-04-15 20:43:54 2024-04-15 21:09:12 0:25:18 0:14:41 0:10:37 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_extra_daemon_features} 2
pass 7657314 2024-04-15 20:11:04 2024-04-15 20:43:54 2024-04-15 21:07:03 0:23:09 0:14:06 0:09:03 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
fail 7657315 2024-04-15 20:11:05 2024-04-15 20:43:54 2024-04-15 21:15:31 0:31:37 0:20:57 0:10:40 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-15T21:09:54.856923+0000 mon.smithi112 (mon.0) 802 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657316 2024-04-15 20:11:06 2024-04-15 20:43:55 2024-04-15 21:16:53 0:32:58 0:23:54 0:09:04 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
fail 7657317 2024-04-15 20:11:07 2024-04-15 20:43:55 2024-04-15 21:38:54 0:54:59 0:44:52 0:10:07 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b97cffaa-fb6a-11ee-bc8f-c7b262605968 -e sha1=87dbfb632e974cb2c28d17423071d2d833b53b29 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7657318 2024-04-15 20:11:08 2024-04-15 20:43:56 2024-04-15 21:18:27 0:34:31 0:25:03 0:09:28 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7657319 2024-04-15 20:11:08 2024-04-15 20:44:56 2024-04-15 21:11:14 0:26:18 0:15:50 0:10:28 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-04-15T21:06:29.685210+0000 mon.a (mon.0) 471 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657320 2024-04-15 20:11:09 2024-04-15 20:45:46 2024-04-15 21:13:17 0:27:31 0:15:52 0:11:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7657321 2024-04-15 20:11:10 2024-04-15 20:47:27 2024-04-15 21:35:11 0:47:44 0:37:52 0:09:52 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7657322 2024-04-15 20:11:11 2024-04-15 20:47:27 2024-04-15 21:06:25 0:18:58 0:10:16 0:08:42 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_adoption} 1
pass 7657323 2024-04-15 20:11:12 2024-04-15 20:47:28 2024-04-15 21:17:43 0:30:15 0:19:33 0:10:42 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/deploy-raw} 2
pass 7657324 2024-04-15 20:11:13 2024-04-15 20:47:58 2024-04-15 21:11:19 0:23:21 0:13:58 0:09:23 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
fail 7657325 2024-04-15 20:11:14 2024-04-15 21:24:18 1595 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-15T21:10:12.518207+0000 mon.a (mon.0) 385 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657326 2024-04-15 20:11:15 2024-04-15 20:48:29 2024-04-15 21:11:38 0:23:09 0:13:42 0:09:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7657327 2024-04-15 20:11:16 2024-04-15 20:48:39 2024-04-15 22:03:58 1:15:19 1:05:32 0:09:47 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7657328 2024-04-15 20:11:17 2024-04-15 20:49:00 2024-04-15 21:18:42 0:29:42 0:20:56 0:08:46 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7657329 2024-04-15 20:11:18 2024-04-15 20:49:00 2024-04-15 21:12:24 0:23:24 0:14:16 0:09:08 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7657330 2024-04-15 20:11:19 2024-04-15 20:49:30 2024-04-15 21:15:03 0:25:33 0:15:03 0:10:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7657331 2024-04-15 20:11:20 2024-04-15 20:49:41 2024-04-15 21:10:54 0:21:13 0:12:06 0:09:07 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7657332 2024-04-15 20:11:21 2024-04-15 21:23:26 1422 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
dead 7657333 2024-04-15 20:11:22 2024-04-15 20:50:32 2024-04-15 21:52:18 1:01:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

Error reimaging machines: reached maximum tries (241) after waiting for 3600 seconds

fail 7657334 2024-04-15 20:11:23 2024-04-15 20:51:52 2024-04-15 21:18:41 0:26:49 0:15:12 0:11:37 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-15T21:16:12.071713+0000 mon.smithi049 (mon.0) 819 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657335 2024-04-15 20:11:24 2024-04-15 20:51:53 2024-04-15 21:21:47 0:29:54 0:19:14 0:10:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-15T21:12:21.480021+0000 mon.a (mon.0) 383 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657336 2024-04-15 20:11:25 2024-04-15 20:52:43 2024-04-15 22:00:09 1:07:26 0:51:28 0:15:58 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi066 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9bcc0d0a-fb6c-11ee-bc8f-c7b262605968 -e sha1=87dbfb632e974cb2c28d17423071d2d833b53b29 -- bash -c 'ceph orch ps'"

fail 7657337 2024-04-15 20:11:26 2024-04-15 20:54:24 2024-04-15 21:24:12 0:29:48 0:19:27 0:10:21 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-15T21:21:47.075752+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7657338 2024-04-15 20:11:27 2024-04-15 20:54:24 2024-04-15 21:25:32 0:31:08 0:18:27 0:12:41 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7657339 2024-04-15 20:11:28 2024-04-15 20:54:34 2024-04-15 21:25:32 0:30:58 0:20:07 0:10:51 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7657340 2024-04-15 20:11:29 2024-04-15 20:55:05 2024-04-15 21:14:35 0:19:30 0:07:17 0:12:13 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/master} 1
Failure Reason:

Command failed on smithi092 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7657341 2024-04-15 20:11:30 2024-04-15 20:57:06 2024-04-15 21:53:12 0:56:06 0:46:52 0:09:14 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
fail 7657342 2024-04-15 20:11:31 2024-04-15 20:57:26 2024-04-15 21:49:13 0:51:47 0:39:36 0:12:11 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-04-15T21:21:06.997315+0000 mon.a (mon.0) 458 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657343 2024-04-15 20:11:31 2024-04-15 20:59:17 2024-04-15 21:24:03 0:24:46 0:15:32 0:09:14 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7657344 2024-04-15 20:11:32 2024-04-15 20:59:17 2024-04-15 21:23:07 0:23:50 0:13:35 0:10:15 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_rgw_multisite} 3
pass 7657345 2024-04-15 20:11:33 2024-04-15 20:59:17 2024-04-15 21:22:49 0:23:32 0:14:02 0:09:30 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
fail 7657346 2024-04-15 20:11:34 2024-04-15 21:24:01 954 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-15T21:17:58.071544+0000 mon.a (mon.0) 653 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657347 2024-04-15 20:11:35 2024-04-15 20:59:18 2024-04-15 21:25:22 0:26:04 0:15:37 0:10:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7657348 2024-04-15 20:11:36 2024-04-15 20:59:19 2024-04-15 21:30:47 0:31:28 0:22:11 0:09:17 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-15T21:27:22.343967+0000 mon.smithi082 (mon.0) 972 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657349 2024-04-15 20:11:37 2024-04-15 20:59:19 2024-04-15 21:53:32 0:54:13 0:44:00 0:10:13 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-15T21:33:54.965653+0000 mon.a (mon.0) 1077 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657350 2024-04-15 20:11:38 2024-04-15 20:59:19 2024-04-15 21:30:51 0:31:32 0:22:04 0:09:28 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-04-15T21:22:26.389324+0000 mon.a (mon.0) 437 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657351 2024-04-15 20:11:39 2024-04-15 20:59:20 2024-04-15 21:47:13 0:47:53 0:37:52 0:10:01 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7657352 2024-04-15 20:11:41 2024-04-15 21:44:19 2100 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
fail 7657353 2024-04-15 20:11:42 2024-04-15 20:59:20 2024-04-15 21:48:16 0:48:56 0:35:02 0:13:54 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/nfs} 1
Failure Reason:

"2024-04-15T21:24:20.004196+0000 mon.a (mon.0) 612 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657354 2024-04-15 20:11:43 2024-04-15 21:03:01 2024-04-15 21:26:04 0:23:03 0:13:27 0:09:36 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-15T21:23:45.667486+0000 mon.a (mon.0) 540 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7657355 2024-04-15 20:11:44 2024-04-15 21:03:02 2024-04-15 21:27:04 0:24:02 0:14:03 0:09:59 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
pass 7657356 2024-04-15 20:11:45 2024-04-15 21:04:02 2024-04-15 21:34:29 0:30:27 0:19:42 0:10:45 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7657357 2024-04-15 20:11:46 2024-04-15 21:04:23 2024-04-15 21:40:29 0:36:06 0:24:48 0:11:18 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-15T21:21:09.282802+0000 mon.a (mon.0) 205 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7657358 2024-04-15 20:11:47 2024-04-15 21:04:53 2024-04-15 21:29:00 0:24:07 0:12:44 0:11:23 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
fail 7657359 2024-04-15 20:11:48 2024-04-15 21:05:54 2024-04-15 21:33:41 0:27:47 0:16:48 0:10:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-15T21:26:57.680656+0000 mon.smithi087 (mon.0) 847 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657360 2024-04-15 20:11:49 2024-04-15 21:06:34 2024-04-15 21:28:17 0:21:43 0:11:29 0:10:14 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
pass 7657361 2024-04-15 20:11:50 2024-04-15 21:06:45 2024-04-15 21:27:48 0:21:03 0:11:24 0:09:39 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7657362 2024-04-15 20:11:50 2024-04-15 21:06:55 2024-04-15 21:53:49 0:46:54 0:38:36 0:08:18 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7657363 2024-04-15 20:11:51 2024-04-15 21:06:56 2024-04-15 21:34:09 0:27:13 0:17:16 0:09:57 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7657364 2024-04-15 20:11:52 2024-04-15 21:07:06 2024-04-15 21:30:33 0:23:27 0:11:12 0:12:15 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi167 with status 5: 'sudo systemctl stop ceph-e5ec3232-fb6e-11ee-bc8f-c7b262605968@mon.a'

fail 7657365 2024-04-15 20:11:53 2024-04-16 01:32:42 2024-04-16 02:04:59 0:32:17 0:21:20 0:10:57 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-16T01:59:55.494561+0000 mon.smithi151 (mon.0) 860 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657366 2024-04-15 20:11:54 2024-04-16 01:32:53 2024-04-16 01:57:07 0:24:14 0:14:13 0:10:01 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
dead 7657367 2024-04-15 20:11:55 2024-04-16 01:34:13 2024-04-16 13:42:15 12:08:02 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

hit max job timeout

pass 7657368 2024-04-15 20:11:56 2024-04-16 01:34:54 2024-04-16 02:02:21 0:27:27 0:16:44 0:10:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm} 1
fail 7657369 2024-04-15 20:11:57 2024-04-16 01:34:54 2024-04-16 02:33:06 0:58:12 0:47:53 0:10:19 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-16T02:07:20.182464+0000 mon.a (mon.0) 998 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657370 2024-04-15 20:11:58 2024-04-16 01:35:55 2024-04-16 02:00:55 0:25:00 0:14:55 0:10:05 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7657371 2024-04-15 20:11:59 2024-04-16 01:36:05 2024-04-16 02:09:00 0:32:55 0:22:41 0:10:14 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
fail 7657372 2024-04-15 20:11:59 2024-04-16 01:36:05 2024-04-16 05:27:58 3:51:53 3:38:03 0:13:50 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi029 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3f7db60e5ba4c4d053b594b7ec58703d2833c55b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

pass 7657373 2024-04-15 20:12:00 2024-04-16 01:36:26 2024-04-16 02:13:37 0:37:11 0:24:56 0:12:15 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli_mon} 5
pass 7657374 2024-04-15 20:12:01 2024-04-16 01:38:06 2024-04-16 02:02:15 0:24:09 0:11:18 0:12:51 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7657375 2024-04-15 20:12:02 2024-04-16 01:41:17 2024-04-16 02:06:26 0:25:09 0:14:49 0:10:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7657376 2024-04-15 20:12:03 2024-04-16 01:41:18 2024-04-16 02:29:24 0:48:06 0:35:44 0:12:22 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7657377 2024-04-15 20:12:04 2024-04-16 01:42:38 2024-04-16 02:14:18 0:31:40 0:20:05 0:11:35 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7657378 2024-04-15 20:12:05 2024-04-16 01:44:39 2024-04-16 01:59:50 0:15:11 0:06:30 0:08:41 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_cephadm_repos} 1
fail 7657379 2024-04-15 20:12:06 2024-04-16 01:45:09 2024-04-16 02:10:18 0:25:09 0:14:46 0:10:23 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-16T02:05:38.946197+0000 mon.smithi042 (mon.0) 830 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657380 2024-04-15 20:12:07 2024-04-16 01:45:10 2024-04-16 02:16:48 0:31:38 0:17:59 0:13:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7657381 2024-04-15 20:12:08 2024-04-16 01:46:50 2024-04-16 02:11:23 0:24:33 0:12:36 0:11:57 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-16T02:06:44.675065+0000 mon.a (mon.0) 451 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657382 2024-04-15 20:12:08 2024-04-16 01:47:41 2024-04-16 02:12:59 0:25:18 0:16:25 0:08:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7657383 2024-04-15 20:12:09 2024-04-16 01:47:41 2024-04-16 02:17:22 0:29:41 0:19:33 0:10:08 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_extra_daemon_features} 2
Failure Reason:

"2024-04-16T02:10:59.080908+0000 mon.a (mon.0) 342 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657384 2024-04-15 20:12:10 2024-04-16 01:47:52 2024-04-16 03:06:34 1:18:42 1:07:52 0:10:50 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7657385 2024-04-15 20:12:11 2024-04-16 01:49:12 2024-04-16 02:06:52 0:17:40 0:07:32 0:10:08 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} 3
Failure Reason:

Command failed on smithi055 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7657386 2024-04-15 20:12:12 2024-04-16 01:49:23 2024-04-16 02:11:11 0:21:48 0:12:28 0:09:20 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_adoption} 1
pass 7657387 2024-04-15 20:12:13 2024-04-16 01:49:33 2024-04-16 02:12:33 0:23:00 0:14:04 0:08:56 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/deploy-raw} 2
pass 7657388 2024-04-15 20:12:14 2024-04-16 01:49:33 2024-04-16 02:19:23 0:29:50 0:19:17 0:10:33 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
pass 7657389 2024-04-15 20:12:15 2024-04-16 01:49:44 2024-04-16 02:20:07 0:30:23 0:19:43 0:10:40 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7657390 2024-04-15 20:12:16 2024-04-16 01:50:14 2024-04-16 02:25:50 0:35:36 0:25:02 0:10:34 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7657391 2024-04-15 20:12:16 2024-04-16 01:50:55 2024-04-16 02:15:05 0:24:10 0:13:38 0:10:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7657392 2024-04-15 20:12:17 2024-04-16 01:52:15 2024-04-16 02:56:05 1:03:50 0:52:54 0:10:56 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
fail 7657393 2024-04-15 20:12:18 2024-04-16 01:53:26 2024-04-16 02:18:45 0:25:19 0:15:24 0:09:55 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-16T02:16:55.868369+0000 mon.a (mon.0) 773 : cluster [WRN] Health check failed: 1 stray host(s) with 1 daemon(s) not managed by cephadm (CEPHADM_STRAY_HOST)" in cluster log

fail 7657394 2024-04-15 20:12:19 2024-04-16 01:53:36 2024-04-16 02:29:16 0:35:40 0:25:26 0:10:14 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-16T02:15:15.873713+0000 mon.a (mon.0) 456 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657395 2024-04-15 20:12:20 2024-04-16 01:53:36 2024-04-16 02:19:18 0:25:42 0:14:50 0:10:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7657396 2024-04-15 20:12:21 2024-04-16 01:53:37 2024-04-16 03:04:47 1:11:10 0:57:15 0:13:55 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
pass 7657397 2024-04-15 20:12:22 2024-04-16 01:57:08 2024-04-16 02:28:40 0:31:32 0:19:05 0:12:27 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7657398 2024-04-15 20:12:23 2024-04-16 01:57:48 2024-04-16 02:29:37 0:31:49 0:22:16 0:09:33 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-16T02:23:50.936223+0000 mon.smithi135 (mon.0) 789 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657399 2024-04-15 20:12:24 2024-04-16 01:57:48 2024-04-16 02:20:57 0:23:09 0:13:47 0:09:22 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-04-16T02:15:43.324131+0000 mon.a (mon.0) 183 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657400 2024-04-15 20:12:25 2024-04-16 01:57:59 2024-04-16 05:48:08 3:50:09 3:36:57 0:13:12 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi079 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3f7db60e5ba4c4d053b594b7ec58703d2833c55b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7657401 2024-04-15 20:12:25 2024-04-16 01:58:49 2024-04-16 02:21:53 0:23:04 0:13:56 0:09:08 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-16T02:20:23.338393+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7657402 2024-04-15 20:12:26 2024-04-16 01:59:00 2024-04-16 02:20:23 0:21:23 0:11:45 0:09:38 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
pass 7657403 2024-04-15 20:12:27 2024-04-16 01:59:30 2024-04-16 02:19:59 0:20:29 0:11:12 0:09:17 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7657404 2024-04-15 20:12:28 2024-04-16 01:59:31 2024-04-16 02:22:48 0:23:17 0:14:06 0:09:11 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7657405 2024-04-15 20:12:29 2024-04-16 01:59:31 2024-04-16 02:20:44 0:21:13 0:12:50 0:08:23 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
fail 7657406 2024-04-15 20:12:30 2024-04-16 01:59:51 2024-04-16 02:25:24 0:25:33 0:12:43 0:12:50 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-16T02:20:01.945835+0000 mon.a (mon.0) 452 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7657407 2024-04-15 20:12:31 2024-04-16 02:02:22 2024-04-16 02:27:46 0:25:24 0:15:33 0:09:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7657408 2024-04-15 20:12:32 2024-04-16 02:02:22 2024-04-16 02:26:12 0:23:50 0:14:03 0:09:47 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7657409 2024-04-15 20:12:33 2024-04-16 02:03:13 2024-04-16 02:57:27 0:54:14 0:44:15 0:09:59 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7657410 2024-04-15 20:12:34 2024-04-16 02:03:13 2024-04-16 02:48:38 0:45:25 0:35:44 0:09:41 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
pass 7657411 2024-04-15 20:12:35 2024-04-16 02:03:34 2024-04-16 02:40:42 0:37:08 0:25:11 0:11:57 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7657412 2024-04-15 20:12:36 2024-04-16 02:04:54 2024-04-16 02:38:18 0:33:24 0:21:17 0:12:07 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-16T02:34:07.724855+0000 mon.smithi139 (mon.0) 941 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7657413 2024-04-15 20:12:36 2024-04-16 02:06:35 2024-04-16 02:32:18 0:25:43 0:15:16 0:10:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7657414 2024-04-15 20:12:37 2024-04-16 02:06:55 2024-04-16 02:32:03 0:25:08 0:14:40 0:10:28 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7657415 2024-04-15 20:12:38 2024-04-16 02:09:06 2024-04-16 02:32:11 0:23:05 0:13:54 0:09:11 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-04-16T02:27:59.918863+0000 mon.a (mon.0) 293 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7657416 2024-04-15 20:12:39 2024-04-16 02:09:17 2024-04-16 03:26:30 1:17:13 1:07:17 0:09:56 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7657417 2024-04-15 20:12:40 2024-04-16 02:09:17 2024-04-16 02:32:07 0:22:50 0:13:02 0:09:48 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-16T02:29:23.831753+0000 mon.a (mon.0) 533 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7657418 2024-04-15 20:12:41 2024-04-16 02:09:17 2024-04-16 02:32:08 0:22:51 0:13:57 0:08:54 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7657419 2024-04-15 20:12:42 2024-04-16 02:09:28 2024-04-16 02:34:27 0:24:59 0:12:57 0:12:02 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
fail 7657420 2024-04-15 20:12:43 2024-04-16 02:11:18 2024-04-16 02:39:42 0:28:24 0:16:44 0:11:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-16T02:32:33.910942+0000 mon.smithi087 (mon.0) 852 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657421 2024-04-15 20:12:44 2024-04-16 02:12:39 2024-04-16 02:50:09 0:37:30 0:24:18 0:13:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
dead 7657422 2024-04-15 20:12:45 2024-04-16 02:12:49 2024-04-16 19:18:37 17:05:48 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

hit max job timeout

pass 7657423 2024-04-15 20:12:46 2024-04-16 02:13:00 2024-04-16 02:39:23 0:26:23 0:15:53 0:10:30 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7657424 2024-04-15 20:12:47 2024-04-16 02:13:40 2024-04-16 03:00:51 0:47:11 0:37:40 0:09:31 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
fail 7657425 2024-04-15 20:12:48 2024-04-16 02:13:41 2024-04-16 02:39:31 0:25:50 0:14:46 0:11:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-16T02:35:37.095599+0000 mon.smithi076 (mon.0) 842 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7657426 2024-04-15 20:12:48 2024-04-16 02:14:01 2024-04-16 02:36:18 0:22:17 0:11:04 0:11:13 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7657427 2024-04-15 20:12:49 2024-04-16 02:15:12 2024-04-16 02:59:10 0:43:58 0:32:48 0:11:10 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} 2
pass 7657428 2024-04-15 20:12:50 2024-04-16 02:15:42 2024-04-16 02:46:49 0:31:07 0:19:11 0:11:56 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7657429 2024-04-15 20:12:51 2024-04-16 02:16:53 2024-04-16 02:45:14 0:28:21 0:14:50 0:13:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-16T02:41:11.556311+0000 mon.smithi059 (mon.0) 790 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7657430 2024-04-15 20:12:52 2024-04-16 02:19:23 2024-04-16 02:40:26 0:21:03 0:11:46 0:09:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

"2024-04-16T02:38:13.556335+0000 mon.a (mon.0) 341 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7657431 2024-04-15 20:12:53 2024-04-16 02:19:24 2024-04-16 02:36:48 0:17:24 0:07:19 0:10:05 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi144 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"