Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7736292 2024-05-31 21:08:45 2024-06-01 00:40:28 2024-06-01 01:09:51 0:29:23 0:19:10 0:10:13 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7736293 2024-05-31 21:08:46 2024-06-01 00:40:28 2024-06-01 01:11:17 0:30:49 0:20:10 0:10:39 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7736294 2024-05-31 21:08:47 2024-06-01 00:40:28 2024-06-01 01:12:41 0:32:13 0:21:21 0:10:52 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-06-01T01:07:51.929141+0000 mon.smithi043 (mon.0) 813 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736295 2024-05-31 21:08:48 2024-06-01 00:40:29 2024-06-01 01:23:29 0:43:00 0:33:11 0:09:49 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

pass 7736296 2024-05-31 21:08:49 2024-06-01 00:40:49 2024-06-01 01:13:44 0:32:55 0:23:50 0:09:05 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
fail 7736297 2024-05-31 21:08:50 2024-06-01 00:40:50 2024-06-01 01:05:14 0:24:24 0:12:17 0:12:07 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi063 with status 5: 'sudo systemctl stop ceph-4eefbd8c-1fb2-11ef-bc9b-c7b262605968@mon.a'

dead 7736298 2024-05-31 21:08:52 2024-06-01 00:41:20 2024-06-01 12:51:42 12:10:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

fail 7736299 2024-05-31 21:08:53 2024-06-01 00:42:00 2024-06-01 01:17:41 0:35:41 0:26:02 0:09:39 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-06-01T01:03:51.334234+0000 mon.a (mon.0) 387 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736300 2024-05-31 21:08:54 2024-06-01 00:42:01 2024-06-01 01:28:31 0:46:30 0:36:23 0:10:07 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
dead 7736301 2024-05-31 21:08:55 2024-06-01 00:42:51 2024-06-01 00:44:05 0:01:14 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi045

pass 7736302 2024-05-31 21:08:56 2024-06-01 00:43:02 2024-06-01 01:08:05 0:25:03 0:15:34 0:09:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm} 1
fail 7736303 2024-05-31 21:08:57 2024-06-01 00:43:02 2024-06-01 01:16:57 0:33:55 0:22:17 0:11:38 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-06-01T01:04:51.940313+0000 mgr.smithi143.ebuuor (mgr.14238) 1 : cluster [ERR] Failed to load ceph-mgr modules: k8sevents" in cluster log

pass 7736304 2024-05-31 21:08:58 2024-06-01 00:43:43 2024-06-01 01:40:25 0:56:42 0:45:00 0:11:42 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
fail 7736305 2024-05-31 21:08:59 2024-06-01 00:45:33 2024-06-01 01:30:15 0:44:42 0:34:43 0:09:59 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/nfs} 1
Failure Reason:

"2024-06-01T01:14:19.976542+0000 mon.a (mon.0) 1395 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736306 2024-05-31 21:09:00 2024-06-01 00:45:33 2024-06-01 01:06:46 0:21:13 0:09:57 0:11:16 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_adoption} 1
pass 7736307 2024-05-31 21:09:01 2024-06-01 00:45:34 2024-06-01 01:11:35 0:26:01 0:14:52 0:11:09 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7736308 2024-05-31 21:09:02 2024-06-01 00:45:44 2024-06-01 01:13:45 0:28:01 0:12:58 0:15:03 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7736309 2024-05-31 21:09:03 2024-06-01 00:50:45 2024-06-01 01:29:16 0:38:31 0:25:49 0:12:42 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-06-01T01:09:30.180641+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7736310 2024-05-31 21:09:04 2024-06-01 00:52:26 2024-06-01 01:22:53 0:30:27 0:14:20 0:16:07 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
pass 7736311 2024-05-31 21:09:05 2024-06-01 00:57:07 2024-06-01 01:22:21 0:25:14 0:13:54 0:11:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7736312 2024-05-31 21:09:06 2024-06-01 00:57:07 2024-06-01 01:18:36 0:21:29 0:11:13 0:10:16 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
pass 7736313 2024-05-31 21:09:07 2024-06-01 00:57:28 2024-06-01 01:18:41 0:21:13 0:11:43 0:09:30 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7736314 2024-05-31 21:09:08 2024-06-01 00:57:28 2024-06-01 01:55:01 0:57:33 0:47:28 0:10:05 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-06-01T01:38:41.308584+0000 mon.a (mon.0) 1183 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736315 2024-05-31 21:09:10 2024-06-01 00:57:59 2024-06-01 01:24:48 0:26:49 0:13:57 0:12:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7736316 2024-05-31 21:09:11 2024-06-01 00:59:39 2024-06-01 02:00:54 1:01:15 0:51:53 0:09:22 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-06-01T01:25:25.224217+0000 mon.a (mon.0) 1086 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736317 2024-05-31 21:09:12 2024-06-01 00:59:40 2024-06-01 01:17:10 0:17:30 0:07:26 0:10:04 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} 3
Failure Reason:

Command failed on smithi092 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7736318 2024-05-31 21:09:13 2024-06-01 00:59:40 2024-06-01 01:16:24 0:16:44 0:06:54 0:09:50 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_cephadm_repos} 1
pass 7736319 2024-05-31 21:09:14 2024-06-01 00:59:40 2024-06-01 01:29:21 0:29:41 0:20:05 0:09:36 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
fail 7736320 2024-05-31 21:09:15 2024-06-01 00:59:41 2024-06-01 01:34:31 0:34:50 0:19:12 0:15:38 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-06-01T01:29:51.826112+0000 mon.smithi063 (mon.0) 795 : cluster [WRN] Health check failed: Failed to apply 2 service(s): jaeger-collector,jaeger-query (CEPHADM_APPLY_SPEC_FAIL)" in cluster log

pass 7736321 2024-05-31 21:09:16 2024-06-01 01:05:22 2024-06-01 01:29:21 0:23:59 0:14:25 0:09:34 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7736322 2024-05-31 21:09:17 2024-06-01 01:06:12 2024-06-01 02:18:00 1:11:48 1:02:23 0:09:25 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
fail 7736323 2024-05-31 21:09:18 2024-06-01 01:06:12 2024-06-01 01:37:35 0:31:23 0:19:01 0:12:22 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_extra_daemon_features} 2
Failure Reason:

"2024-06-01T01:30:33.136847+0000 mon.a (mon.0) 347 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

dead 7736324 2024-05-31 21:09:19 2024-06-01 01:07:23 2024-06-01 01:08:27 0:01:04 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi094

fail 7736325 2024-05-31 21:09:20 2024-06-01 01:07:23 2024-06-01 01:30:28 0:23:05 0:14:05 0:09:00 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-06-01T01:29:39.199938+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

dead 7736326 2024-05-31 21:09:21 2024-06-01 01:07:24 2024-06-01 01:10:58 0:03:34 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi052

pass 7736327 2024-05-31 21:09:22 2024-06-01 01:09:54 2024-06-01 01:36:49 0:26:55 0:14:50 0:12:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7736328 2024-05-31 21:09:23 2024-06-01 01:11:25 2024-06-01 01:36:36 0:25:11 0:15:23 0:09:48 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-06-01T01:28:52.759397+0000 mon.a (mon.0) 414 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736329 2024-05-31 21:09:24 2024-06-01 01:11:35 2024-06-01 01:44:35 0:33:00 0:22:19 0:10:41 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
fail 7736330 2024-05-31 21:09:25 2024-06-01 01:12:36 2024-06-01 01:40:15 0:27:39 0:16:22 0:11:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-06-01T01:37:27.115476+0000 mon.a (mon.0) 735 : cluster [WRN] Health check failed: 1 stray host(s) with 1 daemon(s) not managed by cephadm (CEPHADM_STRAY_HOST)" in cluster log

dead 7736331 2024-05-31 21:09:26 2024-06-01 01:12:46 2024-06-01 01:14:50 0:02:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi096

fail 7736332 2024-05-31 21:09:27 2024-06-01 01:13:47 2024-06-01 01:39:18 0:25:31 0:13:06 0:12:25 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-06-01T01:34:41.889708+0000 mon.a (mon.0) 474 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736333 2024-05-31 21:09:28 2024-06-01 01:16:17 2024-06-01 01:46:03 0:29:46 0:19:38 0:10:08 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7736334 2024-05-31 21:09:29 2024-06-01 01:16:28 2024-06-01 01:58:04 0:41:36 0:32:24 0:09:12 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} 2
pass 7736335 2024-05-31 21:09:30 2024-06-01 01:16:28 2024-06-01 02:24:55 1:08:27 0:58:14 0:10:13 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
pass 7736336 2024-05-31 21:09:31 2024-06-01 01:16:59 2024-06-01 01:42:05 0:25:06 0:15:45 0:09:21 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7736337 2024-05-31 21:09:32 2024-06-01 01:17:19 2024-06-01 01:58:45 0:41:26 0:31:13 0:10:13 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
fail 7736338 2024-05-31 21:09:33 2024-06-01 01:17:50 2024-06-01 01:42:40 0:24:50 0:14:05 0:10:45 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-06-01T01:36:48.540561+0000 mon.a (mon.0) 191 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736339 2024-05-31 21:09:35 2024-06-01 01:17:50 2024-06-01 01:50:00 0:32:10 0:20:23 0:11:47 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-06-01T01:38:59.829529+0000 mgr.smithi186.keyiht (mgr.14238) 1 : cluster [ERR] Failed to load ceph-mgr modules: k8sevents" in cluster log

fail 7736340 2024-05-31 21:09:36 2024-06-01 01:18:51 2024-06-01 01:52:17 0:33:26 0:23:41 0:09:45 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-06-01T01:48:34.436571+0000 mon.a (mon.0) 563 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7736341 2024-05-31 21:09:37 2024-06-01 01:18:51 2024-06-01 01:50:01 0:31:10 0:20:27 0:10:43 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
fail 7736342 2024-05-31 21:09:38 2024-06-01 01:20:31 2024-06-01 01:57:03 0:36:32 0:24:08 0:12:24 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-06-01T01:49:35.563708+0000 mon.smithi052 (mon.0) 869 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7736343 2024-05-31 21:09:39 2024-06-01 01:21:42 2024-06-01 01:48:52 0:27:10 0:16:28 0:10:42 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7736344 2024-05-31 21:09:40 2024-06-01 01:21:42 2024-06-01 01:44:36 0:22:54 0:12:55 0:09:59 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7736345 2024-05-31 21:09:41 2024-06-01 01:21:43 2024-06-01 01:57:24 0:35:41 0:25:57 0:09:44 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7736346 2024-05-31 21:09:42 2024-06-01 01:21:43 2024-06-01 01:48:37 0:26:54 0:15:06 0:11:48 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-06-01T01:44:48.596008+0000 mon.smithi008 (mon.0) 867 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7736347 2024-05-31 21:09:43 2024-06-01 01:23:04 2024-06-01 01:48:14 0:25:10 0:15:43 0:09:27 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7736348 2024-05-31 21:09:44 2024-06-01 01:23:34 2024-06-01 01:56:06 0:32:32 0:21:51 0:10:41 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-06-01T01:51:14.419572+0000 mon.smithi029 (mon.0) 773 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7736349 2024-05-31 21:09:45 2024-06-01 01:23:34 2024-06-01 01:59:45 0:36:11 0:23:45 0:12:26 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7736350 2024-05-31 21:09:46 2024-06-01 01:24:35 2024-06-01 02:12:06 0:47:31 0:37:54 0:09:37 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
fail 7736351 2024-05-31 21:09:47 2024-06-01 01:24:55 2024-06-01 01:50:57 0:26:02 0:14:04 0:11:58 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-06-01T01:46:11.856243+0000 mon.a (mon.0) 408 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736352 2024-05-31 21:09:48 2024-06-01 01:25:56 2024-06-01 02:15:39 0:49:43 0:37:57 0:11:46 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7736353 2024-05-31 21:09:49 2024-06-01 01:28:37 2024-06-01 02:04:32 0:35:55 0:25:20 0:10:35 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli_mon} 5
pass 7736354 2024-05-31 21:09:50 2024-06-01 01:29:27 2024-06-01 01:50:23 0:20:56 0:11:54 0:09:02 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7736355 2024-05-31 21:09:51 2024-06-01 01:29:27 2024-06-01 01:51:56 0:22:29 0:11:27 0:11:02 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7736356 2024-05-31 21:09:52 2024-06-01 01:30:18 2024-06-01 01:56:08 0:25:50 0:13:50 0:12:00 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7736357 2024-05-31 21:09:53 2024-06-01 01:32:59 2024-06-01 01:55:48 0:22:49 0:12:52 0:09:57 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
fail 7736358 2024-05-31 21:09:54 2024-06-01 01:32:59 2024-06-01 01:55:58 0:22:59 0:12:03 0:10:56 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-06-01T01:51:59.855001+0000 mon.a (mon.0) 391 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736359 2024-05-31 21:09:55 2024-06-01 01:34:40 2024-06-01 02:01:43 0:27:03 0:15:25 0:11:38 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-06-01T01:58:35.705388+0000 mon.smithi003 (mon.0) 816 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736360 2024-05-31 21:09:56 2024-06-01 01:36:11 2024-06-01 02:19:52 0:43:41 0:33:02 0:10:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

fail 7736361 2024-05-31 21:09:57 2024-06-01 01:36:11 2024-06-01 02:19:43 0:43:32 0:33:26 0:10:06 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-06-01T01:53:02.310647+0000 mon.a (mon.0) 374 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736362 2024-05-31 21:09:58 2024-06-01 01:36:41 2024-06-01 01:54:05 0:17:24 0:07:35 0:09:49 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/1.7.2} 1
Failure Reason:

Command failed on smithi028 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7736363 2024-05-31 21:09:59 2024-06-01 01:36:52 2024-06-01 02:04:39 0:27:47 0:16:11 0:11:36 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_set_mon_crush_locations} 3
dead 7736364 2024-05-31 21:10:01 2024-06-01 01:37:12 2024-06-01 13:49:23 12:12:11 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7736365 2024-05-31 21:10:02 2024-06-01 01:39:23 2024-06-01 02:07:48 0:28:25 0:15:52 0:12:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7736366 2024-05-31 21:10:03 2024-06-01 01:40:23 2024-06-01 02:29:44 0:49:21 0:38:31 0:10:50 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7736367 2024-05-31 21:10:04 2024-06-01 01:40:24 2024-06-01 01:59:14 0:18:50 0:10:07 0:08:43 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_adoption} 1
pass 7736368 2024-05-31 21:10:05 2024-06-01 01:40:34 2024-06-01 02:09:38 0:29:04 0:18:22 0:10:42 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7736369 2024-05-31 21:10:06 2024-06-01 01:42:14 2024-06-01 02:05:36 0:23:22 0:13:27 0:09:55 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
fail 7736370 2024-05-31 21:10:07 2024-06-01 01:42:45 2024-06-01 02:20:41 0:37:56 0:27:24 0:10:32 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-06-01T02:05:10.772559+0000 mon.a (mon.0) 381 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736372 2024-05-31 21:10:08 2024-06-01 01:43:05 2024-06-01 02:08:21 0:25:16 0:13:37 0:11:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
fail 7736374 2024-05-31 21:10:09 2024-06-01 01:44:36 2024-06-01 02:47:36 1:03:00 0:52:47 0:10:13 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-06-01T02:10:15.655443+0000 mon.a (mon.0) 932 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736376 2024-05-31 21:10:10 2024-06-01 01:44:46 2024-06-01 02:49:04 1:04:18 0:52:23 0:11:55 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-06-01T02:11:51.313151+0000 mon.a (mon.0) 986 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736377 2024-05-31 21:10:11 2024-06-01 01:46:07 2024-06-01 02:10:57 0:24:50 0:12:22 0:12:28 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

"2024-06-01T02:07:58.942327+0000 mon.a (mon.0) 351 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736378 2024-05-31 21:10:12 2024-06-01 01:48:17 2024-06-01 02:19:05 0:30:48 0:19:50 0:10:58 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7736379 2024-05-31 21:10:13 2024-06-01 01:48:38 2024-06-01 02:14:11 0:25:33 0:14:02 0:11:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7736380 2024-05-31 21:10:14 2024-06-01 01:48:58 2024-06-01 02:13:14 0:24:16 0:11:44 0:12:32 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7736381 2024-05-31 21:10:15 2024-06-01 01:50:09 2024-06-01 02:19:39 0:29:30 0:19:40 0:09:50 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_cephadm} 1
fail 7736382 2024-05-31 21:10:16 2024-06-01 01:50:09 2024-06-01 02:15:45 0:25:36 0:14:58 0:10:38 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-06-01T02:12:16.376248+0000 mon.smithi142 (mon.0) 819 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736383 2024-05-31 21:10:17 2024-06-01 01:50:29 2024-06-01 02:14:05 0:23:36 0:13:47 0:09:49 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7736384 2024-05-31 21:10:18 2024-06-01 01:51:00 2024-06-01 02:34:42 0:43:42 0:33:34 0:10:08 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7736385 2024-05-31 21:10:19 2024-06-01 01:52:00 2024-06-01 02:22:17 0:30:17 0:20:04 0:10:13 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-06-01T02:20:26.742620+0000 mon.a (mon.0) 214 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7736386 2024-05-31 21:10:20 2024-06-01 01:52:01 2024-06-01 02:21:42 0:29:41 0:18:44 0:10:57 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7736387 2024-05-31 21:10:21 2024-06-01 01:54:11 2024-06-01 02:26:48 0:32:37 0:21:02 0:11:35 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7736388 2024-05-31 21:10:22 2024-06-01 01:55:12 2024-06-01 02:12:40 0:17:28 0:06:41 0:10:47 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm_repos} 1
pass 7736389 2024-05-31 21:10:23 2024-06-01 01:55:52 2024-06-01 03:08:05 1:12:13 1:01:51 0:10:22 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
fail 7736390 2024-05-31 21:10:24 2024-06-01 01:56:03 2024-06-01 02:58:06 1:02:03 0:51:49 0:10:14 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi089 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80ca5b32-1fbc-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7736391 2024-05-31 21:10:25 2024-06-01 01:56:13 2024-06-01 02:29:20 0:33:07 0:23:28 0:09:39 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
pass 7736392 2024-05-31 21:10:26 2024-06-01 01:56:13 2024-06-01 02:21:53 0:25:40 0:15:43 0:09:57 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7736393 2024-05-31 21:10:27 2024-06-01 01:57:04 2024-06-01 02:22:37 0:25:33 0:15:24 0:10:09 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7736394 2024-05-31 21:10:28 2024-06-01 01:57:24 2024-06-01 02:22:30 0:25:06 0:15:54 0:09:12 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7736395 2024-05-31 21:10:29 2024-06-01 01:57:25 2024-06-01 02:20:58 0:23:33 0:14:10 0:09:23 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_extra_daemon_features} 2
pass 7736396 2024-05-31 21:10:30 2024-06-01 01:58:05 2024-06-01 02:22:00 0:23:55 0:13:51 0:10:04 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7736397 2024-05-31 21:10:31 2024-06-01 01:58:56 2024-06-01 02:31:44 0:32:48 0:21:37 0:11:11 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-06-01T02:28:37.243297+0000 mon.smithi097 (mon.0) 966 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736398 2024-05-31 21:10:32 2024-06-01 01:59:46 2024-06-01 02:50:12 0:50:26 0:38:57 0:11:29 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7736399 2024-05-31 21:10:33 2024-06-01 02:00:57 2024-06-01 02:59:23 0:58:26 0:46:38 0:11:48 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-06-01T02:26:57.925356+0000 mon.smithi003 (mon.0) 742 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7736400 2024-05-31 21:10:34 2024-06-01 02:01:47 2024-06-01 03:05:38 1:03:51 0:52:54 0:10:57 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-06-01T02:27:44.782829+0000 mon.a (mon.0) 333 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

fail 7736401 2024-05-31 21:10:35 2024-06-01 02:01:47 2024-06-01 02:29:32 0:27:45 0:15:06 0:12:39 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-06-01T02:27:24.643235+0000 mon.a (mon.0) 563 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7736402 2024-05-31 21:10:36 2024-06-01 02:04:38 2024-06-01 02:25:42 0:21:04 0:11:45 0:09:19 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
fail 7736403 2024-05-31 21:10:37 2024-06-01 02:04:38 2024-06-01 02:42:14 0:37:36 0:26:00 0:11:36 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-06-01T02:22:27.529726+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7736404 2024-05-31 21:10:38 2024-06-01 02:04:49 2024-06-01 02:27:45 0:22:56 0:13:13 0:09:43 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
fail 7736405 2024-05-31 21:10:39 2024-06-01 02:04:49 2024-06-01 02:32:29 0:27:40 0:16:59 0:10:41 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-06-01T02:26:31.769708+0000 mon.smithi007 (mon.0) 841 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7736406 2024-05-31 21:10:40 2024-06-01 02:05:40 2024-06-01 02:37:17 0:31:37 0:19:37 0:12:00 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
pass 7736407 2024-05-31 21:10:41 2024-06-01 02:07:50 2024-06-01 02:32:00 0:24:10 0:11:50 0:12:20 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7736408 2024-05-31 21:10:42 2024-06-01 02:08:31 2024-06-01 02:44:27 0:35:56 0:23:31 0:12:25 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-06-01T02:34:43.468476+0000 mon.a (mon.0) 436 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736409 2024-05-31 21:10:43 2024-06-01 02:11:02 2024-06-01 02:53:32 0:42:30 0:30:43 0:11:47 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
fail 7736410 2024-05-31 21:10:44 2024-06-01 02:12:12 2024-06-01 02:31:13 0:19:01 0:07:28 0:11:33 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/master} 3
Failure Reason:

Command failed on smithi082 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7736411 2024-05-31 21:10:45 2024-06-01 02:13:23 2024-06-01 02:40:32 0:27:09 0:16:43 0:10:26 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-06-01T02:36:19.101857+0000 mon.smithi047 (mon.0) 989 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736412 2024-05-31 21:10:46 2024-06-01 02:13:23 2024-06-01 03:00:56 0:47:33 0:39:10 0:08:23 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
pass 7736413 2024-05-31 21:10:47 2024-06-01 02:13:23 2024-06-01 02:41:53 0:28:30 0:18:43 0:09:47 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
dead 7736414 2024-05-31 21:10:48 2024-06-01 02:14:14 2024-06-01 02:15:18 0:01:04 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi172

pass 7736415 2024-05-31 21:10:49 2024-06-01 02:14:14 2024-06-01 02:38:53 0:24:39 0:13:49 0:10:50 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7736416 2024-05-31 21:10:50 2024-06-01 02:15:45 2024-06-01 02:40:40 0:24:55 0:15:25 0:09:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-06-01T02:36:49.687231+0000 mon.smithi050 (mon.0) 783 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7736417 2024-05-31 21:10:51 2024-06-01 02:15:55 2024-06-01 02:55:11 0:39:16 0:27:34 0:11:42 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-06-01T02:44:06.972713+0000 mon.a (mon.0) 977 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736418 2024-05-31 21:10:52 2024-06-01 02:18:06 2024-06-01 02:46:49 0:28:43 0:15:32 0:13:11 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-06-01T02:38:18.259172+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736419 2024-05-31 21:10:53 2024-06-01 02:19:06 2024-06-01 02:50:41 0:31:35 0:20:26 0:11:09 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-06-01T02:38:31.382797+0000 mon.smithi116 (mon.0) 121 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7736420 2024-05-31 21:10:54 2024-06-01 02:19:47 2024-06-01 02:58:43 0:38:56 0:26:06 0:12:50 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli_mon} 5
pass 7736421 2024-05-31 21:10:55 2024-06-01 02:20:47 2024-06-01 02:41:48 0:21:01 0:11:04 0:09:57 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7736422 2024-05-31 21:10:56 2024-06-01 02:20:48 2024-06-01 02:44:02 0:23:14 0:14:24 0:08:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7736423 2024-05-31 21:10:58 2024-06-01 02:21:08 2024-06-01 02:52:54 0:31:46 0:20:24 0:11:22 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-06-01T02:42:52.834733+0000 mon.a (mon.0) 461 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736424 2024-05-31 21:10:59 2024-06-01 02:21:49 2024-06-01 02:46:48 0:24:59 0:14:34 0:10:25 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-06-01T02:42:36.615814+0000 mon.smithi052 (mon.0) 818 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736425 2024-05-31 21:11:00 2024-06-01 02:21:59 2024-06-01 03:09:17 0:47:18 0:36:53 0:10:25 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
pass 7736426 2024-05-31 21:11:01 2024-06-01 02:22:09 2024-06-01 02:52:24 0:30:15 0:20:22 0:09:53 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
dead 7736427 2024-05-31 21:11:02 2024-06-01 02:22:40 2024-06-01 14:32:17 12:09:37 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

fail 7736428 2024-05-31 21:11:03 2024-06-01 02:22:40 2024-06-01 02:46:50 0:24:10 0:13:09 0:11:01 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-06-01T02:42:57.141427+0000 mon.a (mon.0) 470 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7736429 2024-05-31 21:11:04 2024-06-01 02:23:21 2024-06-01 03:28:14 1:04:53 0:51:48 0:13:05 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-06-01T02:51:48.388273+0000 mon.a (mon.0) 983 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7736430 2024-05-31 21:11:05 2024-06-01 02:25:02 2024-06-01 02:56:37 0:31:35 0:19:46 0:11:49 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_rgw_multisite} 3
pass 7736431 2024-05-31 21:11:06 2024-06-01 02:26:52 2024-06-01 02:52:38 0:25:46 0:15:38 0:10:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
dead 7736432 2024-05-31 21:11:07 2024-06-01 02:27:53 2024-06-01 02:30:27 0:02:34 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi144

pass 7736433 2024-05-31 21:11:08 2024-06-01 02:29:23 2024-06-01 02:50:53 0:21:30 0:12:35 0:08:55 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_adoption} 1
pass 7736434 2024-05-31 21:11:09 2024-06-01 02:29:24 2024-06-01 02:52:40 0:23:16 0:13:23 0:09:53 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7736435 2024-05-31 21:11:10 2024-06-01 02:29:54 2024-06-01 03:00:49 0:30:55 0:19:16 0:11:39 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
pass 7736436 2024-05-31 21:11:11 2024-06-01 02:31:05 2024-06-01 03:00:56 0:29:51 0:19:19 0:10:32 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
fail 7736437 2024-05-31 21:11:12 2024-06-01 02:31:05 2024-06-01 02:58:39 0:27:34 0:16:29 0:11:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-06-01T02:51:43.052369+0000 mon.a (mon.0) 310 : cluster [WRN] [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm" in cluster log

pass 7736438 2024-05-31 21:11:13 2024-06-01 02:31:15 2024-06-01 02:57:03 0:25:48 0:13:58 0:11:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7736439 2024-05-31 21:11:14 2024-06-01 02:31:46 2024-06-01 03:30:17 0:58:31 0:48:30 0:10:01 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
pass 7736440 2024-05-31 21:11:15 2024-06-01 02:32:06 2024-06-01 03:08:35 0:36:29 0:24:47 0:11:42 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7736441 2024-05-31 21:11:16 2024-06-01 02:32:37 2024-06-01 02:59:27 0:26:50 0:15:21 0:11:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7736442 2024-05-31 21:11:17 2024-06-01 02:34:47 2024-06-01 03:24:43 0:49:56 0:37:34 0:12:22 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
pass 7736443 2024-05-31 21:11:18 2024-06-01 02:59:54 702 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_ca_signed_key} 2
pass 7736444 2024-05-31 21:11:19 2024-06-01 02:38:49 2024-06-01 03:13:53 0:35:04 0:23:04 0:12:00 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
fail 7736445 2024-05-31 21:11:20 2024-06-01 02:40:39 2024-06-01 03:12:20 0:31:41 0:21:40 0:10:01 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-06-01T03:07:24.597173+0000 mon.smithi042 (mon.0) 806 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736446 2024-05-31 21:11:20 2024-06-01 02:40:40 2024-06-01 03:10:15 0:29:35 0:18:58 0:10:37 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7736447 2024-05-31 21:11:21 2024-06-01 02:40:50 2024-06-01 03:26:37 0:45:47 0:34:44 0:11:03 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
dead 7736448 2024-05-31 21:11:22 2024-06-01 02:41:50 2024-06-01 02:42:54 0:01:04 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

Error reimaging machines: Failed to power on smithi161

pass 7736449 2024-05-31 21:11:23 2024-06-01 02:41:51 2024-06-01 03:05:01 0:23:10 0:12:17 0:10:53 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7736450 2024-05-31 21:11:24 2024-06-01 02:42:01 2024-06-01 03:05:51 0:23:50 0:11:57 0:11:53 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7736451 2024-05-31 21:11:25 2024-06-01 02:42:22 2024-06-01 03:09:37 0:27:15 0:14:57 0:12:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7736452 2024-05-31 21:11:26 2024-06-01 02:44:12 2024-06-01 03:05:06 0:20:54 0:12:50 0:08:04 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
fail 7736453 2024-05-31 21:11:27 2024-06-01 02:44:13 2024-06-01 03:07:37 0:23:24 0:12:42 0:10:42 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-06-01T03:02:44.339102+0000 mon.a (mon.0) 466 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7736454 2024-05-31 21:11:28 2024-06-01 02:44:33 2024-06-01 03:12:40 0:28:07 0:14:50 0:13:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_cephadm} 1
fail 7736455 2024-05-31 21:11:29 2024-06-01 02:46:54 2024-06-01 03:04:24 0:17:30 0:07:34 0:09:56 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/calico rook/1.7.2} 3
Failure Reason:

Command failed on smithi008 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7736456 2024-05-31 21:11:30 2024-06-01 02:46:54 2024-06-01 03:11:59 0:25:05 0:15:13 0:09:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7736457 2024-05-31 21:11:31 2024-06-01 02:46:54 2024-06-01 03:54:43 1:07:49 0:57:49 0:10:00 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
fail 7736458 2024-05-31 21:11:32 2024-06-01 02:47:45 2024-06-01 03:20:41 0:32:56 0:22:03 0:10:53 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-06-01T03:17:09.782363+0000 mon.smithi120 (mon.0) 954 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7736459 2024-05-31 21:11:33 2024-06-01 02:49:05 2024-06-01 03:09:13 0:20:08 0:09:02 0:11:06 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi157 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6dca6333905c2b38b02396dfd1a76a97d29c2ee7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

pass 7736460 2024-05-31 21:11:34 2024-06-01 02:49:06 2024-06-01 03:15:38 0:26:32 0:13:51 0:12:41 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7736461 2024-05-31 21:11:35 2024-06-01 02:50:16 2024-06-01 03:15:45 0:25:29 0:15:44 0:09:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7736462 2024-05-31 21:11:36 2024-06-01 02:50:47 2024-06-01 03:16:18 0:25:31 0:14:47 0:10:44 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
dead 7736463 2024-05-31 21:11:37 2024-06-01 02:51:27 2024-06-01 02:53:31 0:02:04 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi162

fail 7736464 2024-05-31 21:11:38 2024-06-01 02:52:28 2024-06-01 03:41:41 0:49:13 0:37:58 0:11:15 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7736465 2024-05-31 21:11:39 2024-06-01 02:52:48 2024-06-01 03:17:42 0:24:54 0:15:24 0:09:30 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-06-01T03:15:33.038714+0000 mon.a (mon.0) 560 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

pass 7736466 2024-05-31 21:11:40 2024-06-01 02:52:48 2024-06-01 03:13:34 0:20:46 0:12:42 0:08:04 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
fail 7736467 2024-05-31 21:11:41 2024-06-01 02:52:49 2024-06-01 03:20:04 0:27:15 0:17:17 0:09:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-06-01T03:13:20.283309+0000 mon.smithi028 (mon.0) 848 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7736468 2024-05-31 21:11:42 2024-06-01 02:52:59 2024-06-01 03:16:49 0:23:50 0:14:11 0:09:39 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_extra_daemon_features} 2
Failure Reason:

"2024-06-01T03:12:54.866443+0000 mon.a (mon.0) 297 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

dead 7736469 2024-05-31 21:11:43 2024-06-01 02:53:40 2024-06-01 02:55:14 0:01:34 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi032

fail 7736470 2024-05-31 21:11:44 2024-06-01 02:54:10 2024-06-01 03:47:09 0:52:59 0:42:43 0:10:16 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi106 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 181e83a8-1fc4-11ef-bc9b-c7b262605968 -e sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7736471 2024-05-31 21:11:45 2024-06-01 02:54:10 2024-06-01 03:20:30 0:26:20 0:14:14 0:12:06 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
dead 7736472 2024-05-31 21:11:46 2024-06-01 02:55:21 2024-06-01 02:57:15 0:01:54 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi196

fail 7736473 2024-05-31 21:11:47 2024-06-01 02:56:11 2024-06-01 03:19:51 0:23:40 0:14:45 0:08:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-06-01T03:16:45.770976+0000 mon.smithi153 (mon.0) 840 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7736474 2024-05-31 21:11:48 2024-06-01 02:56:42 2024-06-01 03:20:22 0:23:40 0:11:43 0:11:57 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
dead 7736475 2024-05-31 21:11:49 2024-06-01 02:57:12 2024-06-01 02:59:47 0:02:35 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

Error reimaging machines: Failed to power on smithi083

fail 7736476 2024-05-31 21:11:50 2024-06-01 02:58:43 2024-06-01 03:23:53 0:25:10 0:15:44 0:09:26 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-06-01T03:20:03.647327+0000 mon.smithi089 (mon.0) 786 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7736477 2024-05-31 21:11:51 2024-06-01 02:58:43 2024-06-01 03:29:45 0:31:02 0:21:49 0:09:13 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-06-01T03:19:02.408828+0000 mgr.smithi184.hnmhbn (mgr.14238) 1 : cluster [ERR] Failed to load ceph-mgr modules: k8sevents" in cluster log

pass 7736478 2024-05-31 21:11:52 2024-06-01 02:58:54 2024-06-01 03:51:53 0:52:59 0:40:33 0:12:26 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_orch_cli_mon} 5
pass 7736479 2024-05-31 21:11:53 2024-06-01 02:59:24 2024-06-01 03:26:57 0:27:33 0:18:47 0:08:46 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2