Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7707152 2024-05-15 15:01:41 2024-05-15 15:04:17 2024-05-15 15:40:18 0:36:01 0:25:33 0:10:28 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7707153 2024-05-15 15:01:42 2024-05-15 15:04:57 2024-05-15 15:30:45 0:25:48 0:16:16 0:09:32 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
pass 7707154 2024-05-15 15:01:44 2024-05-15 15:05:38 2024-05-15 15:48:42 0:43:04 0:33:15 0:09:49 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
dead 7707155 2024-05-15 15:01:45 2024-05-15 15:05:38 2024-05-15 15:10:51 0:05:13 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Error reimaging machines: Expected smithi053's OS to be centos 9 but found ubuntu 22.04

dead 7707156 2024-05-15 15:01:46 2024-05-15 15:05:38 2024-05-15 15:12:50 0:07:12 smithi main ubuntu 22.04 orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

SSH connection to smithi053 was lost: 'sudo apt-get update'

pass 7707157 2024-05-15 15:01:48 2024-05-15 15:05:39 2024-05-15 15:28:56 0:23:17 0:13:22 0:09:55 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
pass 7707158 2024-05-15 15:01:49 2024-05-15 15:05:39 2024-05-15 15:29:27 0:23:48 0:14:09 0:09:39 smithi main ubuntu 22.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
fail 7707159 2024-05-15 15:01:51 2024-05-15 15:05:39 2024-05-15 15:43:49 0:38:10 0:27:56 0:10:14 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-05-15T15:23:52.535746+0000 mon.a (mon.0) 210 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7707160 2024-05-15 15:01:52 2024-05-15 15:06:10 2024-05-15 15:29:48 0:23:38 0:12:59 0:10:39 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
fail 7707161 2024-05-15 15:01:53 2024-05-15 15:06:30 2024-05-15 15:36:24 0:29:54 0:17:04 0:12:50 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-05-15T15:30:00.000254+0000 mon.smithi045 (mon.0) 866 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 2 daemon(s)" in cluster log

pass 7707162 2024-05-15 15:01:55 2024-05-15 15:09:41 2024-05-15 15:30:39 0:20:58 0:11:49 0:09:09 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
pass 7707163 2024-05-15 15:01:56 2024-05-15 15:09:41 2024-05-15 15:31:04 0:21:23 0:11:27 0:09:56 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7707164 2024-05-15 15:01:58 2024-05-15 15:09:42 2024-05-15 16:21:30 1:11:48 0:59:05 0:12:43 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
fail 7707165 2024-05-15 15:01:59 2024-05-15 15:10:52 2024-05-15 16:09:51 0:58:59 0:48:35 0:10:24 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-05-15T15:48:12.521986+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7707166 2024-05-15 15:02:00 2024-05-15 15:11:03 2024-05-15 15:45:42 0:34:39 0:23:29 0:11:10 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7707167 2024-05-15 15:02:02 2024-05-15 15:12:23 2024-05-15 15:35:36 0:23:13 0:14:04 0:09:09 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7707168 2024-05-15 15:02:03 2024-05-15 15:12:24 2024-05-15 15:40:42 0:28:18 0:17:31 0:10:47 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7707169 2024-05-15 15:02:04 2024-05-15 15:13:14 2024-05-15 15:42:49 0:29:35 0:19:27 0:10:08 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7707170 2024-05-15 15:02:06 2024-05-15 15:13:15 2024-05-15 15:53:38 0:40:23 0:21:32 0:18:51 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7707171 2024-05-15 15:02:07 2024-05-15 15:22:05 2024-05-15 15:57:46 0:35:41 0:25:53 0:09:48 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7707172 2024-05-15 15:02:09 2024-05-15 15:22:06 2024-05-15 16:02:42 0:40:36 0:30:24 0:10:12 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7707173 2024-05-15 15:02:10 2024-05-15 15:23:46 2024-05-15 16:37:06 1:13:20 1:01:49 0:11:31 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7707174 2024-05-15 15:02:11 2024-05-15 15:25:27 2024-05-15 16:04:06 0:38:39 0:26:12 0:12:27 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7707175 2024-05-15 15:02:13 2024-05-15 15:25:27 2024-05-15 16:45:15 1:19:48 1:08:45 0:11:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7707176 2024-05-15 15:02:14 2024-05-15 15:25:28 2024-05-15 15:52:31 0:27:03 0:15:33 0:11:30 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7707177 2024-05-15 15:02:15 2024-05-15 15:26:58 2024-05-15 16:02:29 0:35:31 0:25:48 0:09:43 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
pass 7707178 2024-05-15 15:02:17 2024-05-15 15:27:09 2024-05-15 15:48:21 0:21:12 0:11:31 0:09:41 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7707179 2024-05-15 15:02:18 2024-05-15 15:27:19 2024-05-15 15:53:36 0:26:17 0:14:02 0:12:15 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7707180 2024-05-15 15:02:20 2024-05-15 15:28:10 2024-05-15 16:02:16 0:34:06 0:23:25 0:10:41 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
pass 7707181 2024-05-15 15:02:21 2024-05-15 15:29:00 2024-05-15 15:53:00 0:24:00 0:13:49 0:10:11 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7707182 2024-05-15 15:02:22 2024-05-15 15:29:51 2024-05-15 15:53:08 0:23:17 0:14:39 0:08:38 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7707183 2024-05-15 15:02:24 2024-05-15 15:30:11 2024-05-15 16:02:39 0:32:28 0:20:51 0:11:37 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7707184 2024-05-15 15:02:25 2024-05-15 15:30:41 2024-05-15 16:28:43 0:58:02 0:48:19 0:09:43 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7707185 2024-05-15 15:02:27 2024-05-15 15:30:52 2024-05-15 16:12:21 0:41:29 0:31:52 0:09:37 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7707186 2024-05-15 15:02:28 2024-05-15 15:31:12 2024-05-15 15:58:38 0:27:26 0:17:43 0:09:43 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7707187 2024-05-15 15:02:30 2024-05-15 15:31:13 2024-05-15 15:57:24 0:26:11 0:13:07 0:13:04 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
dead 7707188 2024-05-15 15:02:31 2024-05-15 15:31:23 2024-05-16 05:54:51 14:23:28 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

hit max job timeout

pass 7707189 2024-05-15 15:02:33 2024-05-15 15:32:24 2024-05-15 16:05:25 0:33:01 0:24:22 0:08:39 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7707190 2024-05-15 15:02:34 2024-05-15 15:32:34 2024-05-15 16:00:48 0:28:14 0:16:58 0:11:16 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7707191 2024-05-15 15:02:35 2024-05-15 15:33:45 2024-05-15 16:00:19 0:26:34 0:16:14 0:10:20 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7707192 2024-05-15 15:02:37 2024-05-15 15:35:35 2024-05-15 15:59:37 0:24:02 0:13:11 0:10:51 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
pass 7707193 2024-05-15 15:02:38 2024-05-15 15:35:46 2024-05-15 15:59:29 0:23:43 0:13:19 0:10:24 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7707194 2024-05-15 15:02:39 2024-05-15 15:36:16 2024-05-15 16:04:20 0:28:04 0:19:03 0:09:01 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
pass 7707195 2024-05-15 15:02:41 2024-05-15 15:36:46 2024-05-15 16:04:30 0:27:44 0:15:47 0:11:57 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7707196 2024-05-15 15:02:42 2024-05-15 15:39:17 2024-05-15 16:10:11 0:30:54 0:19:57 0:10:57 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7707197 2024-05-15 15:02:44 2024-05-15 15:40:08 2024-05-15 16:53:36 1:13:28 1:04:30 0:08:58 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7707198 2024-05-15 15:02:45 2024-05-15 15:40:08 2024-05-15 16:43:45 1:03:37 0:53:52 0:09:45 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
pass 7707199 2024-05-15 15:02:46 2024-05-15 15:40:28 2024-05-15 16:01:41 0:21:13 0:11:34 0:09:39 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7707201 2024-05-15 15:02:48 2024-05-15 15:40:49 2024-05-15 17:00:21 1:19:32 1:08:29 0:11:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7707203 2024-05-15 15:02:49 2024-05-15 15:40:49 2024-05-15 16:03:39 0:22:50 0:14:27 0:08:23 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7707204 2024-05-15 15:02:51 2024-05-15 15:41:00 2024-05-15 16:24:51 0:43:51 0:33:17 0:10:34 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
pass 7707205 2024-05-15 15:02:52 2024-05-15 15:41:10 2024-05-15 16:12:33 0:31:23 0:20:32 0:10:51 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
pass 7707206 2024-05-15 15:02:53 2024-05-15 15:42:51 2024-05-15 16:09:21 0:26:30 0:14:54 0:11:36 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7707207 2024-05-15 15:02:55 2024-05-15 15:44:21 2024-05-15 16:47:30 1:03:09 0:53:37 0:09:32 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7707208 2024-05-15 15:02:56 2024-05-15 15:44:22 2024-05-15 16:17:56 0:33:34 0:21:49 0:11:45 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7707209 2024-05-15 15:02:57 2024-05-15 15:45:52 2024-05-15 16:18:01 0:32:09 0:19:58 0:12:11 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7707210 2024-05-15 15:02:59 2024-05-15 15:48:23 2024-05-15 16:21:52 0:33:29 0:24:14 0:09:15 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
pass 7707211 2024-05-15 15:03:00 2024-05-15 15:48:43 2024-05-15 16:10:19 0:21:36 0:06:35 0:15:01 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
pass 7707212 2024-05-15 15:03:02 2024-05-15 15:52:34 2024-05-15 16:34:24 0:41:50 0:32:21 0:09:29 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7707213 2024-05-15 15:03:03 2024-05-15 15:53:05 2024-05-15 16:16:03 0:22:58 0:14:34 0:08:24 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
pass 7707214 2024-05-15 15:03:04 2024-05-15 15:53:05 2024-05-15 16:16:52 0:23:47 0:14:08 0:09:39 smithi main ubuntu 22.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7707215 2024-05-15 15:03:06 2024-05-15 15:53:15 2024-05-15 16:15:24 0:22:09 0:11:16 0:10:53 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7707216 2024-05-15 15:03:07 2024-05-15 15:53:46 2024-05-15 16:19:13 0:25:27 0:14:21 0:11:06 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7707217 2024-05-15 15:03:08 2024-05-15 15:53:46 2024-05-15 16:14:39 0:20:53 0:12:36 0:08:17 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
pass 7707218 2024-05-15 15:03:10 2024-05-15 15:53:47 2024-05-15 16:20:01 0:26:14 0:12:17 0:13:57 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
fail 7707219 2024-05-15 15:03:11 2024-05-15 15:56:27 2024-05-15 16:52:20 0:55:53 0:43:49 0:12:04 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi066 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b95da56a-12d5-11ef-bc98-c7b262605968 -e sha1=de5210d58560a4684fdb5b238ef3e6e0285a1de8 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7707220 2024-05-15 15:03:13 2024-05-15 15:56:28 2024-05-15 16:21:43 0:25:15 0:15:29 0:09:46 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7707221 2024-05-15 15:03:14 2024-05-15 15:56:48 2024-05-15 16:21:37 0:24:49 0:14:47 0:10:02 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7707222 2024-05-15 15:03:15 2024-05-15 15:56:48 2024-05-15 16:37:35 0:40:47 0:31:02 0:09:45 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7707223 2024-05-15 15:03:17 2024-05-15 15:56:49 2024-05-15 16:58:08 1:01:19 0:52:40 0:08:39 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
fail 7707224 2024-05-15 15:03:18 2024-05-15 15:56:49 2024-05-15 16:22:52 0:26:03 0:14:30 0:11:33 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-05-15T16:19:34.889123+0000 mon.a (mon.0) 15 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log

pass 7707225 2024-05-15 15:03:19 2024-05-15 15:57:50 2024-05-15 16:30:42 0:32:52 0:21:29 0:11:23 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7707226 2024-05-15 15:03:21 2024-05-15 15:58:40 2024-05-15 16:22:39 0:23:59 0:14:15 0:09:44 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7707227 2024-05-15 15:03:22 2024-05-15 15:59:31 2024-05-15 16:48:51 0:49:20 0:39:01 0:10:19 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7707228 2024-05-15 15:03:24 2024-05-15 15:59:41 2024-05-15 16:25:15 0:25:34 0:15:48 0:09:46 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7707229 2024-05-15 15:03:25 2024-05-15 16:00:22 2024-05-15 16:45:03 0:44:41 0:33:33 0:11:08 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7707230 2024-05-15 15:03:26 2024-05-15 16:00:52 2024-05-15 16:35:23 0:34:31 0:23:37 0:10:54 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
pass 7707231 2024-05-15 15:03:28 2024-05-15 16:01:42 2024-05-15 16:24:58 0:23:16 0:13:46 0:09:30 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
pass 7707232 2024-05-15 15:03:29 2024-05-15 16:02:23 2024-05-15 16:25:51 0:23:28 0:13:21 0:10:07 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
pass 7707233 2024-05-15 15:03:30 2024-05-15 16:02:33 2024-05-15 16:30:01 0:27:28 0:17:38 0:09:50 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7707234 2024-05-15 15:03:32 2024-05-15 16:02:34 2024-05-15 17:04:33 1:01:59 0:51:04 0:10:55 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
pass 7707235 2024-05-15 15:03:33 2024-05-15 16:02:34 2024-05-15 16:37:54 0:35:20 0:24:34 0:10:46 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7707236 2024-05-15 15:03:35 2024-05-15 16:02:44 2024-05-15 16:25:53 0:23:09 0:14:08 0:09:01 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7707237 2024-05-15 15:03:36 2024-05-15 16:02:45 2024-05-15 16:37:17 0:34:32 0:24:10 0:10:22 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7707238 2024-05-15 15:03:37 2024-05-15 16:03:45 2024-05-15 16:27:24 0:23:39 0:13:45 0:09:54 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7707239 2024-05-15 15:03:39 2024-05-15 16:04:16 2024-05-15 16:48:24 0:44:08 0:33:15 0:10:53 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7707240 2024-05-15 15:03:40 2024-05-15 16:04:16 2024-05-15 16:29:29 0:25:13 0:15:17 0:09:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7707241 2024-05-15 15:03:41 2024-05-15 16:04:27 2024-05-15 16:28:40 0:24:13 0:12:39 0:11:34 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7707242 2024-05-15 15:03:43 2024-05-15 16:05:27 2024-05-15 16:56:49 0:51:22 0:38:06 0:13:16 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
fail 7707243 2024-05-15 15:03:44 2024-05-15 16:09:08 2024-05-15 16:44:48 0:35:40 0:25:48 0:09:52 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-05-15T16:30:00.000174+0000 mon.a (mon.0) 411 : cluster 3 [WRN] CEPHADM_STRAY_DAEMON: 1 stray daemon(s) not managed by cephadm" in cluster log

pass 7707244 2024-05-15 15:03:46 2024-05-15 16:09:08 2024-05-15 16:56:39 0:47:31 0:36:35 0:10:56 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
pass 7707245 2024-05-15 15:03:47 2024-05-15 16:09:29 2024-05-15 17:14:03 1:04:34 0:52:33 0:12:01 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7707246 2024-05-15 15:03:48 2024-05-15 16:10:19 2024-05-15 16:41:07 0:30:48 0:19:48 0:11:00 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
dead 7707247 2024-05-15 15:03:50 2024-05-15 16:12:10 2024-05-15 16:13:14 0:01:04 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi133

pass 7707248 2024-05-15 15:03:51 2024-05-15 16:12:10 2024-05-15 17:00:18 0:48:08 0:37:11 0:10:57 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7707249 2024-05-15 15:03:52 2024-05-15 16:12:31 2024-05-15 16:40:43 0:28:12 0:18:27 0:09:45 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7707250 2024-05-15 15:03:54 2024-05-15 16:13:21 2024-05-15 16:44:21 0:31:00 0:19:44 0:11:16 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7707251 2024-05-15 15:03:55 2024-05-15 16:16:11 2024-05-15 17:00:42 0:44:31 0:34:05 0:10:26 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7707252 2024-05-15 15:03:56 2024-05-15 16:17:01 2024-05-15 16:48:15 0:31:14 0:19:49 0:11:25 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7707253 2024-05-15 15:03:58 2024-05-15 16:18:02 2024-05-15 16:49:26 0:31:24 0:20:49 0:10:35 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
fail 7707254 2024-05-15 15:03:59 2024-05-15 16:19:23 2024-05-15 17:08:58 0:49:35 0:39:04 0:10:31 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7707255 2024-05-15 15:04:01 2024-05-15 16:20:03 2024-05-15 17:03:50 0:43:47 0:32:51 0:10:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

pass 7707256 2024-05-15 15:04:02 2024-05-15 16:20:03 2024-05-15 17:30:02 1:09:59 0:56:51 0:13:08 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
pass 7707257 2024-05-15 15:04:03 2024-05-15 16:21:34 2024-05-15 16:50:47 0:29:13 0:18:28 0:10:45 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7707258 2024-05-15 15:04:05 2024-05-15 16:21:44 2024-05-15 16:54:40 0:32:56 0:23:41 0:09:15 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7707259 2024-05-15 15:04:06 2024-05-15 16:21:45 2024-05-15 16:48:03 0:26:18 0:15:46 0:10:32 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_set_mon_crush_locations} 3
pass 7707260 2024-05-15 15:04:07 2024-05-15 16:22:45 2024-05-15 16:56:26 0:33:41 0:20:58 0:12:43 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2