Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7668322 2024-04-22 14:50:57 2024-04-22 14:52:10 2024-04-22 15:23:56 0:31:46 0:22:22 0:09:24 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
fail 7668323 2024-04-22 14:50:58 2024-04-22 14:52:10 2024-04-22 15:20:07 0:27:57 0:17:37 0:10:20 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-22T15:13:51.223434+0000 mon.a (mon.0) 445 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.1 on smithi027 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7668324 2024-04-22 14:50:59 2024-04-22 14:52:11 2024-04-22 15:32:48 0:40:37 0:32:56 0:07:41 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7668325 2024-04-22 14:51:00 2024-04-22 14:52:11 2024-04-22 15:34:40 0:42:29 0:34:22 0:08:07 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
fail 7668326 2024-04-22 14:51:01 2024-04-22 14:52:12 2024-04-22 15:52:44 1:00:32 0:47:21 0:13:11 smithi main ubuntu 22.04 orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-04-22T15:21:17.154631+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

pass 7668327 2024-04-22 14:51:02 2024-04-22 14:52:12 2024-04-22 15:13:41 0:21:29 0:14:05 0:07:24 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
pass 7668328 2024-04-22 14:51:03 2024-04-22 14:52:12 2024-04-22 15:17:55 0:25:43 0:14:57 0:10:46 smithi main ubuntu 22.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7668329 2024-04-22 14:51:04 2024-04-22 14:52:12 2024-04-22 15:24:34 0:32:22 0:19:34 0:12:48 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7668330 2024-04-22 14:51:05 2024-04-22 14:52:13 2024-04-22 15:28:15 0:36:02 0:27:48 0:08:14 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-22T15:07:12.738053+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7668331 2024-04-22 14:51:06 2024-04-22 14:52:13 2024-04-22 15:15:32 0:23:19 0:13:59 0:09:20 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
fail 7668332 2024-04-22 14:51:07 2024-04-22 14:52:14 2024-04-22 15:14:54 0:22:40 0:16:48 0:05:52 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-22T15:09:09.916980+0000 mon.smithi016 (mon.0) 871 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) ["Failed while placing nfs.foo.0.0.smithi016.hdwmkc on smithi016: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n", "Failed while placing nfs.foo.1.0.smithi031.dtijjh on smithi031: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7668333 2024-04-22 14:51:08 2024-04-22 14:52:14 2024-04-22 15:13:50 0:21:36 0:11:39 0:09:57 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
pass 7668334 2024-04-22 14:51:09 2024-04-22 14:52:14 2024-04-22 15:13:39 0:21:25 0:11:35 0:09:50 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7668335 2024-04-22 14:51:10 2024-04-22 14:52:15 2024-04-22 15:48:58 0:56:43 0:48:45 0:07:58 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-22T15:21:53.409400+0000 mon.a (mon.0) 1023 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon iscsi.foo.smithi145.ljpvqb on smithi145 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7668336 2024-04-22 14:51:11 2024-04-22 14:52:15 2024-04-22 15:39:57 0:47:42 0:38:24 0:09:18 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
pass 7668337 2024-04-22 14:51:13 2024-04-22 14:52:15 2024-04-22 15:28:37 0:36:22 0:26:18 0:10:04 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7668338 2024-04-22 14:51:14 2024-04-22 14:52:16 2024-04-22 15:16:01 0:23:45 0:14:57 0:08:48 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7668339 2024-04-22 14:51:15 2024-04-22 14:52:16 2024-04-22 15:18:49 0:26:33 0:17:35 0:08:58 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7668340 2024-04-22 14:51:16 2024-04-22 14:52:16 2024-04-22 15:27:19 0:35:03 0:23:30 0:11:33 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-22T15:21:34.980761+0000 mon.smithi117 (mon.0) 835 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) ["Failed while placing nfs.foo.0.0.smithi117.dbvyja on smithi117: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7668341 2024-04-22 14:51:17 2024-04-22 14:52:17 2024-04-22 15:12:36 0:20:19 0:13:00 0:07:19 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7668342 2024-04-22 14:51:18 2024-04-22 14:52:17 2024-04-22 15:30:39 0:38:22 0:28:48 0:09:34 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-22T15:16:32.941456+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon osd.0 on host smithi067 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7668343 2024-04-22 14:51:19 2024-04-22 14:52:18 2024-04-22 15:30:06 0:37:48 0:25:37 0:12:11 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-22T15:18:53.024606+0000 mon.a (mon.0) 465 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon osd.0 on host smithi033 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7668344 2024-04-22 14:51:20 2024-04-22 14:52:18 2024-04-22 16:11:28 1:19:10 1:09:17 0:09:53 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7668345 2024-04-22 14:51:21 2024-04-22 14:52:18 2024-04-22 15:21:02 0:28:44 0:17:19 0:11:25 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-22T15:15:35.919940+0000 mon.smithi086 (mon.0) 791 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) ["Failed while placing nfs.foo.0.0.smithi086.keogna on smithi086: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7668346 2024-04-22 14:51:22 2024-04-22 14:52:19 2024-04-22 15:28:57 0:36:38 0:26:05 0:10:33 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
pass 7668347 2024-04-22 14:51:23 2024-04-22 14:52:19 2024-04-22 15:12:37 0:20:18 0:11:25 0:08:53 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7668348 2024-04-22 14:51:24 2024-04-22 14:52:30 2024-04-22 15:18:18 0:25:48 0:17:03 0:08:45 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7668349 2024-04-22 14:51:25 2024-04-22 14:52:30 2024-04-22 15:59:17 1:06:47 0:53:32 0:13:15 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
pass 7668350 2024-04-22 14:51:26 2024-04-22 14:52:30 2024-04-22 15:46:05 0:53:35 0:39:29 0:14:06 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7668351 2024-04-22 14:51:27 2024-04-22 14:52:31 2024-04-22 15:16:50 0:24:19 0:15:06 0:09:13 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7668352 2024-04-22 14:51:28 2024-04-22 14:52:31 2024-04-22 15:27:00 0:34:29 0:20:34 0:13:55 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7668353 2024-04-22 14:51:29 2024-04-22 14:52:31 2024-04-22 15:18:45 0:26:14 0:16:18 0:09:56 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7668354 2024-04-22 14:51:30 2024-04-22 14:52:32 2024-04-22 15:36:54 0:44:22 0:34:45 0:09:37 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7668355 2024-04-22 14:51:31 2024-04-22 14:52:32 2024-04-22 15:19:16 0:26:44 0:17:46 0:08:58 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7668356 2024-04-22 14:51:32 2024-04-22 14:52:32 2024-04-22 15:16:16 0:23:44 0:14:20 0:09:24 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-22T15:11:19.229071+0000 mon.a (mon.0) 393 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi084 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7668357 2024-04-22 14:51:33 2024-04-22 14:52:33 2024-04-22 15:44:02 0:51:29 0:39:18 0:12:11 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-04-22T15:15:02.320346+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7668358 2024-04-22 14:51:34 2024-04-22 14:52:33 2024-04-22 15:17:31 0:24:58 0:17:00 0:07:58 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
Failure Reason:

"2024-04-22T15:11:39.053571+0000 mon.a (mon.0) 385 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi132 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7668359 2024-04-22 14:51:35 2024-04-22 14:52:34 2024-04-22 15:20:51 0:28:17 0:17:34 0:10:43 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7668360 2024-04-22 14:51:36 2024-04-22 14:52:34 2024-04-22 15:17:38 0:25:04 0:13:17 0:11:47 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
dead 7668361 2024-04-22 14:51:37 2024-04-22 14:52:34 2024-04-23 03:04:37 12:12:03 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/deploy-raw} 2
Failure Reason:

hit max job timeout

pass 7668362 2024-04-22 14:51:38 2024-04-22 14:52:35 2024-04-22 15:23:18 0:30:43 0:20:23 0:10:20 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
fail 7668363 2024-04-22 14:51:39 2024-04-22 14:52:35 2024-04-22 15:18:30 0:25:55 0:16:21 0:09:34 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-22T15:10:13.209873+0000 mon.a (mon.0) 380 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon osd.0 on host smithi106 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7668364 2024-04-22 14:51:40 2024-04-22 14:52:35 2024-04-22 15:24:05 0:31:30 0:19:05 0:12:25 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7668365 2024-04-22 14:51:41 2024-04-22 14:54:46 2024-04-22 15:57:23 1:02:37 0:54:25 0:08:12 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7668366 2024-04-22 14:51:42 2024-04-22 14:54:46 2024-04-22 15:49:58 0:55:12 0:44:36 0:10:36 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
pass 7668367 2024-04-22 14:51:43 2024-04-22 14:54:47 2024-04-22 15:17:26 0:22:39 0:13:37 0:09:02 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7668368 2024-04-22 14:51:44 2024-04-22 14:54:47 2024-04-22 16:11:40 1:16:53 1:07:33 0:09:20 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7668369 2024-04-22 14:51:45 2024-04-22 14:54:48 2024-04-22 15:31:49 0:37:01 0:12:43 0:24:18 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7668370 2024-04-22 14:51:46 2024-04-22 15:12:40 2024-04-22 15:42:09 0:29:29 0:20:01 0:09:28 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
pass 7668371 2024-04-22 14:51:48 2024-04-22 15:12:41 2024-04-22 15:32:38 0:19:57 0:13:16 0:06:41 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7668372 2024-04-22 14:51:49 2024-04-22 15:13:41 2024-04-22 15:41:35 0:27:54 0:18:55 0:08:59 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7668373 2024-04-22 14:51:50 2024-04-22 15:13:42 2024-04-22 15:45:31 0:31:49 0:22:05 0:09:44 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-22T15:40:06.047544+0000 mon.smithi092 (mon.0) 779 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) ['daemon jaeger-agent.smithi092 on smithi092 is in unknown state', 'daemon jaeger-agent.smithi105 on smithi105 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7668374 2024-04-22 14:51:51 2024-04-22 15:13:52 2024-04-22 15:52:35 0:38:43 0:30:44 0:07:59 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7668375 2024-04-22 14:51:52 2024-04-22 15:15:32 2024-04-22 15:46:08 0:30:36 0:22:07 0:08:29 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} 2
pass 7668376 2024-04-22 14:51:53 2024-04-22 15:16:53 2024-04-22 15:29:43 0:12:50 0:05:36 0:07:14 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
pass 7668377 2024-04-22 14:51:54 2024-04-22 15:16:53 2024-04-22 15:56:33 0:39:40 0:32:12 0:07:28 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7668378 2024-04-22 14:51:55 2024-04-22 15:17:34 2024-04-22 15:36:36 0:19:02 0:13:11 0:05:51 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-22T15:35:26.155668+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7668379 2024-04-22 14:51:56 2024-04-22 15:17:34 2024-04-22 15:35:16 0:17:42 0:11:05 0:06:37 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
pass 7668380 2024-04-22 14:51:57 2024-04-22 15:18:05 2024-04-22 15:35:08 0:17:03 0:10:42 0:06:21 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7668381 2024-04-22 14:51:58 2024-04-22 15:18:25 2024-04-22 15:37:42 0:19:17 0:13:20 0:05:57 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7668382 2024-04-22 14:51:59 2024-04-22 15:18:45 2024-04-22 15:48:24 0:29:39 0:19:33 0:10:06 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
fail 7668383 2024-04-22 14:52:00 2024-04-22 15:18:46 2024-04-22 15:39:21 0:20:35 0:11:39 0:08:56 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-22T15:34:12.906343+0000 mon.a (mon.0) 452 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.1 on smithi130 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7668384 2024-04-22 14:52:01 2024-04-22 15:19:26 2024-04-22 16:11:53 0:52:27 0:43:28 0:08:59 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi131 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f0fe460-00bd-11ef-bc93-c7b262605968 -e sha1=5883d494d3973b30b3f03e4a3f0e1e06e3c131aa -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7668385 2024-04-22 14:52:02 2024-04-22 15:20:57 2024-04-22 15:45:09 0:24:12 0:14:59 0:09:13 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7668386 2024-04-22 14:52:03 2024-04-22 15:23:27 2024-04-22 15:44:51 0:21:24 0:13:17 0:08:07 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7668387 2024-04-22 14:52:04 2024-04-22 15:23:58 2024-04-22 15:45:14 0:21:16 0:14:15 0:07:01 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7668388 2024-04-22 14:52:05 2024-04-22 15:24:08 2024-04-22 15:43:54 0:19:46 0:12:56 0:06:50 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-04-22T15:39:58.074824+0000 mon.a (mon.0) 345 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.1 on smithi156 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7668389 2024-04-22 14:52:06 2024-04-22 15:24:39 2024-04-22 15:59:49 0:35:10 0:23:43 0:11:27 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7668390 2024-04-22 14:52:07 2024-04-22 15:25:29 2024-04-22 16:11:47 0:46:18 0:37:12 0:09:06 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7668391 2024-04-22 14:52:08 2024-04-22 15:25:30 2024-04-22 15:46:24 0:20:54 0:14:54 0:06:00 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7668392 2024-04-22 14:52:09 2024-04-22 15:25:30 2024-04-22 16:25:32 1:00:02 0:48:35 0:11:27 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7668393 2024-04-22 14:52:10 2024-04-22 15:54:26 1371 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rotate-keys} 2
fail 7668394 2024-04-22 14:52:11 2024-04-22 15:59:09 1398 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-04-22T15:56:54.807122+0000 mon.a (mon.0) 743 : cluster [WRN] Health check failed: 1 stray host(s) with ["stray host smithi187 has 1 stray daemons: ['mon.b']"] daemon(s) not managed by cephadm (CEPHADM_STRAY_HOST)" in cluster log

pass 7668395 2024-04-22 14:52:12 2024-04-22 15:25:31 2024-04-22 15:44:16 0:18:45 0:12:21 0:06:24 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
pass 7668396 2024-04-22 14:52:13 2024-04-22 15:25:31 2024-04-22 15:44:40 0:19:09 0:13:04 0:06:05 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7668397 2024-04-22 14:52:14 2024-04-22 15:25:32 2024-04-22 15:44:32 0:19:00 0:13:11 0:05:49 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
fail 7668398 2024-04-22 14:52:15 2024-04-22 15:25:32 2024-04-22 15:51:16 0:25:44 0:16:36 0:09:08 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-22T15:45:18.599219+0000 mon.smithi043 (mon.0) 874 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) ["Failed while placing nfs.foo.0.0.smithi043.nrbomt on smithi043: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\n", "Failed while placing nfs.foo.1.0.smithi072.cfcoxt on smithi072: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

dead 7668399 2024-04-22 14:52:16 2024-04-23 03:37:16 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

hit max job timeout

pass 7668400 2024-04-22 14:52:17 2024-04-22 16:02:41 1449 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7668401 2024-04-22 14:52:18 2024-04-22 15:29:04 2024-04-22 16:06:07 0:37:03 0:30:18 0:06:45 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7668402 2024-04-22 14:52:20 2024-04-22 15:29:04 2024-04-22 15:50:07 0:21:03 0:13:59 0:07:04 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-22T15:45:46.721947+0000 mon.smithi053 (mon.0) 861 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) ["Failed while placing nfs.foo.0.0.smithi053.gqpczy on smithi053: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7668403 2024-04-22 14:52:21 2024-04-22 15:29:05 2024-04-22 15:48:51 0:19:46 0:10:18 0:09:28 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7668404 2024-04-22 14:52:22 2024-04-22 15:31:55 2024-04-22 16:20:10 0:48:15 0:38:09 0:10:06 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
pass 7668405 2024-04-22 14:52:23 2024-04-22 15:32:46 2024-04-22 16:00:26 0:27:40 0:18:43 0:08:57 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7668406 2024-04-22 14:52:24 2024-04-22 15:32:56 2024-04-22 16:10:36 0:37:40 0:25:50 0:11:50 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-22T15:55:50.221581+0000 mon.a (mon.0) 392 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon osd.0 on host smithi119 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7668407 2024-04-22 14:52:25 2024-04-22 15:34:47 2024-04-22 16:35:09 1:00:22 0:49:44 0:10:38 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-22T16:12:03.895678+0000 mon.a (mon.0) 1344 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon laundry.close-test-pid74955 on host smithi062 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7668408 2024-04-22 14:52:26 2024-04-22 15:35:17 2024-04-22 16:28:58 0:53:41 0:44:39 0:09:02 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7668409 2024-04-22 14:52:27 2024-04-22 15:35:18 2024-04-22 16:02:51 0:27:33 0:17:44 0:09:49 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7668410 2024-04-22 14:52:28 2024-04-22 15:37:48 2024-04-22 16:01:16 0:23:28 0:14:19 0:09:09 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-22T15:56:43.281458+0000 mon.smithi018 (mon.0) 790 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) ["Failed while placing nfs.foo.0.0.smithi018.ebzcvb on smithi018: grace tool failed: rados_pool_create: -1\nCan't connect to cluster: -1\nterminate called after throwing an instance of 'std::bad_variant_access'\n what(): std::get: wrong index for variant\n"] (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

pass 7668411 2024-04-22 14:52:29 2024-04-22 15:39:59 2024-04-22 16:29:32 0:49:33 0:36:57 0:12:36 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7668412 2024-04-22 14:52:30 2024-04-22 15:41:10 2024-04-22 16:08:44 0:27:34 0:18:32 0:09:02 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7668413 2024-04-22 14:52:31 2024-04-22 15:41:10 2024-04-22 16:11:01 0:29:51 0:20:02 0:09:49 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7668414 2024-04-22 14:52:32 2024-04-22 15:41:10 2024-04-22 16:10:38 0:29:28 0:19:48 0:09:40 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

"2024-04-22T16:04:08.177498+0000 mon.a (mon.0) 455 : cluster [WRN] Health check failed: 1 stray daemon(s) ['stray daemon osd.0 on host smithi033 not managed by cephadm'] not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7668415 2024-04-22 14:52:33 2024-04-22 16:26:13 2253 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7668416 2024-04-22 14:52:34 2024-04-22 15:41:11 2024-04-22 16:00:05 0:18:54 0:13:14 0:05:40 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7668417 2024-04-22 14:52:35 2024-04-22 15:41:12 2024-04-22 16:02:03 0:20:51 0:13:41 0:07:10 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7668418 2024-04-22 14:52:36 2024-04-22 15:41:22 2024-04-22 16:06:51 0:25:29 0:17:21 0:08:08 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7668419 2024-04-22 14:52:37 2024-04-22 15:41:42 2024-04-22 16:47:17 1:05:35 0:59:15 0:06:20 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7668420 2024-04-22 14:52:38 2024-04-22 15:42:13 2024-04-22 16:13:36 0:31:23 0:22:18 0:09:05 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2