Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7614903 2024-03-21 12:04:37 2024-03-21 12:12:12 2024-03-21 12:43:14 0:31:02 0:22:37 0:08:25 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-03-21T12:40:00.000237+0000 mon.a (mon.0) 1344 : cluster [WRN] Health detail: HEALTH_WARN Reduced data availability: 3 pgs peering; 2 pool(s) do not have an application enabled" in cluster log

fail 7614904 2024-03-21 12:04:38 2024-03-21 12:12:13 2024-03-21 12:39:00 0:26:47 0:15:59 0:10:48 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-03-21T12:32:38.570868+0000 mon.a (mon.0) 449 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614905 2024-03-21 12:04:40 2024-03-21 12:12:13 2024-03-21 12:56:48 0:44:35 0:32:30 0:12:05 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711024896.018443 mon.smithi027 (mon.0) 892 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614906 2024-03-21 12:04:41 2024-03-21 12:12:13 2024-03-21 12:58:47 0:46:34 0:34:42 0:11:52 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-21T12:32:21.542288+0000 mon.smithi022 (mon.0) 495 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7614907 2024-03-21 12:04:42 2024-03-21 12:12:14 2024-03-21 13:10:42 0:58:28 0:46:55 0:11:33 smithi main ubuntu 22.04 orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-03-21T12:38:17.936467+0000 mon.a (mon.0) 320 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

fail 7614908 2024-03-21 12:04:44 2024-03-21 12:12:14 2024-03-21 12:35:09 0:22:55 0:13:14 0:09:41 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
Failure Reason:

"2024-03-21T12:32:31.941472+0000 mon.a (mon.0) 456 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7614909 2024-03-21 12:04:45 2024-03-21 12:12:14 2024-03-21 12:39:45 0:27:31 0:14:02 0:13:29 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
pass 7614910 2024-03-21 12:04:47 2024-03-21 12:12:15 2024-03-21 12:52:03 0:39:48 0:28:50 0:10:58 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
fail 7614911 2024-03-21 12:04:48 2024-03-21 12:12:15 2024-03-21 12:40:02 0:27:47 0:17:07 0:10:40 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-21T12:31:08.040628+0000 mon.smithi081 (mon.0) 582 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614912 2024-03-21 12:04:49 2024-03-21 12:12:15 2024-03-21 12:40:57 0:28:42 0:18:29 0:10:13 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
pass 7614913 2024-03-21 12:04:51 2024-03-21 12:12:16 2024-03-21 12:35:06 0:22:50 0:11:28 0:11:22 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7614914 2024-03-21 12:04:52 2024-03-21 12:12:16 2024-03-21 13:24:42 1:12:26 1:00:31 0:11:55 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-21T12:36:21.922706+0000 mon.a (mon.0) 533 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7614915 2024-03-21 12:04:53 2024-03-21 12:12:16 2024-03-21 13:11:57 0:59:41 0:48:02 0:11:39 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-21T12:31:22.418827+0000 mon.a (mon.0) 477 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7614916 2024-03-21 12:04:55 2024-03-21 12:12:17 2024-03-21 12:47:24 0:35:07 0:23:46 0:11:21 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-21T12:36:19.817603+0000 mon.a (mon.0) 640 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614917 2024-03-21 12:04:56 2024-03-21 12:12:17 2024-03-21 12:36:33 0:24:16 0:13:48 0:10:28 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-03-21T12:33:18.363677+0000 mon.a (mon.0) 294 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614918 2024-03-21 12:04:58 2024-03-21 12:12:18 2024-03-21 12:42:22 0:30:04 0:17:15 0:12:49 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-21T12:35:51.558346+0000 mon.smithi039 (mon.0) 765 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614919 2024-03-21 12:04:59 2024-03-21 12:12:18 2024-03-21 12:41:46 0:29:28 0:19:01 0:10:27 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-21T12:37:20.706089+0000 mon.smithi086 (mon.0) 756 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614920 2024-03-21 12:05:00 2024-03-21 12:12:18 2024-03-21 12:44:41 0:32:23 0:21:43 0:10:40 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-21T12:38:23.942299+0000 mon.smithi097 (mon.0) 752 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614921 2024-03-21 12:05:02 2024-03-21 12:12:19 2024-03-21 12:47:50 0:35:31 0:26:05 0:09:26 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-03-21T12:32:13.741530+0000 mon.a (mon.0) 238 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b" in cluster log

fail 7614922 2024-03-21 12:05:03 2024-03-21 12:12:19 2024-03-21 12:56:21 0:44:02 0:32:53 0:11:09 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-21T12:36:52.770781+0000 mon.a (mon.0) 521 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7614923 2024-03-21 12:05:04 2024-03-21 12:12:19 2024-03-21 13:25:11 1:12:52 1:00:59 0:11:53 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-21T12:34:47.760734+0000 mon.a (mon.0) 175 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7614924 2024-03-21 12:05:06 2024-03-21 12:12:20 2024-03-21 12:49:19 0:36:59 0:25:03 0:11:56 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-03-21T12:35:45.079884+0000 mon.a (mon.0) 259 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7614925 2024-03-21 12:05:07 2024-03-21 12:12:20 2024-03-21 13:36:50 1:24:30 1:08:04 0:16:26 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

dead 7614926 2024-03-21 12:05:08 2024-03-21 12:19:42 2024-03-22 00:37:03 12:17:21 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

hit max job timeout

fail 7614927 2024-03-21 12:05:10 2024-03-21 12:26:43 2024-03-21 13:03:36 0:36:53 0:24:04 0:12:49 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-03-21T12:57:18.889741+0000 mon.a (mon.0) 1147 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

fail 7614928 2024-03-21 12:05:11 2024-03-21 12:26:44 2024-03-21 12:52:54 0:26:10 0:14:27 0:11:43 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-21T12:48:44.771266+0000 mon.smithi023 (mon.0) 768 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614929 2024-03-21 12:05:13 2024-03-21 12:26:44 2024-03-21 12:59:32 0:32:48 0:23:25 0:09:23 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
fail 7614930 2024-03-21 12:05:14 2024-03-21 12:26:45 2024-03-21 12:55:16 0:28:31 0:16:21 0:12:10 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-03-21T12:51:00.709674+0000 mon.a (mon.0) 188 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7614931 2024-03-21 12:05:15 2024-03-21 12:26:45 2024-03-21 12:51:36 0:24:51 0:14:27 0:10:24 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-21T12:47:03.423090+0000 mon.smithi033 (mon.0) 771 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614932 2024-03-21 12:05:17 2024-03-21 12:26:45 2024-03-21 12:56:18 0:29:33 0:20:20 0:09:13 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-21T12:52:13.869111+0000 mon.smithi028 (mon.0) 755 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614933 2024-03-21 12:05:18 2024-03-21 12:26:46 2024-03-21 13:26:50 1:00:04 0:49:12 0:10:52 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-21T13:01:23.488726+0000 mon.a (mon.0) 939 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7614934 2024-03-21 12:05:19 2024-03-21 12:26:46 2024-03-21 13:09:43 0:42:57 0:32:50 0:10:07 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711025688.2639012 mon.smithi042 (mon.0) 902 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614935 2024-03-21 12:05:21 2024-03-21 12:26:46 2024-03-21 12:55:01 0:28:15 0:17:52 0:10:23 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-21T12:47:59.531507+0000 mon.smithi117 (mon.0) 780 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614936 2024-03-21 12:05:22 2024-03-21 12:26:47 2024-03-21 12:50:13 0:23:26 0:12:15 0:11:11 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-21T12:43:58.889903+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7614937 2024-03-21 12:05:24 2024-03-21 12:26:47 2024-03-21 13:20:35 0:53:48 0:41:01 0:12:47 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-03-21T12:46:42.469775+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7614938 2024-03-21 12:05:25 2024-03-21 12:26:48 2024-03-21 13:02:32 0:35:44 0:26:14 0:09:30 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-21T12:51:52.263571+0000 mon.a (mon.0) 643 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614939 2024-03-21 12:05:26 2024-03-21 12:26:48 2024-03-21 12:55:30 0:28:42 0:17:05 0:11:37 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7614940 2024-03-21 12:05:28 2024-03-21 12:26:48 2024-03-21 12:54:13 0:27:25 0:16:55 0:10:30 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-21T12:48:55.328945+0000 mon.smithi029 (mon.0) 790 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614941 2024-03-21 12:05:29 2024-03-21 12:26:49 2024-03-21 12:51:34 0:24:45 0:13:07 0:11:38 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
pass 7614942 2024-03-21 12:05:30 2024-03-21 12:26:49 2024-03-21 12:52:28 0:25:39 0:13:53 0:11:46 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7614943 2024-03-21 12:05:32 2024-03-21 12:26:50 2024-03-21 12:53:12 0:26:22 0:15:17 0:11:05 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-03-21T12:47:09.560762+0000 mon.a (mon.0) 663 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614944 2024-03-21 12:05:33 2024-03-21 12:26:50 2024-03-21 12:57:16 0:30:26 0:20:06 0:10:20 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-21T12:50:34.779297+0000 mon.smithi131 (mon.0) 553 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614945 2024-03-21 12:05:34 2024-03-21 12:26:50 2024-03-21 13:44:01 1:17:11 1:05:41 0:11:30 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-21T13:01:50.481318+0000 mon.a (mon.0) 951 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7614946 2024-03-21 12:05:36 2024-03-21 12:26:51 2024-03-21 13:31:23 1:04:32 0:53:39 0:10:53 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-21T12:54:53.876180+0000 mon.a (mon.0) 1052 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7614947 2024-03-21 12:05:37 2024-03-21 12:27:01 2024-03-21 12:49:06 0:22:05 0:11:18 0:10:47 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7614948 2024-03-21 12:05:38 2024-03-21 12:27:01 2024-03-21 13:31:24 1:04:23 0:52:42 0:11:41 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7614949 2024-03-21 12:05:40 2024-03-21 12:27:02 2024-03-21 12:51:46 0:24:44 0:13:48 0:10:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-21T12:46:32.864817+0000 mon.smithi005 (mon.0) 561 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614950 2024-03-21 12:05:41 2024-03-21 12:27:02 2024-03-21 13:12:47 0:45:45 0:33:00 0:12:45 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
fail 7614951 2024-03-21 12:05:42 2024-03-21 12:27:13 2024-03-21 12:55:01 0:27:48 0:17:14 0:10:34 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7614952 2024-03-21 12:05:44 2024-03-21 12:27:13 2024-03-21 12:55:22 0:28:09 0:14:20 0:13:49 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-21T12:49:33.797038+0000 mon.smithi017 (mon.0) 695 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7614953 2024-03-21 12:05:45 2024-03-21 12:30:14 2024-03-21 13:47:10 1:16:56 1:00:12 0:16:44 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-21T13:00:40.634647+0000 mon.a (mon.0) 845 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7614954 2024-03-21 12:05:46 2024-03-21 12:35:15 2024-03-21 13:10:58 0:35:43 0:22:59 0:12:44 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-21T13:01:12.646964+0000 mon.smithi072 (mon.0) 694 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7614955 2024-03-21 12:05:48 2024-03-21 12:36:16 2024-03-21 13:09:24 0:33:08 0:19:49 0:13:19 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-21T13:06:05.502677+0000 mon.smithi053 (mon.0) 751 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614956 2024-03-21 12:05:49 2024-03-21 12:39:46 2024-03-21 13:16:48 0:37:02 0:25:20 0:11:42 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
Failure Reason:

"2024-03-21T13:05:25.116133+0000 mon.a (mon.0) 638 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614957 2024-03-21 12:05:51 2024-03-21 12:41:07 2024-03-21 13:01:03 0:19:56 0:08:46 0:11:10 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7614958 2024-03-21 12:05:52 2024-03-21 12:42:28 2024-03-21 13:23:39 0:41:11 0:31:20 0:09:51 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711026639.097662 mon.smithi153 (mon.0) 930 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614959 2024-03-21 12:05:53 2024-03-21 12:42:28 2024-03-21 13:10:17 0:27:49 0:16:43 0:11:06 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

"2024-03-21T13:08:06.958430+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7614960 2024-03-21 12:05:55 2024-03-21 12:42:28 2024-03-21 13:03:13 0:20:45 0:11:49 0:08:56 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
pass 7614961 2024-03-21 12:05:56 2024-03-21 12:42:29 2024-03-21 13:07:24 0:24:55 0:14:56 0:09:59 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7614962 2024-03-21 12:05:57 2024-03-21 12:42:39 2024-03-21 13:12:04 0:29:25 0:19:44 0:09:41 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
Failure Reason:

"2024-03-21T13:03:32.793675+0000 mon.smithi120 (mon.0) 318 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614963 2024-03-21 12:05:59 2024-03-21 12:42:39 2024-03-21 13:05:41 0:23:02 0:12:12 0:10:50 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-21T12:59:42.436790+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7614964 2024-03-21 12:06:00 2024-03-21 12:42:40 2024-03-21 13:33:40 0:51:00 0:41:22 0:09:38 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi067 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 409d6aee-e782-11ee-95cd-87774f69a715 -e sha1=960fbbe4645c2becc8084669be194c95183c7753 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

fail 7614965 2024-03-21 12:06:01 2024-03-21 12:42:40 2024-03-21 13:10:29 0:27:49 0:15:19 0:12:30 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-21T13:05:21.885720+0000 mon.smithi039 (mon.0) 780 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614966 2024-03-21 12:06:03 2024-03-21 12:42:40 2024-03-21 13:07:48 0:25:08 0:15:02 0:10:06 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-03-21T13:02:55.234956+0000 mon.a (mon.0) 643 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614967 2024-03-21 12:06:04 2024-03-21 12:42:41 2024-03-21 13:25:52 0:43:11 0:33:01 0:10:10 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-21T13:09:08.438835+0000 mon.a (mon.0) 839 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7614968 2024-03-21 12:06:05 2024-03-21 12:42:41 2024-03-21 13:34:02 0:51:21 0:35:31 0:15:50 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-21T13:14:26.972477+0000 mon.a (mon.0) 898 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614969 2024-03-21 12:06:07 2024-03-21 12:48:52 2024-03-21 13:12:21 0:23:29 0:13:13 0:10:16 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-03-21T13:07:55.880070+0000 mon.a (mon.0) 342 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614970 2024-03-21 12:06:08 2024-03-21 12:49:13 2024-03-21 13:24:08 0:34:55 0:21:33 0:13:22 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-21T13:14:59.449362+0000 mon.smithi037 (mon.0) 548 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614971 2024-03-21 12:06:10 2024-03-21 12:52:13 2024-03-21 13:15:15 0:23:02 0:13:31 0:09:31 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-21T13:12:46.635300+0000 mon.smithi073 (mon.0) 773 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614972 2024-03-21 12:06:11 2024-03-21 12:52:14 2024-03-21 13:41:34 0:49:20 0:38:31 0:10:49 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7614973 2024-03-21 12:06:12 2024-03-21 12:52:34 2024-03-21 13:20:30 0:27:56 0:15:14 0:12:42 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-21T13:15:40.677863+0000 mon.smithi079 (mon.0) 768 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7614974 2024-03-21 12:06:14 2024-03-21 12:55:35 2024-03-21 13:41:19 0:45:44 0:33:27 0:12:17 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
fail 7614975 2024-03-21 12:06:15 2024-03-21 12:57:46 2024-03-21 13:31:33 0:33:47 0:23:32 0:10:15 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-03-21T13:21:21.956986+0000 mon.a (mon.0) 194 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7614976 2024-03-21 12:06:16 2024-03-21 12:57:46 2024-03-21 13:25:47 0:28:01 0:16:35 0:11:26 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
Failure Reason:

"2024-03-21T13:22:42.653394+0000 mon.a (mon.0) 453 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614977 2024-03-21 12:06:18 2024-03-21 12:58:06 2024-03-21 13:26:24 0:28:18 0:16:58 0:11:20 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-21T13:19:50.515582+0000 mon.smithi077 (mon.0) 832 : cluster [WRN] Health check failed: Failed to place 2 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7614978 2024-03-21 12:06:19 2024-03-21 12:58:07 2024-03-21 13:57:38 0:59:31 0:49:39 0:09:52 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-21T13:26:14.148095+0000 mon.a (mon.0) 538 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7614979 2024-03-21 12:06:20 2024-03-21 12:58:07 2024-03-21 13:34:23 0:36:16 0:24:08 0:12:08 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-21T13:22:15.157900+0000 mon.a (mon.0) 645 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614980 2024-03-21 12:06:22 2024-03-21 12:58:08 2024-03-21 13:20:44 0:22:36 0:14:00 0:08:36 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-03-21T13:16:01.101988+0000 mon.a (mon.0) 185 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614981 2024-03-21 12:06:23 2024-03-21 12:58:08 2024-03-21 13:34:02 0:35:54 0:23:19 0:12:35 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-21T13:23:43.614494+0000 mon.smithi066 (mon.0) 555 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614982 2024-03-21 12:06:25 2024-03-21 12:58:08 2024-03-21 13:21:49 0:23:41 0:14:25 0:09:16 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-21T13:19:38.824782+0000 mon.smithi031 (mon.0) 769 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614983 2024-03-21 12:06:26 2024-03-21 12:58:09 2024-03-21 13:39:00 0:40:51 0:31:48 0:09:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1711027525.0186357 mon.smithi033 (mon.0) 917 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7614984 2024-03-21 12:06:27 2024-03-21 13:27:04 895 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-21T13:21:53.207781+0000 mon.smithi023 (mon.0) 777 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614985 2024-03-21 12:06:29 2024-03-21 12:58:09 2024-03-21 13:20:21 0:22:12 0:11:27 0:10:45 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-21T13:17:30.602152+0000 mon.a (mon.0) 451 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614986 2024-03-21 12:06:30 2024-03-21 12:58:10 2024-03-21 13:47:41 0:49:31 0:38:33 0:10:58 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-03-21T13:21:32.642044+0000 mon.a (mon.0) 483 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7614987 2024-03-21 12:06:32 2024-03-21 12:58:10 2024-03-21 13:34:43 0:36:33 0:26:18 0:10:15 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-03-21T13:20:08.063033+0000 mon.a (mon.0) 370 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614988 2024-03-21 12:06:33 2024-03-21 12:58:10 2024-03-21 14:04:56 1:06:46 0:52:17 0:14:29 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 7614989 2024-03-21 12:06:34 2024-03-21 12:58:11 2024-03-21 14:06:38 1:08:27 0:52:17 0:16:10 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-21T13:29:31.728276+0000 mon.a (mon.0) 1076 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614990 2024-03-21 12:06:36 2024-03-21 12:58:11 2024-03-21 13:28:20 0:30:09 0:19:32 0:10:37 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-03-21T13:18:39.821906+0000 mon.a (mon.0) 188 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

fail 7614991 2024-03-21 12:06:37 2024-03-21 12:58:12 2024-03-21 13:26:49 0:28:37 0:15:26 0:13:11 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-21T13:22:08.830025+0000 mon.smithi099 (mon.0) 783 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7614992 2024-03-21 12:06:38 2024-03-21 12:58:12 2024-03-21 13:47:18 0:49:06 0:38:00 0:11:06 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

"2024-03-21T13:24:14.938748+0000 mon.a (mon.0) 225 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,e" in cluster log

fail 7614993 2024-03-21 12:06:40 2024-03-21 12:58:12 2024-03-21 13:29:42 0:31:30 0:20:04 0:11:26 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-21T13:25:57.295114+0000 mon.smithi005 (mon.0) 742 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614994 2024-03-21 12:06:41 2024-03-21 12:58:13 2024-03-21 13:43:00 0:44:47 0:33:56 0:10:51 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-21T13:21:39.845746+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7614995 2024-03-21 12:06:42 2024-03-21 12:58:13 2024-03-21 13:31:43 0:33:30 0:20:50 0:12:40 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

"2024-03-21T13:24:38.183098+0000 mon.a (mon.0) 418 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7614996 2024-03-21 12:06:44 2024-03-21 12:59:34 2024-03-21 13:34:50 0:35:16 0:20:53 0:14:23 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-21T13:29:11.355372+0000 mon.smithi007 (mon.0) 753 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7614997 2024-03-21 12:06:45 2024-03-21 13:03:15 2024-03-21 13:58:21 0:55:06 0:40:20 0:14:46 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7614998 2024-03-21 12:06:46 2024-03-21 13:07:26 2024-03-21 13:38:23 0:30:57 0:14:15 0:16:42 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-21T13:33:56.548225+0000 mon.smithi151 (mon.0) 769 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7614999 2024-03-21 12:06:48 2024-03-21 13:12:57 2024-03-21 14:18:55 1:05:58 0:55:32 0:10:26 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-21T13:38:15.309324+0000 mon.a (mon.0) 849 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7615000 2024-03-21 12:06:49 2024-03-21 13:13:37 2024-03-21 13:40:32 0:26:55 0:17:59 0:08:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-21T13:34:06.781597+0000 mon.smithi050 (mon.0) 796 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7615001 2024-03-21 12:06:51 2024-03-21 13:13:38 2024-03-21 13:46:27 0:32:49 0:23:49 0:09:00 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-03-21T13:35:54.874432+0000 mon.a (mon.0) 646 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7615002 2024-03-21 12:06:52 2024-03-21 13:13:38 2024-03-21 13:38:48 0:25:10 0:15:48 0:09:22 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_set_mon_crush_locations} 3
pass 7615003 2024-03-21 12:06:53 2024-03-21 13:13:38 2024-03-21 13:45:17 0:31:39 0:21:49 0:09:50 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2