Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7645979 2024-04-08 20:10:08 2024-04-14 13:08:19 2024-04-14 13:37:59 0:29:40 0:19:09 0:10:31 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T13:32:35.688421+0000 mon.a (mon.0) 249 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645980 2024-04-08 20:10:09 2024-04-14 13:08:20 2024-04-14 13:38:23 0:30:03 0:19:39 0:10:24 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-14T13:34:26.953753+0000 mon.smithi171 (mon.0) 744 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645981 2024-04-08 20:10:10 2024-04-12 07:34:53 2024-04-12 08:05:42 0:30:49 0:20:30 0:10:19 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-12T08:01:30.616023+0000 mon.smithi097 (mon.0) 766 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7645982 2024-04-08 20:10:11 2024-04-14 13:08:20 2024-04-14 13:35:25 0:27:05 0:14:14 0:12:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-04-14T13:31:00.253037+0000 mon.smithi106 (mon.0) 774 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645983 2024-04-08 20:10:12 2024-04-14 13:10:21 2024-04-14 13:44:46 0:34:25 0:24:14 0:10:11 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

"2024-04-14T13:34:02.317048+0000 mon.a (mon.0) 663 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645984 2024-04-08 20:10:13 2024-04-14 13:11:31 2024-04-14 13:33:44 0:22:13 0:11:20 0:10:53 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi063 with status 5: 'sudo systemctl stop ceph-165e093e-fa63-11ee-bc8e-c7b262605968@mon.a'

fail 7645985 2024-04-08 20:10:13 2024-04-14 13:12:02 2024-04-14 13:41:24 0:29:22 0:18:54 0:10:28 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-04-14T13:33:55.797123+0000 mon.smithi055 (mon.0) 798 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645986 2024-04-08 20:10:14 2024-04-09 15:54:45 2024-04-09 16:30:02 0:35:17 0:25:10 0:10:07 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-09T16:17:58.581213+0000 mon.a (mon.0) 613 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7645987 2024-04-08 20:10:15 2024-04-14 13:12:02 2024-04-14 13:58:04 0:46:02 0:35:47 0:10:15 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-14T13:37:48.037801+0000 mon.a (mon.0) 848 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7645988 2024-04-08 20:10:16 2024-04-14 13:12:43 2024-04-14 13:45:27 0:32:44 0:21:32 0:11:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-04-14T13:39:52.008933+0000 mon.smithi064 (mon.0) 777 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7645989 2024-04-08 20:10:17 2024-04-14 13:13:43 2024-04-14 13:36:27 0:22:44 0:13:25 0:09:19 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi017 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7645990 2024-04-08 20:10:18 2024-04-14 13:13:43 2024-04-14 13:59:06 0:45:23 0:34:10 0:11:13 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-04-14T13:50:00.000138+0000 mon.smithi067 (mon.0) 507 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid" in cluster log

fail 7645991 2024-04-08 20:10:19 2024-04-14 13:14:04 2024-04-14 13:58:50 0:44:46 0:34:52 0:09:54 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-14T13:39:35.441183+0000 mon.smithi008 (mon.0) 863 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7645992 2024-04-08 20:10:20 2024-04-14 13:14:04 2024-04-14 14:14:53 1:00:49 0:48:53 0:11:56 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

"2024-04-14T13:42:23.345702+0000 mon.a (mon.0) 319 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

pass 7645993 2024-04-08 20:10:21 2024-04-14 13:14:05 2024-04-14 13:35:12 0:21:07 0:10:10 0:10:57 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_adoption} 1
pass 7645994 2024-04-08 20:10:22 2024-04-14 13:14:25 2024-04-14 13:37:16 0:22:51 0:12:19 0:10:32 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
fail 7645995 2024-04-08 20:10:23 2024-04-14 13:14:25 2024-04-14 13:37:20 0:22:55 0:13:03 0:09:52 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-14T13:34:14.983778+0000 mon.smithi073 (mon.0) 798 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7645996 2024-04-08 20:10:23 2024-04-14 13:14:26 2024-04-14 13:57:06 0:42:40 0:27:02 0:15:38 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-14T13:37:46.376919+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7645997 2024-04-08 20:10:24 2024-04-14 13:19:17 2024-04-14 13:43:19 0:24:02 0:13:41 0:10:21 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
Failure Reason:

"2024-04-14T13:40:25.357669+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7645998 2024-04-08 20:10:25 2024-04-14 13:19:37 2024-04-14 13:51:12 0:31:35 0:13:13 0:18:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7645999 2024-04-08 20:10:26 2024-04-14 13:28:19 2024-04-14 13:49:36 0:21:17 0:11:13 0:10:04 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
dead 7646000 2024-04-08 20:10:27 2024-04-14 13:28:19 2024-04-15 01:40:31 12:12:12 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

hit max job timeout

fail 7646001 2024-04-08 20:10:28 2024-04-14 13:32:10 2024-04-14 14:34:17 1:02:07 0:47:18 0:14:49 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-14T14:04:54.322417+0000 mon.a (mon.0) 948 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646002 2024-04-08 20:10:29 2024-04-14 13:46:20 2024-04-14 14:09:03 0:22:43 0:14:08 0:08:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-04-14T14:04:02.147193+0000 mon.smithi032 (mon.0) 580 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646003 2024-04-08 20:10:30 2024-04-14 13:46:21 2024-04-14 14:31:27 0:45:06 0:35:52 0:09:14 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-14T14:20:00.000158+0000 mon.a (mon.0) 2832 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log

fail 7646004 2024-04-08 20:10:31 2024-04-14 13:46:21 2024-04-14 14:04:00 0:17:39 0:07:40 0:09:59 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} 3
Failure Reason:

Command failed on smithi047 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646005 2024-04-08 20:10:32 2024-04-14 13:46:22 2024-04-14 14:01:08 0:14:46 0:06:28 0:08:18 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_cephadm_repos} 1
fail 7646006 2024-04-08 20:10:32 2024-04-14 13:46:22 2024-04-14 14:16:38 0:30:16 0:19:45 0:10:31 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-14T14:12:35.016235+0000 mon.smithi043 (mon.0) 744 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646007 2024-04-08 20:10:33 2024-04-14 13:46:42 2024-04-14 14:17:00 0:30:18 0:15:44 0:14:34 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-14T14:12:41.511876+0000 mon.smithi081 (mon.0) 821 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646008 2024-04-08 20:10:34 2024-04-14 13:49:43 2024-04-14 14:16:07 0:26:24 0:14:36 0:11:48 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-14T14:11:55.997515+0000 mon.smithi045 (mon.0) 780 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646009 2024-04-08 20:10:35 2024-04-14 13:51:14 2024-04-14 15:14:41 1:23:27 1:04:14 0:19:13 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-14T14:23:21.734597+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7646010 2024-04-08 20:10:36 2024-04-14 14:00:46 2024-04-14 14:31:00 0:30:14 0:19:34 0:10:40 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_extra_daemon_features} 2
Failure Reason:

"2024-04-14T14:24:14.219027+0000 mon.a (mon.0) 345 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7646011 2024-04-08 20:10:37 2024-04-14 14:00:46 2024-04-14 14:47:46 0:47:00 0:37:42 0:09:18 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646012 2024-04-08 20:10:38 2024-04-14 14:00:46 2024-04-14 14:23:34 0:22:48 0:14:18 0:08:30 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-14T14:22:24.920130+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7646013 2024-04-08 20:10:39 2024-04-14 14:00:47 2024-04-14 14:21:31 0:20:44 0:11:33 0:09:11 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T14:18:14.673889+0000 mon.a (mon.0) 242 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7646014 2024-04-08 20:10:40 2024-04-14 14:00:47 2024-04-14 14:23:32 0:22:45 0:14:47 0:07:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7646015 2024-04-08 20:10:41 2024-04-14 14:00:47 2024-04-14 14:40:50 0:40:03 0:15:11 0:24:52 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-14T14:32:15.948508+0000 mon.a (mon.0) 417 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7646016 2024-04-08 20:10:42 2024-04-14 14:15:50 2024-04-14 14:47:17 0:31:27 0:20:48 0:10:39 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-04-14T14:40:55.093692+0000 mon.smithi017 (mon.0) 743 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646017 2024-04-08 20:10:43 2024-04-14 14:15:50 2024-04-14 14:41:00 0:25:10 0:15:17 0:09:53 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-14T14:38:29.996395+0000 mon.a (mon.0) 618 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646018 2024-04-08 20:10:44 2024-04-14 14:15:51 2024-04-14 14:41:08 0:25:17 0:15:37 0:09:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-14T14:37:06.088451+0000 mon.smithi045 (mon.0) 801 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646019 2024-04-08 20:10:45 2024-04-14 14:16:11 2024-04-14 14:39:22 0:23:11 0:11:52 0:11:19 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-14T14:36:38.754769+0000 mon.a (mon.0) 750 : cluster [WRN] Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN)" in cluster log

fail 7646020 2024-04-08 20:10:45 2024-04-14 14:16:12 2024-04-14 14:45:53 0:29:41 0:19:12 0:10:29 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-14T14:42:06.837014+0000 mon.smithi186 (mon.0) 763 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646021 2024-04-08 20:10:46 2024-04-14 14:16:12 2024-04-14 15:00:50 0:44:38 0:34:09 0:10:29 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-04-14T14:39:48.550852+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7646022 2024-04-08 20:10:47 2024-04-14 14:16:12 2024-04-14 15:23:50 1:07:38 0:56:58 0:10:40 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-14T14:35:05.698691+0000 mon.a (mon.0) 173 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7646023 2024-04-08 20:10:48 2024-04-14 14:16:13 2024-04-14 14:44:46 0:28:33 0:15:16 0:13:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-14T14:40:22.094053+0000 mon.smithi057 (mon.0) 774 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646024 2024-04-08 20:10:49 2024-04-14 14:19:34 2024-04-14 15:01:24 0:41:50 0:30:33 0:11:17 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-14T14:42:17.671976+0000 mon.a (mon.0) 653 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7646025 2024-04-08 20:10:50 2024-04-14 14:20:14 2024-04-14 14:45:27 0:25:13 0:14:17 0:10:56 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-04-14T14:40:19.850860+0000 mon.a (mon.0) 261 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7646026 2024-04-08 20:10:51 2024-04-14 14:20:14 2024-04-14 15:05:15 0:45:01 0:33:36 0:11:25 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7646027 2024-04-08 20:10:52 2024-04-14 14:22:05 2024-04-14 14:51:55 0:29:50 0:19:23 0:10:27 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-14T14:47:24.694971+0000 mon.a (mon.0) 323 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

fail 7646028 2024-04-08 20:10:53 2024-04-14 14:22:06 2024-04-14 14:53:15 0:31:09 0:19:05 0:12:04 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
Failure Reason:

"2024-04-14T14:48:23.925726+0000 mon.a (mon.0) 290 : cluster [WRN] Health check failed: Degraded data redundancy: 22/66 objects degraded (33.333%), 10 pgs degraded (PG_DEGRADED)" in cluster log

fail 7646029 2024-04-08 20:10:54 2024-04-14 14:23:36 2024-04-14 14:59:21 0:35:45 0:23:26 0:12:19 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-14T14:47:50.669525+0000 mon.smithi042 (mon.0) 553 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646030 2024-04-08 20:10:55 2024-04-14 14:25:47 2024-04-14 14:53:29 0:27:42 0:17:27 0:10:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-14T14:46:58.702028+0000 mon.smithi066 (mon.0) 789 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646031 2024-04-08 20:10:56 2024-04-14 14:25:47 2024-04-14 14:52:16 0:26:29 0:13:43 0:12:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-14T14:48:19.114573+0000 mon.smithi050 (mon.0) 790 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7646032 2024-04-08 20:10:57 2024-04-14 14:27:18 2024-04-14 15:04:21 0:37:03 0:25:22 0:11:41 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7646033 2024-04-08 20:10:57 2024-04-14 14:27:48 2024-04-14 14:54:27 0:26:39 0:15:03 0:11:36 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-14T14:50:46.021668+0000 mon.smithi049 (mon.0) 846 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7646034 2024-04-08 20:10:58 2024-04-14 14:27:49 2024-04-14 14:58:40 0:30:51 0:15:11 0:15:40 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-14T14:53:48.131805+0000 mon.a (mon.0) 668 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646035 2024-04-08 20:10:59 2024-04-14 14:31:40 2024-04-14 15:03:55 0:32:15 0:21:10 0:11:05 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-14T14:58:03.500367+0000 mon.smithi097 (mon.0) 773 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7646036 2024-04-08 20:11:00 2024-04-14 14:31:40 2024-04-14 15:06:30 0:34:50 0:24:06 0:10:44 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-04-14T14:53:05.511899+0000 mon.a (mon.0) 535 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7646037 2024-04-08 20:11:01 2024-04-14 14:31:40 2024-04-14 15:18:43 0:47:03 0:37:10 0:09:53 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-14T14:56:26.867916+0000 mon.a (mon.0) 835 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7646038 2024-04-08 20:11:02 2024-04-14 14:31:41 2024-04-14 14:58:22 0:26:41 0:14:59 0:11:42 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

"2024-04-14T14:54:07.321541+0000 mon.a (mon.0) 381 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646039 2024-04-08 20:11:03 2024-04-14 14:33:12 2024-04-14 15:23:04 0:49:52 0:39:33 0:10:19 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646040 2024-04-08 20:11:04 2024-04-14 14:34:02 2024-04-14 15:09:38 0:35:36 0:23:53 0:11:43 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli_mon} 5
Failure Reason:

"2024-04-14T14:54:56.445790+0000 mon.a (mon.0) 305 : cluster [WRN] Health detail: HEALTH_WARN 2/5 mons down, quorum a,e,c" in cluster log

pass 7646041 2024-04-08 20:11:05 2024-04-14 14:34:33 2024-04-14 14:55:41 0:21:08 0:12:20 0:08:48 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
fail 7646042 2024-04-08 20:11:06 2024-04-14 14:34:33 2024-04-14 14:56:06 0:21:33 0:12:00 0:09:33 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T14:53:05.850872+0000 mon.a (mon.0) 240 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646043 2024-04-08 20:11:07 2024-04-14 14:35:03 2024-04-14 15:02:22 0:27:19 0:16:12 0:11:07 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-14T14:57:48.066597+0000 mon.smithi107 (mon.0) 790 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646044 2024-04-08 20:11:08 2024-04-14 14:35:04 2024-04-14 15:04:47 0:29:43 0:19:47 0:09:56 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
Failure Reason:

"2024-04-14T14:56:27.858833+0000 mon.smithi037 (mon.0) 322 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

dead 7646045 2024-04-08 20:11:09 2024-04-14 14:35:24 2024-04-15 02:43:42 12:08:18 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

hit max job timeout

fail 7646046 2024-04-08 20:11:10 2024-04-14 14:35:25 2024-04-14 15:00:52 0:25:27 0:15:49 0:09:38 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-14T14:57:25.772349+0000 mon.smithi026 (mon.0) 787 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646047 2024-04-08 20:11:11 2024-04-14 14:35:25 2024-04-14 15:00:32 0:25:07 0:15:19 0:09:48 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-04-14T14:56:50.619622+0000 mon.smithi112 (mon.0) 792 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

dead 7646048 2024-04-08 20:11:12 2024-04-14 14:35:25 2024-04-15 14:47:33 1 day, 0:12:08 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

hit max job timeout

fail 7646049 2024-04-08 20:11:13 2024-04-14 14:37:46 2024-04-14 14:55:18 0:17:32 0:07:29 0:10:03 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/flannel rook/1.7.2} 1
Failure Reason:

Command failed on smithi052 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646050 2024-04-08 20:11:14 2024-04-14 14:37:46 2024-04-14 15:06:06 0:28:20 0:16:23 0:11:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7646051 2024-04-08 20:11:15 2024-04-14 14:38:47 2024-04-14 15:14:28 0:35:41 0:26:29 0:09:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7646052 2024-04-08 20:11:16 2024-04-14 14:38:57 2024-04-14 15:04:16 0:25:19 0:15:30 0:09:49 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7646053 2024-04-08 20:11:17 2024-04-14 14:38:58 2024-04-14 15:26:09 0:47:11 0:37:52 0:09:19 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7646054 2024-04-08 20:11:18 2024-04-14 14:38:58 2024-04-14 14:58:28 0:19:30 0:10:26 0:09:04 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_adoption} 1
fail 7646055 2024-04-08 20:11:19 2024-04-14 14:39:29 2024-04-14 15:09:42 0:30:13 0:18:42 0:11:31 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-14T15:02:21.670983+0000 mon.smithi080 (mon.0) 575 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646056 2024-04-08 20:11:20 2024-04-14 14:39:29 2024-04-14 15:03:36 0:24:07 0:13:52 0:10:15 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
Failure Reason:

"2024-04-14T15:00:40.661185+0000 mon.a (mon.0) 289 : cluster [WRN] Health check failed: Degraded data redundancy: 7/21 objects degraded (33.333%), 3 pgs degraded (PG_DEGRADED)" in cluster log

fail 7646057 2024-04-08 20:11:21 2024-04-14 14:40:20 2024-04-14 15:17:30 0:37:10 0:25:53 0:11:17 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-14T15:02:59.991104+0000 mon.a (mon.0) 389 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7646058 2024-04-08 20:11:22 2024-04-14 14:41:50 2024-04-14 15:05:03 0:23:13 0:14:09 0:09:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
fail 7646059 2024-04-08 20:11:23 2024-04-14 14:41:51 2024-04-14 15:33:25 0:51:34 0:36:22 0:15:12 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-14T15:11:49.180327+0000 mon.a (mon.0) 865 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7646060 2024-04-08 20:11:24 2024-04-14 14:45:32 2024-04-14 15:33:52 0:48:20 0:36:23 0:11:57 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-14T15:20:00.000124+0000 mon.a (mon.0) 2648 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log

fail 7646061 2024-04-08 20:11:25 2024-04-14 14:47:02 2024-04-14 15:09:48 0:22:46 0:12:41 0:10:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

"2024-04-14T15:06:55.761185+0000 mon.a (mon.0) 344 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7646062 2024-04-08 20:11:26 2024-04-14 14:47:03 2024-04-14 15:16:52 0:29:49 0:20:30 0:09:19 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7646063 2024-04-08 20:11:27 2024-04-14 14:47:03 2024-04-14 15:12:08 0:25:05 0:14:48 0:10:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-14T15:07:51.436077+0000 mon.smithi156 (mon.0) 783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646064 2024-04-08 20:11:28 2024-04-14 14:47:03 2024-04-14 15:09:03 0:22:00 0:11:44 0:10:16 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-14T15:06:25.375979+0000 mon.a (mon.0) 466 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646065 2024-04-08 20:11:29 2024-04-14 14:47:04 2024-04-14 15:14:46 0:27:42 0:17:16 0:10:26 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi146 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7646066 2024-04-08 20:11:30 2024-04-14 14:47:04 2024-04-14 15:11:51 0:24:47 0:14:53 0:09:54 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-14T15:08:18.631269+0000 mon.smithi017 (mon.0) 818 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646067 2024-04-08 20:11:31 2024-04-14 14:47:05 2024-04-14 15:10:42 0:23:37 0:13:22 0:10:15 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-14T15:07:55.545117+0000 mon.smithi067 (mon.0) 783 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646068 2024-04-08 20:11:32 2024-04-14 14:47:06 2024-04-14 18:36:24 3:49:18 3:37:12 0:12:06 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi134 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7646069 2024-04-08 20:11:32 2024-04-14 14:47:06 2024-04-14 15:17:06 0:30:00 0:19:08 0:10:52 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-14T15:13:57.611624+0000 mon.a (mon.0) 216 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7646070 2024-04-08 20:11:33 2024-04-14 14:47:07 2024-04-14 15:16:35 0:29:28 0:19:00 0:10:28 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T15:11:16.643516+0000 mon.a (mon.0) 288 : cluster [WRN] Health check failed: Degraded data redundancy: 21/63 objects degraded (33.333%), 10 pgs degraded (PG_DEGRADED)" in cluster log

pass 7646071 2024-04-08 20:11:34 2024-04-14 14:47:07 2024-04-14 15:19:34 0:32:27 0:20:32 0:11:55 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7646072 2024-04-08 20:11:35 2024-04-14 14:47:07 2024-04-14 15:04:07 0:17:00 0:07:06 0:09:54 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_cephadm_repos} 1
fail 7646073 2024-04-08 20:11:36 2024-04-14 14:47:08 2024-04-14 16:01:47 1:14:39 1:03:32 0:11:07 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-14T15:09:08.794850+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7646074 2024-04-08 20:11:37 2024-04-14 14:48:08 2024-04-14 15:49:15 1:01:07 0:49:34 0:11:33 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi027 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8025d862-fa70-11ee-bc8e-c7b262605968 -e sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7646075 2024-04-08 20:11:38 2024-04-14 14:48:09 2024-04-14 15:22:36 0:34:27 0:23:26 0:11:01 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
dead 7646076 2024-04-08 20:11:39 2024-04-14 14:48:29 2024-04-15 03:04:16 12:15:47 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

hit max job timeout

fail 7646077 2024-04-08 20:11:40 2024-04-14 15:02:27 2024-04-14 15:28:47 0:26:20 0:15:34 0:10:46 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-14T15:23:15.636422+0000 mon.a (mon.0) 548 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7646078 2024-04-08 20:11:41 2024-04-14 15:02:27 2024-04-14 15:27:25 0:24:58 0:15:29 0:09:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-14T15:22:49.462476+0000 mon.smithi063 (mon.0) 791 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7646079 2024-04-08 20:11:42 2024-04-14 15:02:28 2024-04-14 15:27:29 0:25:01 0:14:21 0:10:40 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_extra_daemon_features} 2
fail 7646080 2024-04-08 20:11:42 2024-04-14 15:02:28 2024-04-14 15:25:39 0:23:11 0:14:13 0:08:58 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-14T15:23:26.108384+0000 mon.smithi042 (mon.0) 792 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646081 2024-04-08 20:11:43 2024-04-14 15:02:38 2024-04-14 15:34:06 0:31:28 0:22:14 0:09:14 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-14T15:24:41.852915+0000 mon.smithi104 (mon.0) 551 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646082 2024-04-08 20:11:44 2024-04-14 15:02:39 2024-04-14 15:51:24 0:48:45 0:39:19 0:09:26 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

fail 7646083 2024-04-08 20:11:45 2024-04-14 15:02:39 2024-04-14 15:48:27 0:45:48 0:34:35 0:11:13 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-04-14T15:22:15.805024+0000 mon.smithi050 (mon.0) 505 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7646084 2024-04-08 20:11:46 2024-04-14 15:02:40 2024-04-14 15:47:15 0:44:35 0:36:37 0:07:58 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/nfs} 1
Failure Reason:

"2024-04-14T15:23:26.932054+0000 mon.a (mon.0) 610 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7646085 2024-04-08 20:11:47 2024-04-14 15:02:50 2024-04-14 15:26:51 0:24:01 0:14:03 0:09:58 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-14T15:24:35.950097+0000 mon.a (mon.0) 463 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7646086 2024-04-08 20:11:48 2024-04-14 15:02:50 2024-04-14 15:24:24 0:21:34 0:12:10 0:09:24 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
fail 7646087 2024-04-08 20:11:49 2024-04-14 15:02:51 2024-04-14 15:40:47 0:37:56 0:26:58 0:10:58 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-14T15:21:18.777658+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7646088 2024-04-08 20:11:50 2024-04-14 15:02:51 2024-04-14 15:10:32 0:07:41 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
Failure Reason:

Command failed on smithi059 with status 1: 'sudo yum install -y kernel'

fail 7646089 2024-04-08 20:11:51 2024-04-14 15:02:51 2024-04-14 15:18:06 0:15:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

Failed to reconnect to smithi059

pass 7646090 2024-04-08 20:11:51 2024-04-14 15:02:52 2024-04-14 15:23:55 0:21:03 0:11:13 0:09:50 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
fail 7646091 2024-04-08 20:11:52 2024-04-14 15:02:52 2024-04-14 15:26:29 0:23:37 0:14:17 0:09:20 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-14T15:23:43.114210+0000 mon.a (mon.0) 422 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646092 2024-04-08 20:11:53 2024-04-14 15:02:53 2024-04-14 15:37:05 0:34:12 0:23:30 0:10:42 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-04-14T15:28:15.853448+0000 mon.a (mon.0) 540 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7646093 2024-04-08 20:11:54 2024-04-14 15:02:53 2024-04-14 15:45:19 0:42:26 0:30:52 0:11:34 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-14T15:28:23.805899+0000 mon.a (mon.0) 840 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7646094 2024-04-08 20:11:55 2024-04-14 15:02:54 2024-04-14 15:22:29 0:19:35 0:07:27 0:12:08 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/host rook/master} 3
Failure Reason:

Command failed on smithi049 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7646095 2024-04-08 20:11:56 2024-04-14 15:02:54 2024-04-14 15:30:07 0:27:13 0:17:22 0:09:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-14T15:23:45.845578+0000 mon.smithi005 (mon.0) 785 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646096 2024-04-08 20:11:57 2024-04-14 15:02:54 2024-04-14 15:54:16 0:51:22 0:39:18 0:12:04 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
Failure Reason:

"2024-04-14T15:26:40.851759+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7646097 2024-04-08 20:11:58 2024-04-14 15:04:15 2024-04-14 15:34:53 0:30:38 0:19:47 0:10:51 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-14T15:32:20.127863+0000 mon.smithi083 (mon.0) 773 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646098 2024-04-08 20:11:59 2024-04-14 15:04:25 2024-04-14 15:36:24 0:31:59 0:21:14 0:10:45 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-14T15:30:47.744879+0000 mon.smithi159 (mon.0) 773 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646099 2024-04-08 20:12:00 2024-04-14 15:04:26 2024-04-14 15:27:20 0:22:54 0:14:02 0:08:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-04-14T15:24:30.991444+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646100 2024-04-08 20:12:01 2024-04-14 15:04:26 2024-04-14 15:30:15 0:25:49 0:15:03 0:10:46 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-14T15:24:42.580700+0000 mon.smithi046 (mon.0) 576 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646101 2024-04-08 20:12:02 2024-04-14 15:05:07 2024-04-14 15:42:18 0:37:11 0:27:23 0:09:48 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-14T15:20:31.746025+0000 mon.a (mon.0) 167 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7646102 2024-04-08 20:12:02 2024-04-14 15:05:17 2024-04-14 15:31:37 0:26:20 0:15:32 0:10:48 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-14T15:23:51.682432+0000 mon.a (mon.0) 414 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

dead 7646103 2024-04-08 20:12:03 2024-04-14 15:06:08 2024-04-15 03:24:38 12:18:30 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

fail 7646104 2024-04-08 20:12:04 2024-04-14 15:14:29 2024-04-14 15:52:18 0:37:49 0:25:46 0:12:03 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli_mon} 5
Failure Reason:

"2024-04-14T15:38:50.751101+0000 mon.a (mon.0) 271 : cluster [WRN] Health detail: HEALTH_WARN 1/4 mons down, quorum a,e,c" in cluster log

fail 7646105 2024-04-08 20:12:05 2024-04-14 15:17:00 2024-04-14 15:38:46 0:21:46 0:11:40 0:10:06 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T15:35:44.436756+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646106 2024-04-08 20:12:06 2024-04-14 15:18:01 2024-04-14 15:41:41 0:23:40 0:14:07 0:09:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-04-14T15:38:55.691277+0000 mon.smithi117 (mon.0) 768 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646107 2024-04-08 20:12:07 2024-04-14 15:18:01 2024-04-14 15:48:59 0:30:58 0:20:48 0:10:10 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-14T15:39:04.396297+0000 mon.a (mon.0) 385 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646108 2024-04-08 20:12:08 2024-04-14 15:18:01 2024-04-14 15:43:33 0:25:32 0:14:48 0:10:44 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-14T15:37:35.326434+0000 mon.smithi149 (mon.0) 720 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7646109 2024-04-08 20:12:09 2024-04-14 15:18:02 2024-04-14 16:05:48 0:47:46 0:37:16 0:10:30 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-14T15:45:46.742455+0000 mon.a (mon.0) 997 : cluster [ERR] Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL)" in cluster log

fail 7646110 2024-04-08 20:12:10 2024-04-14 15:18:02 2024-04-14 15:48:29 0:30:27 0:20:16 0:10:11 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-04-14T15:44:54.993394+0000 mon.smithi070 (mon.0) 753 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7646111 2024-04-08 20:12:10 2024-04-14 15:18:03 2024-04-14 15:48:43 0:30:40 0:18:38 0:12:02 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7646112 2024-04-08 20:12:11 2024-04-14 15:18:03 2024-04-14 15:42:09 0:24:06 0:12:52 0:11:14 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-04-14T15:37:12.467096+0000 mon.a (mon.0) 399 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646113 2024-04-08 20:12:12 2024-04-14 15:18:03 2024-04-14 16:04:49 0:46:46 0:36:44 0:10:02 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-04-14T15:44:23.758636+0000 mon.a (mon.0) 1148 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7646114 2024-04-08 20:12:13 2024-04-14 15:18:04 2024-04-14 15:48:22 0:30:18 0:19:42 0:10:36 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_rgw_multisite} 3
fail 7646115 2024-04-08 20:12:14 2024-04-14 15:18:04 2024-04-14 15:26:21 0:08:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

Command failed on smithi178 with status 1: 'sudo yum install -y kernel'

fail 7646116 2024-04-08 20:12:15 2024-04-14 15:18:05 2024-04-14 15:34:22 0:16:17 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Failed to reconnect to smithi178

pass 7646117 2024-04-08 20:12:16 2024-04-14 15:18:05 2024-04-14 15:39:05 0:21:00 0:12:39 0:08:21 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_adoption} 1
pass 7646118 2024-04-08 20:12:17 2024-04-14 15:18:05 2024-04-14 15:42:14 0:24:09 0:13:33 0:10:36 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7646119 2024-04-08 20:12:18 2024-04-14 15:18:06 2024-04-14 15:49:23 0:31:17 0:19:11 0:12:06 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
Failure Reason:

"2024-04-14T15:44:49.396534+0000 mon.a (mon.0) 250 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7646120 2024-04-08 20:12:19 2024-04-14 15:18:06 2024-04-14 15:48:24 0:30:18 0:19:28 0:10:50 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7646121 2024-04-08 20:12:20 2024-04-14 15:18:07 2024-04-14 15:46:32 0:28:25 0:16:32 0:11:53 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_set_mon_crush_locations} 3
pass 7646122 2024-04-08 20:12:21 2024-04-14 15:18:07 2024-04-14 15:43:55 0:25:48 0:14:37 0:11:11 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7646123 2024-04-08 20:12:22 2024-04-14 15:18:08 2024-04-14 16:25:14 1:07:06 0:54:50 0:12:16 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-04-14T15:41:37.371213+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7646124 2024-04-08 20:12:22 2024-04-14 15:18:08 2024-04-14 15:54:31 0:36:23 0:25:27 0:10:56 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-14T15:45:20.061148+0000 mon.a (mon.0) 604 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7646125 2024-04-08 20:12:23 2024-04-14 15:18:09 2024-04-14 15:44:44 0:26:35 0:15:00 0:11:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-04-14T15:40:37.105284+0000 mon.smithi080 (mon.0) 778 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646126 2024-04-08 20:12:24 2024-04-14 15:18:09 2024-04-14 16:11:38 0:53:29 0:37:40 0:15:49 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-04-14T15:46:57.806481+0000 mon.a (mon.0) 480 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7646127 2024-04-08 20:12:25 2024-04-14 15:22:40 2024-04-14 15:46:54 0:24:14 0:12:33 0:11:41 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_ca_signed_key} 2
fail 7646128 2024-04-08 20:12:26 2024-04-14 15:24:01 2024-04-14 15:57:40 0:33:39 0:23:45 0:09:54 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-04-14T15:50:00.000199+0000 mon.a (mon.0) 851 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log

fail 7646129 2024-04-08 20:12:27 2024-04-14 15:24:31 2024-04-14 15:57:26 0:32:55 0:20:02 0:12:53 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-04-14T15:53:54.901654+0000 mon.smithi064 (mon.0) 801 : cluster [WRN] Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646130 2024-04-08 20:12:28 2024-04-14 15:27:32 2024-04-14 16:01:12 0:33:40 0:19:00 0:14:40 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-14T15:58:32.809272+0000 mon.smithi042 (mon.0) 745 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7646131 2024-04-08 20:12:29 2024-04-14 15:33:33 2024-04-15 03:42:13 12:08:40 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

fail 7646132 2024-04-08 20:12:30 2024-04-14 15:33:33 2024-04-14 15:57:48 0:24:15 0:13:56 0:10:19 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_cephadm_timeout} 1
Failure Reason:

"2024-04-14T15:56:29.814504+0000 mon.a (mon.0) 213 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

pass 7646133 2024-04-08 20:12:31 2024-04-14 15:33:34 2024-04-14 15:57:01 0:23:27 0:13:52 0:09:35 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
fail 7646134 2024-04-08 20:12:32 2024-04-14 15:33:34 2024-04-14 15:54:35 0:21:01 0:11:00 0:10:01 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T15:50:43.274936+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7646135 2024-04-08 20:12:33 2024-04-14 15:33:34 2024-04-14 15:58:49 0:25:15 0:14:22 0:10:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7646136 2024-04-08 20:12:34 2024-04-14 15:33:35 2024-04-14 15:56:20 0:22:45 0:13:07 0:09:38 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
Failure Reason:

"2024-04-14T15:49:49.551288+0000 mon.smithi137 (mon.0) 242 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646137 2024-04-08 20:12:35 2024-04-14 15:33:35 2024-04-14 15:57:46 0:24:11 0:11:49 0:12:22 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-14T15:52:18.440277+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646138 2024-04-08 20:12:36 2024-04-14 15:33:36 2024-04-14 15:58:00 0:24:24 0:13:54 0:10:30 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7646139 2024-04-08 20:12:37 2024-04-14 15:33:36 2024-04-14 15:51:18 0:17:42 0:07:34 0:10:08 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/3-node k8s/1.21 net/calico rook/1.7.2} 3
Failure Reason:

Command failed on smithi052 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7646140 2024-04-08 20:12:38 2024-04-14 15:33:36 2024-04-14 16:01:23 0:27:47 0:17:03 0:10:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
fail 7646141 2024-04-08 20:12:39 2024-04-14 15:33:47 2024-04-14 16:33:15 0:59:28 0:48:22 0:11:06 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-04-14T15:59:57.788189+0000 mon.a (mon.0) 855 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7646142 2024-04-08 20:12:39 2024-04-14 15:33:47 2024-04-14 16:06:05 0:32:18 0:21:54 0:10:24 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-04-14T16:00:30.443191+0000 mon.smithi046 (mon.0) 792 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646143 2024-04-08 20:12:40 2024-04-14 15:33:48 2024-04-14 15:51:18 0:17:30 0:08:59 0:08:31 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7646144 2024-04-08 20:12:41 2024-04-14 15:33:48 2024-04-14 15:57:36 0:23:48 0:14:01 0:09:47 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-14T15:55:01.585702+0000 mon.smithi131 (mon.0) 794 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646145 2024-04-08 20:12:42 2024-04-14 15:33:48 2024-04-14 16:00:35 0:26:47 0:15:14 0:11:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-04-14T15:56:26.410590+0000 mon.smithi049 (mon.0) 783 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646146 2024-04-08 20:12:43 2024-04-14 15:59:29 899 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-14T15:53:31.970361+0000 mon.a (mon.0) 562 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

dead 7646147 2024-04-08 20:12:44 2024-04-14 15:33:49 2024-04-14 15:46:41 0:12:52 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

Error reimaging machines: Expected smithi032's OS to be ubuntu 22.04 but found centos 9

dead 7646148 2024-04-08 20:12:45 2024-04-14 15:39:11 2024-04-14 15:47:17 0:08:06 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Error reimaging machines: list index out of range

fail 7646149 2024-04-08 20:12:46 2024-04-14 15:42:21 2024-04-14 16:05:14 0:22:53 0:13:43 0:09:10 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-04-14T16:03:05.864580+0000 mon.a (mon.0) 462 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646150 2024-04-08 20:12:48 2024-04-14 15:42:22 2024-04-14 16:07:20 0:24:58 0:13:33 0:11:25 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
Failure Reason:

"2024-04-14T16:04:38.656483+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646151 2024-04-08 20:12:49 2024-04-14 15:44:02 2024-04-14 16:15:42 0:31:40 0:17:22 0:14:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-04-14T16:08:39.784445+0000 mon.smithi008 (mon.0) 780 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646152 2024-04-08 20:12:50 2024-04-14 15:46:33 2024-04-14 16:09:48 0:23:15 0:13:29 0:09:46 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_extra_daemon_features} 2
Failure Reason:

"2024-04-14T16:05:28.351609+0000 mon.a (mon.0) 344 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7646153 2024-04-08 20:12:51 2024-04-14 15:46:54 2024-04-14 16:20:25 0:33:31 0:24:19 0:09:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-04-14T16:12:41.418269+0000 mon.smithi087 (mon.0) 773 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7646154 2024-04-08 20:12:51 2024-04-14 15:46:54 2024-04-14 16:40:24 0:53:30 0:43:48 0:09:42 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi032 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 13203638-fa78-11ee-bc8e-c7b262605968 -e sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

fail 7646155 2024-04-08 20:12:52 2024-04-14 15:47:35 2024-04-14 16:13:17 0:25:42 0:14:27 0:11:15 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-14T16:10:12.047703+0000 mon.smithi038 (mon.0) 794 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646156 2024-04-08 20:12:53 2024-04-14 15:48:25 2024-04-14 16:29:23 0:40:58 0:32:01 0:08:57 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-04-14T16:12:28.136353+0000 mon.a (mon.0) 869 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS)" in cluster log

fail 7646157 2024-04-08 20:12:54 2024-04-14 15:48:26 2024-04-14 16:13:41 0:25:15 0:15:41 0:09:34 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-04-14T16:10:00.000096+0000 mon.smithi120 (mon.0) 803 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds; 1 pool(s) do not have an application enabled" in cluster log

pass 7646158 2024-04-08 20:12:55 2024-04-14 15:48:46 2024-04-14 16:10:26 0:21:40 0:12:24 0:09:16 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
fail 7646159 2024-04-08 20:12:56 2024-04-14 15:48:47 2024-04-14 16:17:25 0:28:38 0:16:09 0:12:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-14T16:14:05.877337+0000 mon.a (mon.0) 617 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7646160 2024-04-08 20:12:57 2024-04-14 15:49:07 2024-04-14 16:15:17 0:26:10 0:15:40 0:10:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-04-14T16:11:00.146650+0000 mon.smithi043 (mon.0) 792 : cluster [WRN] Health check failed: Failed to place 1 daemon(s) (CEPHADM_DAEMON_PLACE_FAIL)" in cluster log

fail 7646161 2024-04-08 20:12:58 2024-04-14 15:49:07 2024-04-14 19:38:16 3:49:09 3:36:50 0:12:19 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi104 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7646162 2024-04-08 20:12:59 2024-04-14 15:49:08 2024-04-14 16:37:53 0:48:45 0:38:15 0:10:30 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_orch_cli_mon} 5
Failure Reason:

"2024-04-14T16:14:35.409901+0000 mon.a (mon.0) 243 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,e" in cluster log

fail 7646163 2024-04-08 20:13:00 2024-04-14 15:49:08 2024-04-14 16:19:26 0:30:18 0:18:42 0:11:36 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
Failure Reason:

"2024-04-14T16:13:54.491366+0000 mon.a (mon.0) 247 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log