Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7558010 2024-02-13 11:42:03 2024-02-13 13:54:06 2024-02-13 14:16:50 0:22:44 0:13:33 0:09:11 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7558011 2024-02-13 11:42:04 2024-02-13 13:54:06 2024-02-13 14:39:09 0:45:03 0:35:16 0:09:47 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-13T14:20:00.000143+0000 mon.a (mon.0) 1065 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7558012 2024-02-13 11:42:05 2024-02-13 13:54:06 2024-02-13 14:27:16 0:33:10 0:22:41 0:10:29 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-02-13T14:20:00.000190+0000 mon.a (mon.0) 1068 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

fail 7558013 2024-02-13 11:42:05 2024-02-13 13:54:07 2024-02-13 14:22:43 0:28:36 0:16:32 0:12:04 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-02-13T14:12:24.340206+0000 mon.smithi071 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558014 2024-02-13 11:42:06 2024-02-13 13:55:37 2024-02-13 14:41:27 0:45:50 0:34:24 0:11:26 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-13T14:15:00.009043+0000 mon.smithi046 (mon.0) 503 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7558015 2024-02-13 11:42:07 2024-02-13 13:56:18 2024-02-13 14:39:22 0:43:04 0:33:43 0:09:21 smithi main centos 9.stream orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7558016 2024-02-13 11:42:08 2024-02-13 13:56:18 2024-02-13 14:17:28 0:21:10 0:11:35 0:09:35 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
fail 7558017 2024-02-13 11:42:09 2024-02-13 13:56:29 2024-02-13 14:26:55 0:30:26 0:19:02 0:11:24 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-02-13T14:17:19.311355+0000 mon.smithi135 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558018 2024-02-13 11:42:10 2024-02-13 13:57:09 2024-02-13 14:32:35 0:35:26 0:25:30 0:09:56 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7558019 2024-02-13 11:42:11 2024-02-13 13:57:29 2024-02-13 14:22:40 0:25:11 0:15:04 0:10:07 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7558020 2024-02-13 11:42:11 2024-02-13 13:57:40 2024-02-13 14:25:14 0:27:34 0:18:49 0:08:45 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
fail 7558021 2024-02-13 11:42:12 2024-02-13 13:57:40 2024-02-13 14:18:40 0:21:00 0:11:48 0:09:12 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-02-13T14:13:45.287104+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7558022 2024-02-13 11:42:13 2024-02-13 13:57:40 2024-02-13 14:30:56 0:33:16 0:18:48 0:14:28 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7558023 2024-02-13 11:42:14 2024-02-13 14:01:41 2024-02-13 14:36:24 0:34:43 0:22:13 0:12:30 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-02-13T14:24:14.284728+0000 mon.smithi039 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558024 2024-02-13 11:42:15 2024-02-13 14:01:42 2024-02-13 14:26:48 0:25:06 0:14:44 0:10:22 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-02-13T14:18:39.813808+0000 mon.smithi142 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558025 2024-02-13 11:42:16 2024-02-13 14:01:42 2024-02-13 14:24:33 0:22:51 0:13:05 0:09:46 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_orch_cli} 1
fail 7558026 2024-02-13 11:42:16 2024-02-13 14:01:42 2024-02-13 14:28:43 0:27:01 0:14:05 0:12:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-02-13T14:20:45.286054+0000 mon.smithi023 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558027 2024-02-13 11:42:17 2024-02-13 14:02:33 2024-02-13 14:53:40 0:51:07 0:38:58 0:12:09 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
fail 7558028 2024-02-13 11:42:18 2024-02-13 14:03:03 2024-02-13 15:19:16 1:16:13 1:05:23 0:10:50 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-13T14:26:13.844945+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7558029 2024-02-13 11:42:19 2024-02-13 14:03:14 2024-02-13 14:54:02 0:50:48 0:39:34 0:11:14 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

"2024-02-13T14:26:10.860469+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7558030 2024-02-13 11:42:20 2024-02-13 14:04:24 2024-02-13 14:48:41 0:44:17 0:32:26 0:11:51 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1707834554.0297945 mon.smithi022 (mon.0) 692 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7558031 2024-02-13 11:42:20 2024-02-13 14:05:55 2024-02-13 14:31:38 0:25:43 0:14:37 0:11:06 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-02-13T14:23:16.601675+0000 mon.smithi153 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558032 2024-02-13 11:42:21 2024-02-13 14:05:55 2024-02-13 14:35:54 0:29:59 0:20:28 0:09:31 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-02-13T14:26:49.116224+0000 mon.smithi081 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558033 2024-02-13 11:42:22 2024-02-13 14:06:06 2024-02-13 14:30:53 0:24:47 0:13:20 0:11:27 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7558034 2024-02-13 11:42:23 2024-02-13 14:06:56 2024-02-13 14:34:11 0:27:15 0:17:58 0:09:17 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-02-13T14:23:43.385231+0000 mon.smithi120 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558035 2024-02-13 11:42:24 2024-02-13 14:07:07 2024-02-13 14:57:38 0:50:31 0:39:44 0:10:47 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-02-13T14:27:15.004414+0000 mon.a (mon.0) 438 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7558036 2024-02-13 11:42:25 2024-02-13 14:07:27 2024-02-13 14:33:55 0:26:28 0:15:59 0:10:29 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7558037 2024-02-13 11:42:26 2024-02-13 14:08:38 2024-02-13 14:33:40 0:25:02 0:16:00 0:09:02 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-02-13T14:25:18.619712+0000 mon.smithi115 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558038 2024-02-13 11:42:26 2024-02-13 14:08:38 2024-02-13 14:32:25 0:23:47 0:12:53 0:10:54 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-02-13T14:26:09.459606+0000 mon.smithi052 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558039 2024-02-13 11:42:27 2024-02-13 14:09:18 2024-02-13 14:45:20 0:36:02 0:25:40 0:10:22 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7558040 2024-02-13 11:42:28 2024-02-13 14:09:39 2024-02-13 14:40:29 0:30:50 0:19:19 0:11:31 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-02-13T14:31:11.513197+0000 mon.smithi131 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558041 2024-02-13 11:42:29 2024-02-13 14:10:59 2024-02-13 14:41:07 0:30:08 0:12:32 0:17:36 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
fail 7558042 2024-02-13 11:42:30 2024-02-13 14:11:40 2024-02-13 15:25:59 1:14:19 1:03:51 0:10:28 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-13T14:40:00.000211+0000 mon.a (mon.0) 1092 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

fail 7558043 2024-02-13 11:42:31 2024-02-13 14:12:51 2024-02-13 15:25:22 1:12:31 1:01:47 0:10:44 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-13T14:35:42.790926+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7558044 2024-02-13 11:42:32 2024-02-13 14:13:11 2024-02-13 14:35:30 0:22:19 0:12:24 0:09:55 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_adoption} 1
fail 7558045 2024-02-13 11:42:33 2024-02-13 14:13:51 2024-02-13 14:37:20 0:23:29 0:13:53 0:09:36 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-02-13T14:30:48.866268+0000 mon.smithi122 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558046 2024-02-13 11:42:33 2024-02-13 14:14:12 2024-02-13 14:36:51 0:22:39 0:11:35 0:11:04 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_ca_signed_key} 2
fail 7558047 2024-02-13 11:42:34 2024-02-13 14:14:32 2024-02-13 18:05:41 3:51:09 3:36:41 0:14:28 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi118 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e1b79d5a4193a484176863ab96c0ca28c00e1662 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7558048 2024-02-13 11:42:35 2024-02-13 14:15:43 2024-02-13 14:41:47 0:26:04 0:14:17 0:11:47 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-02-13T14:33:26.349613+0000 mon.smithi040 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558049 2024-02-13 11:42:36 2024-02-13 14:17:03 2024-02-13 14:42:30 0:25:27 0:15:04 0:10:23 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm} 1
fail 7558050 2024-02-13 11:42:37 2024-02-13 14:17:04 2024-02-13 14:48:44 0:31:40 0:21:30 0:10:10 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-02-13T14:37:15.005053+0000 mon.smithi121 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558051 2024-02-13 11:42:38 2024-02-13 14:17:14 2024-02-13 14:47:05 0:29:51 0:19:53 0:09:58 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-02-13T14:37:43.367808+0000 mon.smithi106 (mon.0) 255 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558052 2024-02-13 11:42:38 2024-02-13 14:17:34 2024-02-13 15:00:19 0:42:45 0:31:39 0:11:06 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7558053 2024-02-13 11:42:39 2024-02-13 14:19:25 2024-02-13 14:50:48 0:31:23 0:22:14 0:09:09 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} 2
fail 7558054 2024-02-13 11:42:40 2024-02-13 14:19:35 2024-02-13 14:37:39 0:18:04 0:08:36 0:09:28 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi196 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e1b79d5a4193a484176863ab96c0ca28c00e1662 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7558055 2024-02-13 11:42:41 2024-02-13 14:20:06 2024-02-13 14:46:39 0:26:33 0:14:28 0:12:05 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-02-13T14:38:51.265861+0000 mon.smithi002 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558056 2024-02-13 11:42:42 2024-02-13 14:21:46 2024-02-13 15:15:40 0:53:54 0:43:11 0:10:43 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi190 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eaff6a8-ca7d-11ee-95b9-87774f69a715 -e sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7558057 2024-02-13 11:42:43 2024-02-13 14:22:37 2024-02-13 14:45:28 0:22:51 0:13:54 0:08:57 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_timeout} 1
fail 7558058 2024-02-13 11:42:44 2024-02-13 14:22:37 2024-02-13 14:47:40 0:25:03 0:15:44 0:09:19 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-02-13T14:39:31.429885+0000 mon.smithi038 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558059 2024-02-13 11:42:45 2024-02-13 14:22:48 2024-02-13 14:57:25 0:34:37 0:23:57 0:10:40 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-02-13T14:45:13.795225+0000 mon.smithi090 (mon.0) 255 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558060 2024-02-13 11:42:45 2024-02-13 14:22:48 2024-02-13 14:46:18 0:23:30 0:13:31 0:09:59 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-02-13T14:43:31.540146+0000 mon.a (mon.0) 46 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log

fail 7558061 2024-02-13 11:42:46 2024-02-13 14:23:18 2024-02-13 15:07:40 0:44:22 0:32:44 0:11:38 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-02-13T15:00:00.000185+0000 mon.smithi123 (mon.0) 628 : cluster 3 [WRN] FS_DEGRADED: 1 filesystem is degraded" in cluster log

pass 7558062 2024-02-13 11:42:47 2024-02-13 14:24:39 2024-02-13 14:47:58 0:23:19 0:11:52 0:11:27 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
dead 7558063 2024-02-13 11:42:48 2024-02-13 14:26:50 2024-02-13 14:47:17 0:20:27 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds

pass 7558064 2024-02-13 11:42:49 2024-02-13 14:29:00 2024-02-13 14:55:14 0:26:14 0:15:13 0:11:01 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7558065 2024-02-13 11:42:50 2024-02-13 14:30:11 2024-02-13 15:00:13 0:30:02 0:20:04 0:09:58 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
fail 7558066 2024-02-13 11:42:50 2024-02-13 14:30:11 2024-02-13 14:52:53 0:22:42 0:11:37 0:11:05 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-02-13T14:47:57.436636+0000 mon.a (mon.0) 246 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7558067 2024-02-13 11:42:51 2024-02-13 14:31:02 2024-02-13 14:56:20 0:25:18 0:14:56 0:10:22 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-02-13T14:48:17.058772+0000 mon.smithi032 (mon.0) 258 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558068 2024-02-13 11:42:52 2024-02-13 14:31:02 2024-02-13 15:31:43 1:00:41 0:49:19 0:11:22 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-13T14:54:39.107970+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7558069 2024-02-13 11:42:53 2024-02-13 14:31:13 2024-02-13 15:07:04 0:35:51 0:24:19 0:11:32 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rotate-keys} 2
pass 7558070 2024-02-13 11:42:54 2024-02-13 14:31:43 2024-02-13 15:06:24 0:34:41 0:23:55 0:10:46 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
fail 7558071 2024-02-13 11:42:55 2024-02-13 14:32:44 2024-02-13 15:00:04 0:27:20 0:18:26 0:08:54 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-02-13T14:50:27.815789+0000 mon.smithi169 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558072 2024-02-13 11:42:56 2024-02-13 14:32:54 2024-02-13 14:59:12 0:26:18 0:14:15 0:12:03 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7558073 2024-02-13 11:42:56 2024-02-13 14:33:25 2024-02-13 15:07:07 0:33:42 0:23:53 0:09:49 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-02-13T14:54:24.707345+0000 mon.smithi094 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558074 2024-02-13 11:42:57 2024-02-13 14:33:25 2024-02-13 14:56:22 0:22:57 0:14:14 0:08:43 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-02-13T14:50:23.369055+0000 mon.smithi026 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558075 2024-02-13 11:42:58 2024-02-13 14:33:26 2024-02-13 15:02:26 0:29:00 0:18:45 0:10:15 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7558076 2024-02-13 11:42:59 2024-02-13 14:33:26 2024-02-13 14:58:23 0:24:57 0:15:44 0:09:13 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-02-13T14:50:21.343298+0000 mon.smithi170 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558077 2024-02-13 11:43:00 2024-02-13 14:33:26 2024-02-13 15:21:10 0:47:44 0:37:46 0:09:58 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-02-13T14:51:12.916301+0000 mon.a (mon.0) 172 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7558078 2024-02-13 11:43:01 2024-02-13 14:33:27 2024-02-13 15:47:21 1:13:54 1:03:09 0:10:45 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-13T14:55:48.232135+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7558079 2024-02-13 11:43:01 2024-02-13 14:33:27 2024-02-13 15:45:37 1:12:10 1:01:15 0:10:55 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-13T14:55:38.701932+0000 mon.a (mon.0) 181 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7558080 2024-02-13 11:43:02 2024-02-13 14:33:27 2024-02-13 14:59:26 0:25:59 0:14:54 0:11:05 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-02-13T14:51:11.406712+0000 mon.smithi115 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558081 2024-02-13 11:43:03 2024-02-13 14:33:28 2024-02-13 15:03:25 0:29:57 0:20:09 0:09:48 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli} 1
fail 7558082 2024-02-13 11:43:04 2024-02-13 14:33:28 2024-02-13 18:23:34 3:50:06 3:36:04 0:14:02 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi119 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e1b79d5a4193a484176863ab96c0ca28c00e1662 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7558083 2024-02-13 11:43:05 2024-02-13 14:33:59 2024-02-13 15:04:17 0:30:18 0:20:31 0:09:47 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-02-13T14:55:46.818595+0000 mon.smithi033 (mon.0) 257 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558084 2024-02-13 11:43:06 2024-02-13 14:34:29 2024-02-13 15:11:20 0:36:51 0:24:15 0:12:36 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_orch_cli_mon} 5
fail 7558085 2024-02-13 11:43:07 2024-02-13 14:38:10 2024-02-13 15:11:03 0:32:53 0:21:03 0:11:50 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-02-13T15:00:20.315923+0000 mon.smithi005 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558086 2024-02-13 11:43:07 2024-02-13 14:38:10 2024-02-13 15:06:37 0:28:27 0:15:12 0:13:15 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7558087 2024-02-13 11:43:08 2024-02-13 14:41:11 2024-02-13 15:03:55 0:22:44 0:11:20 0:11:24 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-02-13T14:59:14.400753+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7558088 2024-02-13 11:43:09 2024-02-13 14:42:32 2024-02-13 15:10:00 0:27:28 0:13:50 0:13:38 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-02-13T15:03:55.328853+0000 mon.smithi006 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558089 2024-02-13 11:43:10 2024-02-13 14:45:02 2024-02-13 15:08:27 0:23:25 0:13:22 0:10:03 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_rgw_multisite} 3
fail 7558090 2024-02-13 11:43:11 2024-02-13 14:45:23 2024-02-13 15:13:57 0:28:34 0:18:07 0:10:27 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-02-13T15:03:26.446862+0000 mon.smithi003 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558091 2024-02-13 11:43:12 2024-02-13 14:47:03 2024-02-13 16:00:21 1:13:18 1:04:21 0:08:57 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-13T15:20:00.000114+0000 mon.a (mon.0) 1247 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

pass 7558092 2024-02-13 11:43:12 2024-02-13 14:47:04 2024-02-13 15:20:34 0:33:30 0:22:37 0:10:53 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7558093 2024-02-13 11:43:13 2024-02-13 14:47:14 2024-02-13 15:24:15 0:37:01 0:21:31 0:15:30 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7558094 2024-02-13 11:43:14 2024-02-13 14:47:35 2024-02-13 15:19:45 0:32:10 0:21:35 0:10:35 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-02-13T15:08:31.373765+0000 mon.smithi017 (mon.0) 255 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558095 2024-02-13 11:43:15 2024-02-13 14:48:06 2024-02-13 15:31:22 0:43:16 0:33:02 0:10:14 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1707837095.9563854 mon.smithi050 (mon.0) 712 : cluster [WRN] Replacing daemon mds.cephfs.smithi050.lbkann as rank 0 with standby daemon mds.cephfs.smithi064.kuvmgs" in cluster log

fail 7558096 2024-02-13 11:43:16 2024-02-13 14:48:36 2024-02-13 15:11:48 0:23:12 0:12:25 0:10:47 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-02-13T15:07:01.294656+0000 mon.smithi022 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558097 2024-02-13 11:43:17 2024-02-13 14:48:57 2024-02-13 15:11:54 0:22:57 0:13:19 0:09:38 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-02-13T15:05:10.632277+0000 mon.smithi042 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7558098 2024-02-13 11:43:17 2024-02-13 14:48:57 2024-02-13 15:25:53 0:36:56 0:28:21 0:08:35 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-02-13T15:03:30.599385+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7558099 2024-02-13 11:43:18 2024-02-13 14:48:57 2024-02-13 15:07:47 0:18:50 0:09:56 0:08:54 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_adoption} 1
fail 7558100 2024-02-13 11:43:19 2024-02-13 14:48:58 2024-02-13 15:11:49 0:22:51 0:13:27 0:09:24 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-02-13T15:05:36.737892+0000 mon.smithi037 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7558101 2024-02-13 11:43:20 2024-02-13 14:48:58 2024-02-13 15:09:47 0:20:49 0:11:20 0:09:29 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7558102 2024-02-13 11:43:21 2024-02-13 14:48:59 2024-02-13 15:29:41 0:40:42 0:32:37 0:08:05 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-13T15:20:00.000113+0000 mon.a (mon.0) 1370 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log