Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7579040 2024-03-01 21:00:09 2024-03-02 09:23:19 2024-03-02 09:43:04 0:19:45 0:07:38 0:12:07 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} 3
Failure Reason:

Command failed on smithi007 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7579041 2024-03-01 21:00:10 2024-03-02 09:25:30 2024-03-02 09:58:16 0:32:46 0:22:02 0:10:44 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-02T09:47:39.572548+0000 mon.smithi089 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579042 2024-03-01 21:00:11 2024-03-02 09:25:30 2024-03-02 09:47:34 0:22:04 0:10:57 0:11:07 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi073 with status 5: 'sudo systemctl stop ceph-6677c11e-d879-11ee-95c5-87774f69a715@mon.a'

pass 7579043 2024-03-01 21:00:11 2024-03-02 09:25:41 2024-03-02 10:01:13 0:35:32 0:23:18 0:12:14 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
fail 7579044 2024-03-01 21:00:12 2024-03-02 09:35:17 2024-03-02 10:20:08 0:44:51 0:34:38 0:10:13 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-02T09:53:48.458133+0000 mon.smithi047 (mon.0) 505 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7579045 2024-03-01 21:00:13 2024-03-02 09:35:17 2024-03-02 10:29:01 0:53:44 0:44:49 0:08:55 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7579046 2024-03-01 21:00:14 2024-03-02 09:35:17 2024-03-02 09:56:31 0:21:14 0:11:36 0:09:38 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
pass 7579047 2024-03-01 21:00:15 2024-03-02 09:35:18 2024-03-02 10:11:25 0:36:07 0:23:08 0:12:59 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7579048 2024-03-01 21:00:15 2024-03-02 09:36:08 2024-03-02 10:15:24 0:39:16 0:27:46 0:11:30 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7579049 2024-03-01 21:00:16 2024-03-02 09:36:39 2024-03-02 10:04:21 0:27:42 0:16:52 0:10:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-02T09:54:54.332543+0000 mon.smithi157 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579050 2024-03-01 21:00:17 2024-03-02 09:36:39 2024-03-02 09:57:39 0:21:00 0:11:13 0:09:47 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/basic 3-final} 1
pass 7579051 2024-03-01 21:00:18 2024-03-02 09:36:39 2024-03-02 09:58:17 0:21:38 0:11:27 0:10:11 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7579052 2024-03-01 21:00:19 2024-03-02 09:37:00 2024-03-02 10:00:13 0:23:13 0:14:13 0:09:00 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_cephadm} 1
fail 7579053 2024-03-01 21:00:19 2024-03-02 09:37:00 2024-03-02 10:05:23 0:28:23 0:16:46 0:11:37 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-02T09:54:41.609049+0000 mon.smithi019 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579054 2024-03-01 21:00:20 2024-03-02 09:38:21 2024-03-02 13:30:02 3:51:41 3:36:14 0:15:27 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi139 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7579055 2024-03-01 21:00:21 2024-03-02 09:40:52 2024-03-02 10:11:03 0:30:11 0:19:16 0:10:55 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-02T10:01:52.818496+0000 mon.smithi028 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579056 2024-03-01 21:00:22 2024-03-02 09:40:52 2024-03-02 10:56:09 1:15:17 1:04:50 0:10:27 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-02T10:20:00.000250+0000 mon.a (mon.0) 1026 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7579057 2024-03-01 21:00:23 2024-03-02 09:42:23 2024-03-02 10:14:48 0:32:25 0:21:18 0:11:07 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-02T10:03:45.220387+0000 mon.smithi151 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579058 2024-03-01 21:00:24 2024-03-02 09:42:43 2024-03-02 09:57:40 0:14:57 0:05:48 0:09:09 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7579059 2024-03-01 21:00:24 2024-03-02 09:42:44 2024-03-02 10:08:35 0:25:51 0:15:21 0:10:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-02T10:00:39.331957+0000 mon.smithi037 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579060 2024-03-01 21:00:25 2024-03-02 09:43:04 2024-03-02 10:08:11 0:25:07 0:14:12 0:10:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-02T10:00:45.195196+0000 mon.smithi079 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579061 2024-03-01 21:00:26 2024-03-02 09:43:04 2024-03-02 10:12:59 0:29:55 0:19:22 0:10:33 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi135 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm_timeout.py'

fail 7579062 2024-03-01 21:00:27 2024-03-02 09:43:05 2024-03-02 10:10:40 0:27:35 0:14:32 0:13:03 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-02T10:02:20.429839+0000 mon.smithi001 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579063 2024-03-01 21:00:28 2024-03-02 09:45:46 2024-03-02 10:16:50 0:31:04 0:19:49 0:11:15 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-02T10:08:10.943630+0000 mon.smithi066 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579064 2024-03-01 21:00:29 2024-03-02 09:45:46 2024-03-02 10:47:58 1:02:12 0:51:29 0:10:43 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-02T10:20:00.000206+0000 mon.a (mon.0) 2659 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled" in cluster log

fail 7579065 2024-03-01 21:00:29 2024-03-02 09:46:46 2024-03-02 10:31:25 0:44:39 0:31:38 0:13:01 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709374442.4131591 mon.smithi007 (mon.0) 678 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7579066 2024-03-01 21:00:30 2024-03-02 09:50:38 2024-03-02 10:15:48 0:25:10 0:14:09 0:11:01 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7579067 2024-03-01 21:00:31 2024-03-02 09:50:39 2024-03-02 10:19:30 0:28:51 0:18:23 0:10:28 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-02T10:07:24.084131+0000 mon.smithi073 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579068 2024-03-01 21:00:32 2024-03-02 09:50:39 2024-03-02 10:13:28 0:22:49 0:11:41 0:11:08 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-02T10:08:26.508362+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579069 2024-03-01 21:00:33 2024-03-02 09:52:20 2024-03-02 10:52:03 0:59:43 0:48:20 0:11:23 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-02T10:08:07.763310+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7579070 2024-03-01 21:00:34 2024-03-02 09:53:00 2024-03-02 10:16:18 0:23:18 0:13:17 0:10:01 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_extra_daemon_features} 2
fail 7579071 2024-03-01 21:00:34 2024-03-02 09:53:01 2024-03-02 10:37:20 0:44:19 0:31:10 0:13:09 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-02T10:30:00.000146+0000 mon.a (mon.0) 1870 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

fail 7579072 2024-03-01 21:00:35 2024-03-02 09:56:22 2024-03-02 10:21:34 0:25:12 0:15:33 0:09:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-02T10:13:32.037413+0000 mon.smithi106 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579073 2024-03-01 21:00:36 2024-03-02 09:56:22 2024-03-02 10:22:45 0:26:23 0:15:34 0:10:49 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_host_drain} 3
fail 7579074 2024-03-01 21:00:37 2024-03-02 09:57:33 2024-03-02 10:21:00 0:23:27 0:12:45 0:10:42 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-02T10:14:44.667901+0000 mon.smithi088 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579075 2024-03-01 21:00:38 2024-03-02 09:57:33 2024-03-02 10:27:23 0:29:50 0:19:38 0:10:12 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-02T10:18:04.057762+0000 mon.smithi115 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579076 2024-03-01 21:00:38 2024-03-02 09:57:43 2024-03-02 10:21:46 0:24:03 0:13:57 0:10:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-02T10:15:32.646540+0000 mon.smithi069 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579077 2024-03-01 21:00:39 2024-03-02 09:58:24 2024-03-02 10:21:29 0:23:05 0:13:30 0:09:35 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7579078 2024-03-01 21:00:40 2024-03-02 09:58:24 2024-03-02 10:23:54 0:25:30 0:14:42 0:10:48 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-02T10:15:56.063058+0000 mon.smithi077 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579079 2024-03-01 21:00:41 2024-03-02 09:58:45 2024-03-02 11:04:05 1:05:20 0:51:05 0:14:15 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi033 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c1b7363c-d87d-11ee-95c5-87774f69a715 -e sha1=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 -- bash -c 'ceph orch ps'"

fail 7579080 2024-03-01 21:00:42 2024-03-02 09:58:45 2024-03-02 10:16:36 0:17:51 0:07:30 0:10:21 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi057 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7579081 2024-03-01 21:00:42 2024-03-02 09:58:45 2024-03-02 10:31:34 0:32:49 0:22:17 0:10:32 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-02T10:20:23.545620+0000 mon.smithi100 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579082 2024-03-01 21:00:43 2024-03-02 09:59:46 2024-03-02 10:29:20 0:29:34 0:18:51 0:10:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7579083 2024-03-01 21:00:44 2024-03-02 10:00:16 2024-03-02 10:33:59 0:33:43 0:20:37 0:13:06 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-02T10:24:00.016863+0000 mon.smithi027 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579084 2024-03-01 21:00:45 2024-03-02 10:01:17 2024-03-02 10:46:37 0:45:20 0:35:17 0:10:03 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-02T10:19:50.665814+0000 mon.a (mon.0) 174 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7579085 2024-03-01 21:00:46 2024-03-02 10:01:17 2024-03-02 10:45:04 0:43:47 0:31:49 0:11:58 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-03-02T10:40:00.000270+0000 mon.a (mon.0) 1287 : cluster 3 [WRN] OSD_DOWN: 3 osds down" in cluster log

pass 7579086 2024-03-01 21:00:46 2024-03-02 10:03:28 2024-03-02 10:26:50 0:23:22 0:11:54 0:11:28 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7579087 2024-03-01 21:00:47 2024-03-02 10:05:49 2024-03-02 10:31:03 0:25:14 0:15:27 0:09:47 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
dead 7579088 2024-03-01 21:00:48 2024-03-02 10:05:49 2024-03-02 22:13:54 12:08:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

hit max job timeout

pass 7579089 2024-03-01 21:00:49 2024-03-02 10:05:50 2024-03-02 10:36:28 0:30:38 0:20:17 0:10:21 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
fail 7579090 2024-03-01 21:00:50 2024-03-02 10:05:50 2024-03-02 10:28:39 0:22:49 0:11:51 0:10:58 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-02T10:22:32.918778+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7579091 2024-03-01 21:00:50 2024-03-02 10:05:50 2024-03-02 10:28:34 0:22:44 0:12:52 0:09:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_orch_cli} 1
fail 7579092 2024-03-01 21:00:51 2024-03-02 10:05:51 2024-03-02 10:31:55 0:26:04 0:14:47 0:11:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-02T10:24:58.482851+0000 mon.smithi032 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579093 2024-03-01 21:00:52 2024-03-02 10:05:51 2024-03-02 10:57:39 0:51:48 0:40:30 0:11:18 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-03-02T10:23:41.268365+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7579094 2024-03-01 21:00:53 2024-03-02 10:05:51 2024-03-02 10:37:19 0:31:28 0:21:49 0:09:39 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-02T10:26:09.086194+0000 mon.smithi055 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579095 2024-03-01 21:00:54 2024-03-02 10:05:52 2024-03-02 10:30:57 0:25:05 0:13:29 0:11:36 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-02T10:23:36.463932+0000 mon.smithi046 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579096 2024-03-01 21:00:55 2024-03-02 10:05:52 2024-03-02 10:56:46 0:50:54 0:38:51 0:12:03 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
fail 7579097 2024-03-01 21:00:55 2024-03-02 10:06:03 2024-03-02 10:47:47 0:41:44 0:31:31 0:10:13 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709375364.486888 mon.smithi148 (mon.0) 681 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7579098 2024-03-01 21:00:56 2024-03-02 10:06:03 2024-03-02 10:31:52 0:25:49 0:15:26 0:10:23 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-02T10:23:47.793816+0000 mon.smithi107 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579099 2024-03-01 21:00:57 2024-03-02 10:06:03 2024-03-02 11:01:29 0:55:26 0:44:32 0:10:54 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-02T10:40:00.000158+0000 mon.a (mon.0) 3067 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

fail 7579100 2024-03-01 21:00:58 2024-03-02 10:06:34 2024-03-02 10:38:52 0:32:18 0:16:44 0:15:34 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-02T10:29:11.125137+0000 mon.smithi167 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579101 2024-03-01 21:00:59 2024-03-02 10:11:35 2024-03-02 10:38:47 0:27:12 0:13:28 0:13:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7579102 2024-03-01 21:00:59 2024-03-02 10:15:26 2024-03-02 10:49:57 0:34:31 0:24:05 0:10:26 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-02T10:37:36.680482+0000 mon.smithi043 (mon.0) 257 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579103 2024-03-01 21:01:00 2024-03-02 10:15:56 2024-03-02 10:47:42 0:31:46 0:22:41 0:09:05 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7579104 2024-03-01 21:01:01 2024-03-02 10:16:27 2024-03-02 10:43:53 0:27:26 0:13:45 0:13:41 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-02T10:37:55.103954+0000 mon.smithi081 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579105 2024-03-01 21:01:02 2024-03-02 10:21:08 2024-03-02 10:48:26 0:27:18 0:16:29 0:10:49 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7579106 2024-03-01 21:01:03 2024-03-02 10:21:28 2024-03-02 10:58:33 0:37:05 0:24:30 0:12:35 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7579107 2024-03-01 21:01:04 2024-03-02 10:21:28 2024-03-02 10:46:58 0:25:30 0:14:26 0:11:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-02T10:38:36.376755+0000 mon.smithi001 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579108 2024-03-01 21:01:04 2024-03-02 10:21:29 2024-03-02 10:45:09 0:23:40 0:11:55 0:11:45 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-02T10:39:14.520915+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579109 2024-03-01 21:01:05 2024-03-02 10:21:30 2024-03-02 10:48:56 0:27:26 0:15:41 0:11:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-02T10:40:10.286190+0000 mon.smithi006 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579110 2024-03-01 21:01:06 2024-03-02 10:21:30 2024-03-02 11:27:20 1:05:50 0:51:12 0:14:38 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi079 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f18500d0-d880-11ee-95c5-87774f69a715 -e sha1=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 -- bash -c 'ceph orch ps'"

pass 7579111 2024-03-01 21:01:07 2024-03-02 10:21:30 2024-03-02 10:43:14 0:21:44 0:12:05 0:09:39 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_adoption} 1
fail 7579112 2024-03-01 21:01:08 2024-03-02 10:21:31 2024-03-02 10:50:54 0:29:23 0:19:56 0:09:27 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-02T10:41:28.564631+0000 mon.smithi144 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579113 2024-03-01 21:01:09 2024-03-02 10:21:31 2024-03-02 11:33:20 1:11:49 1:00:55 0:10:54 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-02T11:00:00.000281+0000 mon.a (mon.0) 1039 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7579114 2024-03-01 21:01:09 2024-03-02 10:21:32 2024-03-02 10:53:58 0:32:26 0:21:45 0:10:41 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-02T10:42:47.055991+0000 mon.smithi073 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579115 2024-03-01 21:01:10 2024-03-02 10:21:32 2024-03-02 10:47:21 0:25:49 0:14:34 0:11:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-02T10:40:00.568968+0000 mon.smithi088 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579116 2024-03-01 21:01:11 2024-03-02 10:21:32 2024-03-02 11:17:13 0:55:41 0:43:52 0:11:49 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9977dd7c-d880-11ee-95c5-87774f69a715 -e sha1=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7579117 2024-03-01 21:01:12 2024-03-02 10:21:33 2024-03-02 10:43:07 0:21:34 0:11:39 0:09:55 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_ca_signed_key} 2
fail 7579118 2024-03-01 21:01:13 2024-03-02 10:21:34 2024-03-02 10:52:12 0:30:38 0:18:19 0:12:19 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-02T10:40:24.012936+0000 mon.smithi149 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579119 2024-03-01 21:01:13 2024-03-02 10:22:54 2024-03-02 10:49:45 0:26:51 0:16:03 0:10:48 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm} 1
fail 7579120 2024-03-01 21:01:14 2024-03-02 10:22:55 2024-03-02 10:41:27 0:18:32 0:07:21 0:11:11 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} 1
Failure Reason:

Command failed on smithi071 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7579121 2024-03-01 21:01:15 2024-03-02 10:23:55 2024-03-02 10:58:54 0:34:59 0:21:33 0:13:26 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-02T10:48:03.439052+0000 mon.smithi016 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579122 2024-03-01 21:01:16 2024-03-02 10:26:56 2024-03-02 11:29:42 1:02:46 0:51:23 0:11:23 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-02T11:00:00.000227+0000 mon.a (mon.0) 2494 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled" in cluster log

fail 7579123 2024-03-01 21:01:17 2024-03-02 10:28:36 2024-03-02 11:10:46 0:42:10 0:31:57 0:10:13 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709376768.1647508 mon.smithi162 (mon.0) 723 : cluster [WRN] Replacing daemon mds.cephfs.smithi162.vfqneq as rank 0 with standby daemon mds.cephfs.smithi162.urnelc" in cluster log

fail 7579124 2024-03-01 21:01:18 2024-03-02 10:29:27 2024-03-02 11:12:22 0:42:55 0:34:26 0:08:29 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-02T10:47:29.010674+0000 mon.smithi132 (mon.0) 493 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7579125 2024-03-01 21:01:18 2024-03-02 10:29:27 2024-03-02 11:12:51 0:43:24 0:33:37 0:09:47 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7579126 2024-03-01 21:01:19 2024-03-02 10:29:48 2024-03-02 10:50:34 0:20:46 0:11:52 0:08:54 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
fail 7579127 2024-03-01 21:01:20 2024-03-02 10:29:48 2024-03-02 10:53:16 0:23:28 0:13:05 0:10:23 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-02T10:47:09.456065+0000 mon.smithi052 (mon.0) 257 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579128 2024-03-01 21:01:21 2024-03-02 10:29:59 2024-03-02 11:04:59 0:35:00 0:22:53 0:12:07 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7579129 2024-03-01 21:01:22 2024-03-02 10:31:39 2024-03-02 10:56:48 0:25:09 0:14:32 0:10:37 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7579130 2024-03-01 21:01:22 2024-03-02 10:31:40 2024-03-02 10:57:58 0:26:18 0:13:09 0:13:09 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-02T10:50:30.340307+0000 mon.smithi098 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579131 2024-03-01 21:01:23 2024-03-02 10:32:50 2024-03-02 11:00:16 0:27:26 0:18:31 0:08:55 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
fail 7579132 2024-03-01 21:01:24 2024-03-02 10:32:51 2024-03-02 10:58:04 0:25:13 0:11:23 0:13:50 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-02T10:53:44.458549+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579133 2024-03-01 21:01:25 2024-03-02 10:36:31 2024-03-02 10:54:25 0:17:54 0:08:22 0:09:32 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7579134 2024-03-01 21:01:26 2024-03-02 10:37:02 2024-03-02 11:02:47 0:25:45 0:13:28 0:12:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-02T10:56:17.180887+0000 mon.smithi193 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579135 2024-03-01 21:01:26 2024-03-02 10:37:02 2024-03-02 11:17:54 0:40:52 0:30:37 0:10:15 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-02T11:10:00.000130+0000 mon.a (mon.0) 1874 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

fail 7579136 2024-03-01 21:01:27 2024-03-02 10:37:03 2024-03-02 11:06:40 0:29:37 0:20:05 0:09:32 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-02T10:57:11.543837+0000 mon.smithi077 (mon.0) 254 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579137 2024-03-01 21:01:28 2024-03-02 10:37:03 2024-03-02 11:03:06 0:26:03 0:13:46 0:12:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_timeout} 1
dead 7579138 2024-03-01 21:01:29 2024-03-02 10:37:03 2024-03-02 22:45:23 12:08:20 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

hit max job timeout

fail 7579139 2024-03-01 21:01:30 2024-03-02 10:37:04 2024-03-02 10:59:36 0:22:32 0:13:58 0:08:34 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-02T10:53:19.366842+0000 mon.smithi003 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579140 2024-03-01 21:01:30 2024-03-02 10:37:04 2024-03-02 11:02:58 0:25:54 0:14:44 0:11:10 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-02T10:55:57.256377+0000 mon.smithi039 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579141 2024-03-01 21:01:31 2024-03-02 10:37:04 2024-03-02 11:00:05 0:23:01 0:13:33 0:09:28 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
fail 7579142 2024-03-01 21:01:32 2024-03-02 10:37:05 2024-03-02 11:25:59 0:48:54 0:36:43 0:12:11 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi110 with status 126: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7579143 2024-03-01 21:01:33 2024-03-02 10:37:05 2024-03-02 11:25:17 0:48:12 0:37:44 0:10:28 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-03-02T10:59:06.675366+0000 mon.a (mon.0) 475 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7579144 2024-03-01 21:01:34 2024-03-02 10:37:05 2024-03-02 11:09:36 0:32:31 0:21:21 0:11:10 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-02T10:58:37.543522+0000 mon.smithi107 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579145 2024-03-01 21:01:35 2024-03-02 10:37:06 2024-03-02 11:23:46 0:46:40 0:33:30 0:13:10 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-02T11:02:19.639114+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7579146 2024-03-01 21:01:35 2024-03-02 10:38:56 2024-03-02 11:17:05 0:38:09 0:23:00 0:15:09 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
fail 7579147 2024-03-01 21:01:36 2024-03-02 10:42:37 2024-03-02 11:29:32 0:46:55 0:36:31 0:10:24 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-02T11:10:00.000234+0000 mon.a (mon.0) 1152 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

pass 7579148 2024-03-01 21:01:37 2024-03-02 10:42:37 2024-03-02 11:08:02 0:25:25 0:14:51 0:10:34 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7579149 2024-03-01 21:01:38 2024-03-02 10:43:08 2024-03-02 11:13:01 0:29:53 0:14:56 0:14:57 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-02T11:05:39.665784+0000 mon.smithi086 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579150 2024-03-01 21:01:39 2024-03-02 10:47:49 2024-03-02 11:11:29 0:23:40 0:12:23 0:11:17 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-02T11:05:00.213907+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579151 2024-03-01 21:01:39 2024-03-02 10:48:29 2024-03-02 11:19:45 0:31:16 0:19:58 0:11:18 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-02T11:11:17.135470+0000 mon.smithi137 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579152 2024-03-01 21:01:40 2024-03-02 10:49:50 2024-03-02 11:17:31 0:27:41 0:15:16 0:12:25 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-02T11:08:53.573870+0000 mon.smithi022 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579153 2024-03-01 21:01:41 2024-03-02 10:50:41 2024-03-02 11:15:21 0:24:40 0:12:52 0:11:48 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7579154 2024-03-01 21:01:42 2024-03-02 10:52:21 2024-03-02 11:25:59 0:33:38 0:23:56 0:09:42 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-02T11:13:15.042554+0000 mon.smithi149 (mon.0) 250 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579155 2024-03-01 21:01:43 2024-03-02 10:52:21 2024-03-02 11:36:35 0:44:14 0:31:41 0:12:33 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709378331.5649815 mon.smithi043 (mon.0) 717 : cluster [WRN] Replacing daemon mds.cephfs.smithi043.tmwksj as rank 0 with standby daemon mds.cephfs.smithi167.znlfjv" in cluster log

fail 7579156 2024-03-01 21:01:43 2024-03-02 10:52:22 2024-03-02 11:18:56 0:26:34 0:17:12 0:09:22 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-02T11:08:45.527744+0000 mon.smithi002 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579157 2024-03-01 21:01:44 2024-03-02 10:52:22 2024-03-02 11:21:42 0:29:20 0:18:50 0:10:30 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7579158 2024-03-01 21:01:45 2024-03-02 10:52:23 2024-03-02 11:17:18 0:24:55 0:13:20 0:11:35 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-02T11:10:16.434826+0000 mon.smithi081 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579159 2024-03-01 21:01:46 2024-03-02 10:52:23 2024-03-02 11:17:38 0:25:15 0:14:25 0:10:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-02T11:09:44.177877+0000 mon.smithi169 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579160 2024-03-01 21:01:47 2024-03-02 10:52:23 2024-03-02 11:53:57 1:01:34 0:51:25 0:10:09 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi038 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 7579161 2024-03-01 21:01:48 2024-03-02 10:52:24 2024-03-02 11:10:58 0:18:34 0:07:30 0:11:04 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} 3
Failure Reason:

Command failed on smithi006 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7579162 2024-03-01 21:01:49 2024-03-02 10:52:24 2024-03-02 11:21:39 0:29:15 0:20:54 0:08:21 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli} 1
fail 7579163 2024-03-01 21:01:49 2024-03-02 10:52:24 2024-03-02 11:24:41 0:32:17 0:21:51 0:10:26 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-02T11:14:27.738262+0000 mon.smithi097 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579164 2024-03-01 21:01:50 2024-03-02 10:52:25 2024-03-02 11:27:59 0:35:34 0:23:43 0:11:51 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7579165 2024-03-01 21:01:51 2024-03-02 10:52:25 2024-03-02 11:16:56 0:24:31 0:13:52 0:10:39 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7579166 2024-03-01 21:01:52 2024-03-02 10:52:25 2024-03-02 11:29:01 0:36:36 0:25:16 0:11:20 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7579167 2024-03-01 21:01:53 2024-03-02 10:52:26 2024-03-02 11:16:12 0:23:46 0:13:17 0:10:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-02T11:09:18.120484+0000 mon.smithi078 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579168 2024-03-01 21:01:53 2024-03-02 10:52:26 2024-03-02 11:13:56 0:21:30 0:12:27 0:09:03 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7579169 2024-03-01 21:01:54 2024-03-02 10:52:26 2024-03-02 11:17:56 0:25:30 0:11:31 0:13:59 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-02T11:13:21.949593+0000 mon.a (mon.0) 249 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579170 2024-03-01 21:01:55 2024-03-02 10:56:47 2024-03-02 11:36:46 0:39:59 0:28:39 0:11:20 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-02T11:15:25.232374+0000 mon.a (mon.0) 475 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7579171 2024-03-01 21:01:56 2024-03-02 10:56:48 2024-03-02 11:35:05 0:38:17 0:23:58 0:14:19 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_orch_cli_mon} 5
fail 7579172 2024-03-01 21:01:57 2024-03-02 10:58:38 2024-03-02 11:25:17 0:26:39 0:14:46 0:11:53 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-02T11:16:42.200304+0000 mon.smithi100 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579173 2024-03-01 21:01:58 2024-03-02 11:00:09 2024-03-02 11:29:14 0:29:05 0:14:03 0:15:02 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-02T11:22:45.986339+0000 mon.smithi027 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

dead 7579174 2024-03-01 21:01:58 2024-03-02 11:03:10 2024-03-02 23:14:03 12:10:53 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

fail 7579175 2024-03-01 21:01:59 2024-03-02 11:05:01 2024-03-02 11:40:56 0:35:55 0:25:25 0:10:30 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-02T11:25:58.036975+0000 mon.smithi120 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579176 2024-03-01 21:02:00 2024-03-02 11:05:01 2024-03-02 11:33:11 0:28:10 0:15:36 0:12:34 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_rgw_multisite} 3
fail 7579177 2024-03-01 21:02:01 2024-03-02 11:07:52 2024-03-02 11:34:01 0:26:09 0:15:03 0:11:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-02T11:25:34.510144+0000 mon.smithi040 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579178 2024-03-01 21:02:02 2024-03-02 11:07:52 2024-03-02 12:16:56 1:09:04 0:58:50 0:10:14 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-02T11:40:00.000192+0000 mon.a (mon.0) 1045 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

pass 7579179 2024-03-01 21:02:03 2024-03-02 11:07:53 2024-03-02 11:39:33 0:31:40 0:21:56 0:09:44 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7579180 2024-03-01 21:02:03 2024-03-02 11:07:53 2024-03-02 11:35:31 0:27:38 0:18:20 0:09:18 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-02T11:27:48.493331+0000 mon.smithi032 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579181 2024-03-01 21:02:04 2024-03-02 11:07:53 2024-03-02 11:32:04 0:24:11 0:13:05 0:11:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-02T11:26:45.163452+0000 mon.smithi039 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579182 2024-03-01 21:02:05 2024-03-02 11:07:54 2024-03-02 11:38:18 0:30:24 0:20:09 0:10:15 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-02T11:29:05.592156+0000 mon.smithi045 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579183 2024-03-01 21:02:06 2024-03-02 11:07:54 2024-03-02 12:10:39 1:02:45 0:52:56 0:09:49 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-02T11:40:00.000168+0000 mon.a (mon.0) 2570 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled" in cluster log

pass 7579184 2024-03-01 21:02:07 2024-03-02 11:07:54 2024-03-02 11:26:30 0:18:36 0:09:25 0:09:11 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_adoption} 1
fail 7579185 2024-03-01 21:02:08 2024-03-02 11:07:55 2024-03-02 11:48:56 0:41:01 0:30:52 0:10:09 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709379037.2805867 mon.smithi099 (mon.0) 683 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

fail 7579186 2024-03-01 21:02:08 2024-03-02 11:07:55 2024-03-02 11:32:56 0:25:01 0:15:15 0:09:46 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-03-02T11:30:00.000143+0000 mon.a (mon.0) 997 : cluster 3 [WRN] CEPHADM_AGENT_DOWN: 2 Cephadm Agent(s) are not reporting. Hosts may be offline" in cluster log

fail 7579187 2024-03-01 21:02:09 2024-03-02 11:07:55 2024-03-02 11:33:59 0:26:04 0:13:49 0:12:15 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-02T11:27:48.038913+0000 mon.smithi016 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579188 2024-03-01 21:02:10 2024-03-02 11:10:56 2024-03-02 11:36:41 0:25:45 0:13:52 0:11:53 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-02T11:30:36.115532+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7579189 2024-03-01 21:02:11 2024-03-02 11:11:37 2024-03-02 11:38:00 0:26:23 0:15:07 0:11:16 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-02T11:29:36.665632+0000 mon.smithi136 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579190 2024-03-01 21:02:12 2024-03-02 11:12:57 2024-03-02 11:37:21 0:24:24 0:13:35 0:10:49 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-02T11:31:16.716641+0000 mon.smithi064 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579191 2024-03-01 21:02:13 2024-03-02 11:14:28 2024-03-02 11:35:45 0:21:17 0:11:25 0:09:52 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7579192 2024-03-01 21:02:14 2024-03-02 11:14:28 2024-03-02 12:29:08 1:14:40 1:01:11 0:13:29 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-03-02T11:34:49.364800+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7579193 2024-03-01 21:02:14 2024-03-02 11:16:59 2024-03-02 12:00:52 0:43:53 0:32:38 0:11:15 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-02T11:36:17.993792+0000 mon.a (mon.0) 176 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7579194 2024-03-01 21:02:15 2024-03-02 11:17:09 2024-03-02 11:46:48 0:29:39 0:20:12 0:09:27 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-02T11:37:09.892993+0000 mon.smithi179 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7579195 2024-03-01 21:02:16 2024-03-02 11:17:10 2024-03-02 11:51:38 0:34:28 0:19:47 0:14:41 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
fail 7579196 2024-03-01 21:02:17 2024-03-02 11:21:11 2024-03-02 11:48:16 0:27:05 0:14:52 0:12:13 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-02T11:39:50.111786+0000 mon.smithi069 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

dead 7579197 2024-03-01 21:02:18 2024-03-02 11:23:21 2024-03-02 23:31:17 12:07:56 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

hit max job timeout

fail 7579198 2024-03-01 21:02:19 2024-03-02 11:23:21 2024-03-02 11:46:08 0:22:47 0:13:57 0:08:50 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-02T11:39:48.436858+0000 mon.smithi037 (mon.0) 258 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7579199 2024-03-01 21:02:20 2024-03-02 11:23:22 2024-03-02 12:32:02 1:08:40 0:56:30 0:12:10 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi042 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 73bb01e6-d889-11ee-95c5-87774f69a715 -e sha1=4c76c50a73f63ba48ccdf0adccce03b00d1d80c7 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''