Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7586680 2024-03-08 15:55:22 2024-03-09 07:23:03 2024-03-09 07:42:04 0:19:01 0:07:38 0:11:23 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} 3
Failure Reason:

Command failed on smithi018 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7586681 2024-03-08 15:55:23 2024-03-09 07:24:34 2024-03-09 07:57:02 0:32:28 0:22:22 0:10:06 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-09T07:46:32.667807+0000 mon.smithi055 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586682 2024-03-08 15:55:24 2024-03-09 07:25:24 2024-03-09 07:47:15 0:21:51 0:11:39 0:10:12 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi120 with status 5: 'sudo systemctl stop ceph-c1128176-dde8-11ee-95c6-87774f69a715@mon.a'

pass 7586683 2024-03-08 15:55:25 2024-03-09 07:25:45 2024-03-09 08:06:43 0:40:58 0:25:36 0:15:22 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
fail 7586684 2024-03-08 15:55:26 2024-03-09 07:25:45 2024-03-09 08:13:49 0:48:04 0:34:57 0:13:07 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-09T07:47:14.990507+0000 mon.smithi134 (mon.0) 516 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7586685 2024-03-08 15:55:27 2024-03-09 07:25:45 2024-03-09 08:11:32 0:45:47 0:34:04 0:11:43 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7586686 2024-03-08 15:55:27 2024-03-09 07:26:36 2024-03-09 07:55:12 0:28:36 0:14:07 0:14:29 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
pass 7586687 2024-03-08 15:55:28 2024-03-09 07:27:17 2024-03-09 08:03:27 0:36:10 0:24:34 0:11:36 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7586688 2024-03-08 15:55:29 2024-03-09 07:27:47 2024-03-09 08:05:50 0:38:03 0:25:47 0:12:16 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
dead 7586689 2024-03-08 15:55:30 2024-03-09 07:29:58 2024-03-09 19:40:43 12:10:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7586690 2024-03-08 15:55:31 2024-03-09 07:31:48 2024-03-09 08:00:48 0:29:00 0:18:39 0:10:21 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
fail 7586691 2024-03-08 15:55:32 2024-03-09 07:33:09 2024-03-09 07:58:50 0:25:41 0:11:32 0:14:09 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-09T07:53:37.936282+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7586692 2024-03-08 15:55:33 2024-03-09 07:33:49 2024-03-09 08:04:32 0:30:43 0:14:57 0:15:46 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_cephadm} 1
fail 7586693 2024-03-08 15:55:33 2024-03-09 07:36:50 2024-03-09 08:03:35 0:26:45 0:16:34 0:10:11 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-09T07:53:30.104475+0000 mon.smithi038 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

dead 7586694 2024-03-08 15:55:34 2024-03-09 07:36:50 2024-03-09 19:46:47 12:09:57 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

fail 7586695 2024-03-08 15:55:35 2024-03-09 07:37:11 2024-03-09 08:06:55 0:29:44 0:19:27 0:10:17 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-09T07:57:58.818761+0000 mon.smithi052 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586696 2024-03-08 15:55:36 2024-03-09 07:37:11 2024-03-09 08:54:56 1:17:45 1:07:27 0:10:18 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-09T07:58:53.085215+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7586697 2024-03-08 15:55:37 2024-03-09 07:37:22 2024-03-09 08:09:24 0:32:02 0:21:29 0:10:33 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-09T07:58:11.830820+0000 mon.smithi086 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586698 2024-03-08 15:55:38 2024-03-09 07:37:22 2024-03-09 07:52:42 0:15:20 0:06:23 0:08:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7586699 2024-03-08 15:55:38 2024-03-09 07:37:22 2024-03-09 08:09:17 0:31:55 0:15:55 0:16:00 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-09T08:01:26.316679+0000 mon.smithi029 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586700 2024-03-08 15:55:39 2024-03-09 07:37:53 2024-03-09 08:03:11 0:25:18 0:14:28 0:10:50 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-09T07:55:19.246019+0000 mon.smithi080 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586701 2024-03-08 15:55:40 2024-03-09 07:37:53 2024-03-09 08:08:23 0:30:30 0:19:15 0:11:15 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_timeout} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94a539707733325322964c6a187c6a1b41be1daf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm_timeout.py'

fail 7586702 2024-03-08 15:55:41 2024-03-09 07:38:34 2024-03-09 08:08:05 0:29:31 0:14:45 0:14:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-09T08:00:36.392865+0000 mon.smithi032 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586703 2024-03-08 15:55:42 2024-03-09 07:38:34 2024-03-09 08:10:57 0:32:23 0:20:31 0:11:52 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-09T08:00:22.082780+0000 mon.smithi044 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586704 2024-03-08 15:55:43 2024-03-09 07:39:14 2024-03-09 08:26:57 0:47:43 0:36:05 0:11:38 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-09T07:58:35.129353+0000 mon.a (mon.0) 176 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7586705 2024-03-08 15:55:44 2024-03-09 07:39:15 2024-03-09 08:22:22 0:43:07 0:31:41 0:11:26 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709971474.3869572 mon.smithi094 (mon.0) 686 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7586706 2024-03-08 15:55:45 2024-03-09 07:41:15 2024-03-09 08:12:12 0:30:57 0:14:26 0:16:31 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7586707 2024-03-08 15:55:46 2024-03-09 07:41:16 2024-03-09 08:12:02 0:30:46 0:17:33 0:13:13 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-09T08:00:41.179010+0000 mon.smithi112 (mon.0) 260 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586708 2024-03-08 15:55:46 2024-03-09 07:41:16 2024-03-09 08:05:35 0:24:19 0:12:04 0:12:15 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-09T08:00:23.255646+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586709 2024-03-08 15:55:47 2024-03-09 07:42:17 2024-03-09 08:45:38 1:03:21 0:48:26 0:14:55 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-09T08:04:55.534623+0000 mon.a (mon.0) 474 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7586710 2024-03-08 15:55:48 2024-03-09 07:42:17 2024-03-09 08:05:02 0:22:45 0:13:35 0:09:10 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_extra_daemon_features} 2
fail 7586711 2024-03-08 15:55:49 2024-03-09 07:42:18 2024-03-09 08:28:32 0:46:14 0:33:16 0:12:58 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-09T08:20:00.000263+0000 mon.a (mon.0) 1556 : cluster 3 [WRN] OSD_DOWN: 2 osds down" in cluster log

fail 7586712 2024-03-08 15:55:50 2024-03-09 07:45:18 2024-03-09 08:12:36 0:27:18 0:15:31 0:11:47 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-09T08:03:06.207061+0000 mon.smithi154 (mon.0) 259 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586713 2024-03-08 15:55:51 2024-03-09 07:45:19 2024-03-09 08:18:29 0:33:10 0:15:56 0:17:14 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_host_drain} 3
fail 7586714 2024-03-08 15:55:51 2024-03-09 07:51:20 2024-03-09 08:14:34 0:23:14 0:13:15 0:09:59 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-09T08:08:47.905318+0000 mon.smithi123 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586715 2024-03-08 15:55:52 2024-03-09 07:51:20 2024-03-09 08:22:26 0:31:06 0:20:48 0:10:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-09T08:14:20.344527+0000 mon.smithi008 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586716 2024-03-08 15:55:53 2024-03-09 07:52:51 2024-03-09 08:23:04 0:30:13 0:15:49 0:14:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-09T08:15:34.016660+0000 mon.smithi002 (mon.0) 293 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586717 2024-03-08 15:55:54 2024-03-09 07:55:22 2024-03-09 08:21:48 0:26:26 0:13:58 0:12:28 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7586718 2024-03-08 15:55:55 2024-03-09 07:55:22 2024-03-09 08:23:15 0:27:53 0:14:20 0:13:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-09T08:17:28.319761+0000 mon.smithi018 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586719 2024-03-08 15:55:56 2024-03-09 07:56:42 2024-03-09 09:05:39 1:08:57 0:54:47 0:14:10 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi001 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51e00cba-dded-11ee-95c6-87774f69a715 -e sha1=94a539707733325322964c6a187c6a1b41be1daf -- bash -c 'ceph orch ps'"

fail 7586720 2024-03-08 15:55:56 2024-03-09 07:56:43 2024-03-09 08:14:11 0:17:28 0:07:24 0:10:04 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi098 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7586721 2024-03-08 15:55:57 2024-03-09 07:56:43 2024-03-09 08:28:08 0:31:25 0:22:15 0:09:10 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-09T08:16:57.286449+0000 mon.smithi064 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586722 2024-03-08 15:55:58 2024-03-09 07:56:43 2024-03-09 08:29:51 0:33:08 0:21:17 0:11:51 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7586723 2024-03-08 15:55:59 2024-03-09 07:56:44 2024-03-09 08:30:42 0:33:58 0:20:39 0:13:19 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-09T08:20:36.976798+0000 mon.smithi027 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586724 2024-03-08 15:56:00 2024-03-09 07:57:54 2024-03-09 08:51:29 0:53:35 0:36:47 0:16:48 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-09T08:30:00.000102+0000 mon.a (mon.0) 1011 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

dead 7586725 2024-03-08 15:56:01 2024-03-09 07:57:55 2024-03-09 08:59:20 1:01:25 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

Error reimaging machines: reached maximum tries (241) after waiting for 3600 seconds

pass 7586726 2024-03-08 15:56:02 2024-03-09 07:58:05 2024-03-09 08:24:45 0:26:40 0:13:44 0:12:56 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/on orchestrator_cli} 2
pass 7586727 2024-03-08 15:56:02 2024-03-09 08:00:16 2024-03-09 08:27:19 0:27:03 0:15:40 0:11:23 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
dead 7586728 2024-03-08 15:56:03 2024-03-09 08:00:16 2024-03-09 20:09:40 12:09:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

hit max job timeout

pass 7586729 2024-03-08 15:56:04 2024-03-09 08:01:57 2024-03-09 08:24:53 0:22:56 0:13:24 0:09:32 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7586730 2024-03-08 15:56:05 2024-03-09 08:01:57 2024-03-09 08:27:44 0:25:47 0:12:28 0:13:19 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-09T08:22:17.300540+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

pass 7586731 2024-03-08 15:56:06 2024-03-09 08:02:18 2024-03-09 08:27:11 0:24:53 0:14:56 0:09:57 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_orch_cli} 1
fail 7586732 2024-03-08 15:56:07 2024-03-09 08:02:18 2024-03-09 08:29:36 0:27:18 0:14:54 0:12:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-09T08:22:23.392731+0000 mon.smithi138 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586733 2024-03-08 15:56:08 2024-03-09 08:02:18 2024-03-09 08:53:36 0:51:18 0:40:27 0:10:51 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-03-09T08:21:00.899578+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7586734 2024-03-08 15:56:09 2024-03-09 08:03:34 2024-03-09 08:36:54 0:33:20 0:21:50 0:11:30 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-09T08:25:42.896674+0000 mon.smithi047 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586735 2024-03-08 15:56:10 2024-03-09 08:04:35 2024-03-09 08:32:03 0:27:28 0:13:19 0:14:09 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-09T08:24:43.919236+0000 mon.smithi005 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586736 2024-03-08 15:56:10 2024-03-09 08:05:06 2024-03-09 08:58:21 0:53:15 0:38:10 0:15:05 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
fail 7586737 2024-03-08 15:56:11 2024-03-09 08:09:17 2024-03-09 08:58:40 0:49:23 0:32:34 0:16:49 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709973600.000111 mon.smithi006 (mon.0) 676 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds" in cluster log

fail 7586738 2024-03-08 15:56:12 2024-03-09 08:11:38 2024-03-09 08:39:22 0:27:44 0:16:15 0:11:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-09T08:30:08.103842+0000 mon.smithi082 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586739 2024-03-08 15:56:13 2024-03-09 08:12:18 2024-03-09 09:00:20 0:48:02 0:37:10 0:10:52 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi149 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94a539707733325322964c6a187c6a1b41be1daf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 7586740 2024-03-08 15:56:14 2024-03-09 08:12:29 2024-03-09 08:46:06 0:33:37 0:17:50 0:15:47 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-09T08:35:18.434691+0000 mon.smithi042 (mon.0) 272 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586741 2024-03-08 15:56:15 2024-03-09 08:12:29 2024-03-09 08:39:18 0:26:49 0:13:55 0:12:54 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7586742 2024-03-08 15:56:16 2024-03-09 08:12:30 2024-03-09 08:48:52 0:36:22 0:24:39 0:11:43 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-09T08:40:00.000349+0000 mon.smithi026 (mon.0) 783 : cluster 4 [ERR] MDS_ALL_DOWN: 1 filesystem is offline" in cluster log

pass 7586743 2024-03-08 15:56:16 2024-03-09 08:12:30 2024-03-09 08:48:31 0:36:01 0:23:52 0:12:09 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7586744 2024-03-08 15:56:17 2024-03-09 08:12:30 2024-03-09 08:38:27 0:25:57 0:13:45 0:12:12 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-09T08:32:15.505761+0000 mon.smithi038 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586745 2024-03-08 15:56:18 2024-03-09 08:12:31 2024-03-09 08:41:19 0:28:48 0:17:38 0:11:10 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7586746 2024-03-08 15:56:19 2024-03-09 08:12:31 2024-03-09 08:46:43 0:34:12 0:24:58 0:09:14 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7586747 2024-03-08 15:56:20 2024-03-09 08:12:31 2024-03-09 08:38:18 0:25:47 0:15:23 0:10:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-09T08:30:49.486008+0000 mon.smithi052 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586748 2024-03-08 15:56:21 2024-03-09 08:12:32 2024-03-09 08:36:27 0:23:55 0:11:42 0:12:13 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-09T08:30:21.388842+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586749 2024-03-08 15:56:22 2024-03-09 08:12:32 2024-03-09 08:42:12 0:29:40 0:15:45 0:13:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-09T08:34:42.030557+0000 mon.smithi112 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586750 2024-03-08 15:56:23 2024-03-09 08:12:33 2024-03-09 09:18:06 1:05:33 0:52:27 0:13:06 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi080 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1ed2a132-ddef-11ee-95c6-87774f69a715 -e sha1=94a539707733325322964c6a187c6a1b41be1daf -- bash -c 'ceph orch ps'"

pass 7586751 2024-03-08 15:56:23 2024-03-09 08:12:33 2024-03-09 08:35:09 0:22:36 0:12:11 0:10:25 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_adoption} 1
fail 7586752 2024-03-08 15:56:24 2024-03-09 08:12:33 2024-03-09 08:48:16 0:35:43 0:20:18 0:15:25 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-09T08:39:11.837555+0000 mon.smithi071 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586753 2024-03-08 15:56:25 2024-03-09 08:46:47 2024-03-09 10:12:48 1:26:01 1:14:18 0:11:43 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-09T09:10:42.779988+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7586754 2024-03-08 15:56:26 2024-03-09 08:48:38 2024-03-09 09:32:23 0:43:45 0:23:57 0:19:48 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-09T09:22:31.905302+0000 mon.smithi063 (mon.0) 279 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586755 2024-03-08 15:56:27 2024-03-09 08:58:30 2024-03-09 09:28:55 0:30:25 0:16:46 0:13:39 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-09T09:21:01.080270+0000 mon.smithi066 (mon.0) 307 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586756 2024-03-08 15:56:27 2024-03-09 08:58:30 2024-03-09 10:03:34 1:05:04 0:53:00 0:12:04 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-03-09T09:17:40.780772+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7586757 2024-03-08 15:56:28 2024-03-09 08:58:31 2024-03-09 09:22:08 0:23:37 0:12:16 0:11:21 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_ca_signed_key} 2
fail 7586758 2024-03-08 15:56:29 2024-03-09 08:59:11 2024-03-09 09:27:58 0:28:47 0:17:52 0:10:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-09T09:17:45.074948+0000 mon.smithi042 (mon.0) 267 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586759 2024-03-08 15:56:30 2024-03-09 08:59:12 2024-03-09 09:23:38 0:24:26 0:16:22 0:08:04 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm} 1
fail 7586760 2024-03-08 15:56:31 2024-03-09 08:59:12 2024-03-09 09:17:02 0:17:50 0:07:24 0:10:26 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} 1
Failure Reason:

Command failed on smithi155 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

fail 7586761 2024-03-08 15:56:32 2024-03-09 08:59:12 2024-03-09 09:30:42 0:31:30 0:21:44 0:09:46 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-09T09:20:00.578660+0000 mon.smithi022 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586762 2024-03-08 15:56:33 2024-03-09 08:59:13 2024-03-09 09:46:05 0:46:52 0:36:01 0:10:51 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-09T09:30:00.000208+0000 mon.a (mon.0) 2371 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log

fail 7586763 2024-03-08 15:56:34 2024-03-09 08:59:13 2024-03-09 09:44:06 0:44:53 0:34:25 0:10:28 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709976383.5575693 mon.smithi006 (mon.0) 747 : cluster [WRN] Replacing daemon mds.cephfs.smithi006.uomglg as rank 0 with standby daemon mds.cephfs.smithi089.hkthoc" in cluster log

fail 7586764 2024-03-08 15:56:34 2024-03-09 08:59:14 2024-03-09 09:46:30 0:47:16 0:36:20 0:10:56 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-03-09T09:20:50.609968+0000 mon.smithi088 (mon.0) 553 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 7586765 2024-03-08 15:56:35 2024-03-09 08:59:14 2024-03-09 09:41:52 0:42:38 0:33:54 0:08:44 smithi main centos 9.stream orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7586766 2024-03-08 15:56:36 2024-03-09 08:59:14 2024-03-09 10:27:07 1:27:53 0:24:57 1:02:56 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
fail 7586767 2024-03-08 15:56:37 2024-03-09 08:59:35 2024-03-09 09:37:27 0:37:52 0:12:36 0:25:16 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-09T09:32:40.159330+0000 mon.smithi002 (mon.0) 266 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586768 2024-03-08 15:56:38 2024-03-09 09:14:27 2024-03-09 09:47:44 0:33:17 0:23:40 0:09:37 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7586769 2024-03-08 15:56:39 2024-03-09 09:14:38 2024-03-09 09:39:32 0:24:54 0:15:10 0:09:44 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7586770 2024-03-08 15:56:39 2024-03-09 09:14:38 2024-03-09 09:47:18 0:32:40 0:13:32 0:19:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-09T09:40:11.744669+0000 mon.smithi081 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586771 2024-03-08 15:56:40 2024-03-09 09:22:10 2024-03-09 09:44:10 0:22:00 0:11:48 0:10:12 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
fail 7586772 2024-03-08 15:56:41 2024-03-09 09:23:40 2024-03-09 09:52:46 0:29:06 0:11:45 0:17:21 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-09T09:47:36.639555+0000 mon.a (mon.0) 193 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586773 2024-03-08 15:56:42 2024-03-09 09:29:52 2024-03-09 09:47:26 0:17:34 0:08:33 0:09:01 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi094 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94a539707733325322964c6a187c6a1b41be1daf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

fail 7586774 2024-03-08 15:56:43 2024-03-09 09:29:52 2024-03-09 09:55:15 0:25:23 0:14:02 0:11:21 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-09T09:48:12.342693+0000 mon.smithi072 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586775 2024-03-08 15:56:44 2024-03-09 09:29:52 2024-03-09 10:12:49 0:42:57 0:32:03 0:10:54 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
fail 7586776 2024-03-08 15:56:44 2024-03-09 09:29:53 2024-03-09 09:59:46 0:29:53 0:20:19 0:09:34 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-09T09:50:21.382999+0000 mon.smithi084 (mon.0) 250 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586777 2024-03-08 15:56:45 2024-03-09 09:29:53 2024-03-09 09:52:58 0:23:05 0:13:23 0:09:42 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_timeout} 1
fail 7586778 2024-03-08 15:56:46 2024-03-09 09:29:53 2024-03-09 09:56:48 0:26:55 0:15:01 0:11:54 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-09T09:49:22.648325+0000 mon.smithi066 (mon.0) 268 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586779 2024-03-08 15:56:47 2024-03-09 09:30:24 2024-03-09 09:55:15 0:24:51 0:13:35 0:11:16 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-09T09:47:58.136833+0000 mon.smithi042 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586780 2024-03-08 15:56:48 2024-03-09 09:30:24 2024-03-09 09:55:43 0:25:19 0:15:13 0:10:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-09T09:48:39.376710+0000 mon.smithi026 (mon.0) 264 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586781 2024-03-08 15:56:49 2024-03-09 09:30:25 2024-03-09 09:55:18 0:24:53 0:14:08 0:10:45 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
Failure Reason:

"2024-03-09T09:51:37.815855+0000 mon.a (mon.0) 26 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log

fail 7586782 2024-03-08 15:56:50 2024-03-09 09:30:25 2024-03-09 10:39:35 1:09:10 0:56:25 0:12:45 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi080 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ce01f8e2-ddf9-11ee-95c7-87774f69a715 -e sha1=94a539707733325322964c6a187c6a1b41be1daf -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''

fail 7586783 2024-03-08 15:56:50 2024-03-09 09:30:25 2024-03-09 10:21:32 0:51:07 0:37:06 0:14:01 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-03-09T10:06:00.500413+0000 osd.3 (osd.3) 3 : cluster [WRN] failed to encode map e107 with expected crc" in cluster log

fail 7586784 2024-03-08 15:56:51 2024-03-09 09:33:36 2024-03-09 10:11:37 0:38:01 0:21:52 0:16:09 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-09T10:01:03.570754+0000 mon.smithi001 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586785 2024-03-08 15:56:52 2024-03-09 09:39:37 2024-03-09 10:23:38 0:44:01 0:32:14 0:11:47 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

"2024-03-09T10:20:00.000255+0000 mon.a (mon.0) 1409 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log

pass 7586786 2024-03-08 15:56:53 2024-03-09 09:41:58 2024-03-09 10:19:11 0:37:13 0:24:05 0:13:08 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
fail 7586787 2024-03-08 15:56:54 2024-03-09 09:45:29 2024-03-09 10:34:39 0:49:10 0:36:59 0:12:11 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-09T10:20:00.000122+0000 mon.a (mon.0) 1579 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub flag(s) set" in cluster log

pass 7586788 2024-03-08 15:56:55 2024-03-09 09:45:29 2024-03-09 10:10:21 0:24:52 0:15:32 0:09:20 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7586789 2024-03-08 15:56:56 2024-03-09 09:45:30 2024-03-09 10:13:17 0:27:47 0:15:40 0:12:07 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-09T10:04:36.274909+0000 mon.smithi053 (mon.0) 265 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586790 2024-03-08 15:56:57 2024-03-09 09:45:30 2024-03-09 10:10:34 0:25:04 0:12:18 0:12:46 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-09T10:04:27.775481+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586791 2024-03-08 15:56:57 2024-03-09 09:45:31 2024-03-09 10:18:09 0:32:38 0:20:49 0:11:49 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-09T10:08:39.400269+0000 mon.smithi146 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586792 2024-03-08 15:56:58 2024-03-09 09:45:31 2024-03-09 10:14:08 0:28:37 0:15:53 0:12:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

"2024-03-09T10:04:31.586762+0000 mon.smithi118 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586793 2024-03-08 15:56:59 2024-03-09 09:45:31 2024-03-09 10:10:22 0:24:51 0:13:43 0:11:08 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
fail 7586794 2024-03-08 15:57:00 2024-03-09 09:45:32 2024-03-09 10:22:40 0:37:08 0:26:25 0:10:43 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

"2024-03-09T10:08:58.940418+0000 mon.smithi067 (mon.0) 252 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586795 2024-03-08 15:57:01 2024-03-09 09:45:32 2024-03-09 10:29:49 0:44:17 0:32:53 0:11:24 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7586796 2024-03-08 15:57:02 2024-03-09 09:45:32 2024-03-09 10:12:36 0:27:04 0:17:23 0:09:41 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

"2024-03-09T10:02:55.453685+0000 mon.smithi097 (mon.0) 269 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586797 2024-03-08 15:57:03 2024-03-09 09:45:33 2024-03-09 10:15:12 0:29:39 0:18:53 0:10:46 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
fail 7586798 2024-03-08 15:57:03 2024-03-09 09:45:33 2024-03-09 10:10:26 0:24:53 0:13:57 0:10:56 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-03-09T10:04:17.223105+0000 mon.smithi046 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586799 2024-03-08 15:57:04 2024-03-09 09:45:34 2024-03-09 10:11:31 0:25:57 0:15:24 0:10:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

"2024-03-09T10:03:46.164736+0000 mon.smithi078 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586800 2024-03-08 15:57:05 2024-03-09 09:45:34 2024-03-09 10:52:55 1:07:21 0:54:26 0:12:55 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-03-09T10:09:40.184088+0000 mon.a (mon.0) 182 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

fail 7586801 2024-03-08 15:57:06 2024-03-09 09:45:34 2024-03-09 10:04:46 0:19:12 0:07:41 0:11:31 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} 3
Failure Reason:

Command failed on smithi087 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7586802 2024-03-08 15:57:07 2024-03-09 09:45:35 2024-03-09 10:17:57 0:32:22 0:21:24 0:10:58 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_orch_cli} 1
fail 7586803 2024-03-08 15:57:08 2024-03-09 09:45:55 2024-03-09 10:18:32 0:32:37 0:21:39 0:10:58 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
Failure Reason:

"2024-03-09T10:09:19.572285+0000 mon.smithi002 (mon.0) 255 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586804 2024-03-08 15:57:09 2024-03-09 09:45:56 2024-03-09 10:20:07 0:34:11 0:24:27 0:09:44 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7586805 2024-03-08 15:57:09 2024-03-09 09:45:56 2024-03-09 10:10:09 0:24:13 0:14:17 0:09:56 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7586806 2024-03-08 15:57:10 2024-03-09 09:45:56 2024-03-09 10:24:41 0:38:45 0:26:28 0:12:17 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
fail 7586807 2024-03-08 15:57:11 2024-03-09 09:45:57 2024-03-09 10:12:58 0:27:01 0:14:17 0:12:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
Failure Reason:

"2024-03-09T10:05:13.686693+0000 mon.smithi120 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586808 2024-03-08 15:57:12 2024-03-09 09:47:48 2024-03-09 10:19:33 0:31:45 0:21:36 0:10:09 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
fail 7586809 2024-03-08 15:57:13 2024-03-09 09:47:48 2024-03-09 10:17:25 0:29:37 0:11:58 0:17:39 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-03-09T10:11:16.337725+0000 mon.a (mon.0) 247 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586810 2024-03-08 15:57:13 2024-03-09 09:54:30 2024-03-09 10:33:46 0:39:16 0:29:07 0:10:09 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-03-09T10:13:07.098771+0000 mon.a (mon.0) 470 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7586811 2024-03-08 15:57:14 2024-03-09 09:54:30 2024-03-09 10:38:11 0:43:41 0:25:11 0:18:30 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_orch_cli_mon} 5
fail 7586812 2024-03-08 15:57:15 2024-03-09 10:01:02 2024-03-09 10:27:42 0:26:40 0:14:45 0:11:55 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-03-09T10:19:46.419772+0000 mon.smithi017 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586813 2024-03-08 15:57:16 2024-03-09 10:01:02 2024-03-09 10:27:42 0:26:40 0:14:56 0:11:44 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
Failure Reason:

"2024-03-09T10:21:11.247838+0000 mon.smithi117 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586814 2024-03-08 15:57:17 2024-03-09 10:01:02 2024-03-09 10:35:29 0:34:27 0:24:06 0:10:21 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi193 with status 126: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94a539707733325322964c6a187c6a1b41be1daf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7586815 2024-03-08 15:57:18 2024-03-09 10:01:03 2024-03-09 10:40:22 0:39:19 0:26:30 0:12:49 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

"2024-03-09T10:24:20.549090+0000 mon.smithi172 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586816 2024-03-08 15:57:18 2024-03-09 10:01:03 2024-03-09 10:27:39 0:26:36 0:15:12 0:11:24 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_rgw_multisite} 3
fail 7586817 2024-03-08 15:57:19 2024-03-09 10:01:04 2024-03-09 10:26:01 0:24:57 0:16:11 0:08:46 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
Failure Reason:

"2024-03-09T10:17:57.459849+0000 mon.smithi032 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586818 2024-03-08 15:57:20 2024-03-09 10:01:04 2024-03-09 11:01:53 1:00:49 0:51:56 0:08:53 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-03-09T10:40:00.000187+0000 mon.a (mon.0) 1736 : cluster 3 [WRN] OSDMAP_FLAGS: nodeep-scrub flag(s) set" in cluster log

pass 7586819 2024-03-08 15:57:21 2024-03-09 10:01:04 2024-03-09 10:34:46 0:33:42 0:24:14 0:09:28 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_set_mon_crush_locations} 3
fail 7586820 2024-03-08 15:57:22 2024-03-09 10:01:05 2024-03-09 10:35:22 0:34:17 0:21:16 0:13:01 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-03-09T10:26:13.086925+0000 mon.smithi029 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586821 2024-03-08 15:57:23 2024-03-09 10:01:05 2024-03-09 10:24:06 0:23:01 0:13:29 0:09:32 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

"2024-03-09T10:18:14.919000+0000 mon.smithi055 (mon.0) 273 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586822 2024-03-08 15:57:23 2024-03-09 10:01:05 2024-03-09 10:33:25 0:32:20 0:21:06 0:11:14 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

"2024-03-09T10:23:12.267482+0000 mon.smithi047 (mon.0) 253 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586823 2024-03-08 15:57:24 2024-03-09 10:01:06 2024-03-09 11:12:16 1:11:10 0:58:43 0:12:27 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
Failure Reason:

"2024-03-09T10:40:00.000216+0000 mon.a (mon.0) 2589 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log

pass 7586824 2024-03-08 15:57:25 2024-03-09 10:01:16 2024-03-09 10:22:28 0:21:12 0:11:28 0:09:44 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_adoption} 1
fail 7586825 2024-03-08 15:57:26 2024-03-09 10:01:16 2024-03-09 10:46:56 0:45:40 0:34:06 0:11:34 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"1709980002.09383 mon.smithi005 (mon.0) 682 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7586826 2024-03-08 15:57:27 2024-03-09 10:01:17 2024-03-09 10:27:35 0:26:18 0:15:38 0:10:40 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7586827 2024-03-08 15:57:28 2024-03-09 10:01:27 2024-03-09 10:28:54 0:27:27 0:14:32 0:12:55 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
Failure Reason:

"2024-03-09T10:21:59.535907+0000 mon.smithi033 (mon.0) 270 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586828 2024-03-08 15:57:29 2024-03-09 10:01:28 2024-03-09 10:26:11 0:24:43 0:12:02 0:12:41 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-09T10:20:14.121696+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log

fail 7586829 2024-03-08 15:57:29 2024-03-09 10:01:28 2024-03-09 10:27:44 0:26:16 0:14:48 0:11:28 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

"2024-03-09T10:19:52.848739+0000 mon.smithi042 (mon.0) 261 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586830 2024-03-08 15:57:30 2024-03-09 10:01:28 2024-03-09 10:27:36 0:26:08 0:13:49 0:12:19 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-09T10:20:07.836891+0000 mon.smithi019 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586831 2024-03-08 15:57:31 2024-03-09 10:01:29 2024-03-09 10:25:21 0:23:52 0:12:50 0:11:02 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7586832 2024-03-08 15:57:32 2024-03-09 10:01:29 2024-03-09 10:39:57 0:38:28 0:27:24 0:11:04 smithi main ubuntu 20.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi143 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3b549cd4-ddfe-11ee-95c7-87774f69a715 -e sha1=94a539707733325322964c6a187c6a1b41be1daf -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | length == 2\'"\'"\'\''

fail 7586833 2024-03-08 15:57:33 2024-03-09 10:01:29 2024-03-09 10:46:16 0:44:47 0:32:42 0:12:05 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-03-09T10:30:00.000246+0000 mon.a (mon.0) 915 : cluster 3 [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7586834 2024-03-08 15:57:34 2024-03-09 10:01:30 2024-03-09 10:34:10 0:32:40 0:22:08 0:10:32 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

"2024-03-09T10:24:47.973938+0000 mon.smithi084 (mon.0) 251 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7586835 2024-03-08 15:57:34 2024-03-09 10:01:30 2024-03-09 10:31:16 0:29:46 0:19:58 0:09:48 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
fail 7586836 2024-03-08 15:57:35 2024-03-09 10:01:31 2024-03-09 10:35:10 0:33:39 0:15:22 0:18:17 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

"2024-03-09T10:26:43.076807+0000 mon.smithi003 (mon.0) 262 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586837 2024-03-08 15:57:36 2024-03-09 10:10:12 2024-03-09 10:35:30 0:25:18 0:15:59 0:09:19 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

"2024-03-09T10:26:56.818797+0000 mon.smithi098 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586838 2024-03-08 15:57:37 2024-03-09 10:10:22 2024-03-09 10:37:59 0:27:37 0:14:58 0:12:39 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-03-09T10:31:43.343769+0000 mon.smithi123 (mon.0) 291 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

fail 7586839 2024-03-08 15:57:38 2024-03-09 10:12:53 2024-03-09 14:08:15 3:55:22 3:39:22 0:16:00 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi100 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94a539707733325322964c6a187c6a1b41be1daf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'