Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7561652 2024-02-15 17:34:14 2024-02-15 17:43:03 2024-02-15 18:39:37 0:56:34 0:43:48 0:12:46 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down" in cluster log

fail 7561653 2024-02-15 17:34:15 2024-02-15 17:46:24 2024-02-15 18:33:04 0:46:40 0:37:50 0:08:50 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} 1
Failure Reason:

"2024-02-15T18:20:14.544763+0000 mon.a (mon.0) 509 : cluster [WRN] Replacing daemon mds.a.smithi005.lvnvyk as rank 0 with standby daemon mds.user_test_fs.smithi005.shkleu" in cluster log

fail 7561654 2024-02-15 17:34:15 2024-02-15 17:46:25 2024-02-15 18:23:05 0:36:40 0:24:46 0:11:54 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-02-15T18:07:41.653702+0000 mon.a (mon.0) 165 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] is down (out of quorum)" in cluster log

fail 7561655 2024-02-15 17:34:16 2024-02-15 17:46:35 2024-02-15 18:30:17 0:43:42 0:33:28 0:10:14 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-15T18:09:56.279459+0000 mon.smithi026 (mon.0) 362 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7561656 2024-02-15 17:34:17 2024-02-15 17:46:45 2024-02-15 18:23:38 0:36:53 0:27:09 0:09:44 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-02-15T18:21:06.763115+0000 mon.a (mon.0) 458 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

fail 7561657 2024-02-15 17:34:18 2024-02-15 17:46:46 2024-02-15 18:37:57 0:51:11 0:41:49 0:09:22 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:23:04.429740+0000 mon.smithi067 (mon.0) 504 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7561658 2024-02-15 17:34:19 2024-02-15 17:46:46 2024-02-15 18:43:28 0:56:42 0:41:54 0:14:48 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} 5
fail 7561659 2024-02-15 17:34:20 2024-02-15 17:52:07 2024-02-15 18:09:41 0:17:34 0:07:08 0:10:26 smithi main rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1

fail 7561660 2024-02-15 17:34:20 2024-02-15 17:52:08 2024-02-15 18:39:32 0:47:24 0:37:29 0:09:55 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:24:12.690103+0000 mon.smithi084 (mon.0) 507 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7561661 2024-02-15 17:34:21 2024-02-15 17:52:18 2024-02-15 18:29:37 0:37:19 0:25:04 0:12:15 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} 1
fail 7561662 2024-02-15 17:34:22 2024-02-15 17:54:19 2024-02-15 18:42:57 0:48:38 0:35:48 0:12:50 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:29:18.653809+0000 mon.smithi012 (mon.0) 7 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data" in cluster log

fail 7561663 2024-02-15 17:34:23 2024-02-15 17:55:39 2024-02-15 18:31:09 0:35:30 0:25:07 0:10:23 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Command failed on smithi018 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 240ddaae-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7561664 2024-02-15 17:34:24 2024-02-15 17:56:20 2024-02-15 18:42:36 0:46:16 0:34:51 0:11:25 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:34:34.277890+0000 mon.smithi032 (mon.0) 530 : cluster [WRN] daemon mds.cephfs.smithi091.sntctf compat changed unexpectedly" in cluster log

fail 7561665 2024-02-15 17:34:25 2024-02-15 17:57:20 2024-02-15 18:59:32 1:02:12 0:51:05 0:11:07 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} 2
Failure Reason:

"2024-02-15T18:50:00.000176+0000 mon.a (mon.0) 571 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 53/609 objects degraded (8.703%), 12 pgs degraded" in cluster log

fail 7561666 2024-02-15 17:34:25 2024-02-15 17:57:20 2024-02-15 18:38:13 0:40:53 0:31:32 0:09:21 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} 1
Failure Reason:

"2024-02-15T18:25:09.196725+0000 mon.a (mon.0) 500 : cluster [WRN] Replacing daemon mds.a.smithi037.qfhiku as rank 0 with standby daemon mds.user_test_fs.smithi037.wqqjny" in cluster log

fail 7561667 2024-02-15 17:34:26 2024-02-15 17:57:31 2024-02-15 18:32:01 0:34:30 0:24:18 0:10:12 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-02-15T18:18:09.443860+0000 mon.a (mon.0) 165 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.111:3301/0,v1:172.21.15.111:6790/0] is down (out of quorum)" in cluster log

fail 7561668 2024-02-15 17:34:27 2024-02-15 17:57:51 2024-02-15 18:39:38 0:41:47 0:31:45 0:10:02 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-15T18:20:03.827529+0000 mon.smithi043 (mon.0) 356 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7561669 2024-02-15 17:34:28 2024-02-15 17:58:12 2024-02-15 18:53:38 0:55:26 0:42:04 0:13:22 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

expected string or bytes-like object

pass 7561670 2024-02-15 17:34:29 2024-02-15 17:58:12 2024-02-15 18:50:08 0:51:56 0:37:15 0:14:41 smithi main centos 8.stream rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
fail 7561671 2024-02-15 17:34:30 2024-02-15 18:00:43 2024-02-15 18:28:49 0:28:06 0:17:41 0:10:25 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

"2024-02-15T18:27:00.314074+0000 mon.a (mon.0) 466 : cluster [WRN] Health check failed: cephadm background work is paused (CEPHADM_PAUSED)" in cluster log

fail 7561672 2024-02-15 17:34:31 2024-02-15 18:01:53 2024-02-15 18:44:47 0:42:54 0:32:06 0:10:48 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:28:46.677499+0000 mon.smithi033 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7561673 2024-02-15 17:34:31 2024-02-15 18:02:04 2024-02-15 18:49:05 0:47:01 0:34:03 0:12:58 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} 5
fail 7561674 2024-02-15 17:34:32 2024-02-15 18:03:54 2024-02-15 18:22:09 0:18:15 0:07:05 0:11:10 smithi main rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1

fail 7561675 2024-02-15 17:34:33 2024-02-15 18:04:35 2024-02-15 18:47:47 0:43:12 0:31:54 0:11:18 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:32:57.924130+0000 mon.smithi038 (mon.0) 506 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log

pass 7561676 2024-02-15 17:34:34 2024-02-15 18:04:35 2024-02-15 18:34:08 0:29:33 0:18:31 0:11:02 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} 1
fail 7561677 2024-02-15 17:34:35 2024-02-15 18:04:35 2024-02-15 18:47:15 0:42:40 0:31:35 0:11:05 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

"2024-02-15T18:35:14.041354+0000 mon.smithi082 (mon.0) 7 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data" in cluster log

fail 7561678 2024-02-15 17:34:36 2024-02-15 18:05:56 2024-02-15 18:41:52 0:35:56 0:21:59 0:13:57 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Command failed on smithi049 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a823d9dc-cc2f-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''