Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7565605 2024-02-17 17:35:36 2024-02-18 07:36:31 2024-02-18 08:17:47 0:41:16 0:31:25 0:09:51 smithi main centos 8.stream upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} 4
pass 7565606 2024-02-17 17:35:37 2024-02-18 07:36:31 2024-02-18 09:23:52 1:47:21 1:36:07 0:11:14 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} 5
pass 7565607 2024-02-17 17:35:38 2024-02-18 07:37:22 2024-02-18 10:20:18 2:42:56 2:32:31 0:10:25 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} 5
fail 7565608 2024-02-17 17:35:39 2024-02-18 07:38:12 2024-02-18 08:12:04 0:33:52 0:27:01 0:06:51 smithi main rhel 8.6 upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
Failure Reason:

Command failed on smithi064 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8359654-ce33-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'"

fail 7565609 2024-02-17 17:35:40 2024-02-18 07:38:53 2024-02-18 08:10:15 0:31:22 0:21:53 0:09:29 smithi main ubuntu 18.04 upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

Command failed on smithi131 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7fbf4f4-ce32-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7565610 2024-02-17 17:35:41 2024-02-18 07:38:53 2024-02-18 08:59:30 1:20:37 1:09:30 0:11:07 smithi main ubuntu 20.04 upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T07:59:34.437835+0000 mon.a (mon.0) 188 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7565611 2024-02-17 17:35:42 2024-02-18 07:39:04 2024-02-18 09:55:37 2:16:33 2:06:10 0:10:23 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

"2024-02-18T07:58:26.622027+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7565612 2024-02-17 17:35:42 2024-02-18 07:40:04 2024-02-18 08:59:10 1:19:06 1:06:37 0:12:29 smithi main centos 8.stream upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:03:01.556939+0000 mon.a (mon.0) 260 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 7565613 2024-02-17 17:35:43 2024-02-18 07:41:15 2024-02-18 08:21:45 0:40:30 0:25:09 0:15:21 smithi main centos 8.stream upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} 4
pass 7565614 2024-02-17 17:35:44 2024-02-18 07:44:46 2024-02-18 09:38:17 1:53:31 1:41:29 0:12:02 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} 5
pass 7565615 2024-02-17 17:35:45 2024-02-18 07:45:46 2024-02-18 11:03:36 3:17:50 3:07:16 0:10:34 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} 5
fail 7565616 2024-02-17 17:35:46 2024-02-18 07:46:17 2024-02-18 08:22:26 0:36:09 0:23:19 0:12:50 smithi main ubuntu 20.04 upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
Failure Reason:

Command failed on smithi135 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 668c381e-ce34-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'"

fail 7565617 2024-02-17 17:35:47 2024-02-18 07:46:17 2024-02-18 09:09:47 1:23:30 1:15:57 0:07:33 smithi main rhel 8.6 upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:13:57.240721+0000 mon.a (mon.0) 295 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7565618 2024-02-17 17:35:48 2024-02-18 07:46:18 2024-02-18 09:06:11 1:19:53 1:11:18 0:08:35 smithi main rhel 8.6 upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:14:21.277783+0000 mon.a (mon.0) 328 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7565619 2024-02-17 17:35:49 2024-02-18 07:48:48 2024-02-18 08:24:51 0:36:03 0:22:50 0:13:13 smithi main centos 8.stream upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

Command failed on smithi103 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1874576e-ce35-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'"

pass 7565620 2024-02-17 17:35:49 2024-02-18 07:51:09 2024-02-18 08:31:22 0:40:13 0:30:22 0:09:51 smithi main centos 8.stream upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} 4
pass 7565621 2024-02-17 17:35:50 2024-02-18 07:51:40 2024-02-18 09:45:28 1:53:48 1:43:07 0:10:41 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} 5
pass 7565622 2024-02-17 17:35:51 2024-02-18 07:52:00 2024-02-18 10:40:02 2:48:02 2:34:44 0:13:18 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} 5
fail 7565623 2024-02-17 17:35:52 2024-02-18 07:53:21 2024-02-18 09:11:23 1:18:02 1:06:54 0:11:08 smithi main ubuntu 18.04 upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:12:27.171667+0000 mon.a (mon.0) 188 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7565624 2024-02-17 17:35:53 2024-02-18 07:53:21 2024-02-18 10:19:45 2:26:24 2:18:01 0:08:23 smithi main rhel 8.6 upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
Failure Reason:

"2024-02-18T08:21:23.076152+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7565625 2024-02-17 17:35:54 2024-02-18 07:54:32 2024-02-18 08:26:42 0:32:10 0:22:03 0:10:07 smithi main ubuntu 20.04 upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

Command failed on smithi001 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f526118-ce35-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\''

fail 7565626 2024-02-17 17:35:55 2024-02-18 07:54:42 2024-02-18 08:30:07 0:35:25 0:28:23 0:07:02 smithi main rhel 8.6 upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

Command failed on smithi029 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 54dcff66-ce36-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'"

fail 7565627 2024-02-17 17:35:55 2024-02-18 07:54:43 2024-02-18 09:12:10 1:17:27 1:07:29 0:09:58 smithi main centos 8.stream upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:16:54.851329+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7565628 2024-02-17 17:35:56 2024-02-18 07:54:53 2024-02-18 08:34:41 0:39:48 0:23:34 0:16:14 smithi main centos 8.stream upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} 4
Failure Reason:

"2024-02-18T08:26:57.282459+0000 mon.a (mon.0) 642 : cluster [WRN] Health check failed: 1/433 objects unfound (0.231%) (OBJECT_UNFOUND)" in cluster log

pass 7565629 2024-02-17 17:35:57 2024-02-18 08:01:34 2024-02-18 09:44:53 1:43:19 1:24:13 0:19:06 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} 5
pass 7565630 2024-02-17 17:35:58 2024-02-18 08:10:46 2024-02-18 11:26:57 3:16:11 3:05:10 0:11:01 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} 5
fail 7565631 2024-02-17 17:35:59 2024-02-18 08:13:07 2024-02-18 09:38:07 1:25:00 1:14:03 0:10:57 smithi main rhel 8.6 upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

"2024-02-18T08:42:55.018244+0000 mon.a (mon.0) 292 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7565632 2024-02-17 17:36:00 2024-02-18 08:16:58 2024-02-18 08:48:11 0:31:13 0:22:00 0:09:13 smithi main ubuntu 18.04 upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
Failure Reason:

Command failed on smithi133 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37c14f7a-ce38-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'"

dead 7565633 2024-02-17 17:36:01 2024-02-18 08:16:58 2024-02-18 08:18:02 0:01:04 smithi main rhel 8.6 upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi187