Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7332664 2023-07-10 18:50:32 2023-07-10 21:07:07 2023-07-10 21:44:22 0:37:15 0:27:02 0:10:13 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Found coredumps on ubuntu@smithi018.front.sepia.ceph.com

fail 7332666 2023-07-10 18:50:32 2023-07-10 21:07:58 2023-07-10 21:51:58 0:44:00 0:32:53 0:11:07 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} 2
Failure Reason:

"2023-07-10T21:38:10.946840+0000 osd.1 (osd.1) 3 : cluster [WRN] OSD bench result of 131838.733279 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.1. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log

fail 7332668 2023-07-10 18:50:33 2023-07-10 21:08:39 2023-07-10 21:47:54 0:39:15 0:25:13 0:14:02 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

Found coredumps on ubuntu@smithi060.front.sepia.ceph.com

pass 7332670 2023-07-10 18:50:34 2023-07-10 21:10:40 2023-07-10 21:56:41 0:46:01 0:36:02 0:09:59 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} 2
pass 7332672 2023-07-10 18:50:35 2023-07-10 21:11:01 2023-07-10 21:39:05 0:28:04 0:14:49 0:13:15 smithi main ubuntu 20.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} 2
fail 7332674 2023-07-10 18:50:36 2023-07-10 21:11:42 2023-07-10 21:58:06 0:46:24 0:33:56 0:12:28 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Found coredumps on ubuntu@smithi033.front.sepia.ceph.com

fail 7332676 2023-07-10 18:50:36 2023-07-10 21:13:03 2023-07-10 21:33:50 0:20:47 0:08:49 0:11:58 smithi main fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=0040a8bda2cc6ebfdc4c726c9b2054572d16417d

fail 7332679 2023-07-10 18:50:37 2023-07-10 21:15:05 2023-07-10 22:18:56 1:03:51 0:52:29 0:11:22 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} 2
Failure Reason:

"2023-07-10T21:39:42.828291+0000 mon.a (mon.0) 437 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7332680 2023-07-10 18:50:38 2023-07-10 21:15:36 2023-07-10 21:52:46 0:37:10 0:26:56 0:10:14 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Found coredumps on ubuntu@smithi130.front.sepia.ceph.com

pass 7332682 2023-07-10 18:50:39 2023-07-10 21:16:47 2023-07-10 21:56:12 0:39:25 0:31:06 0:08:19 smithi main rhel 8.4 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
fail 7332684 2023-07-10 18:50:40 2023-07-10 21:17:08 2023-07-10 21:53:08 0:36:00 0:24:37 0:11:23 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

Found coredumps on ubuntu@smithi017.front.sepia.ceph.com

pass 7332685 2023-07-10 18:50:40 2023-07-10 21:17:49 2023-07-10 21:44:17 0:26:28 0:14:54 0:11:34 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
pass 7332686 2023-07-10 18:50:41 2023-07-10 21:17:59 2023-07-10 21:42:08 0:24:09 0:13:52 0:10:17 smithi main ubuntu 20.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} 2
pass 7332687 2023-07-10 18:50:42 2023-07-10 21:18:00 2023-07-10 23:33:13 2:15:13 2:03:47 0:11:26 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} 3
fail 7332688 2023-07-10 18:50:43 2023-07-10 21:19:30 2023-07-10 22:03:03 0:43:33 0:32:27 0:11:06 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Found coredumps on ubuntu@smithi005.front.sepia.ceph.com

pass 7332689 2023-07-10 18:50:44 2023-07-10 21:19:31 2023-07-10 21:58:16 0:38:45 0:27:32 0:11:13 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} 2
fail 7332690 2023-07-10 18:50:45 2023-07-10 22:27:19 2023-07-10 23:10:41 0:43:22 0:28:56 0:14:26 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Found coredumps on ubuntu@smithi012.front.sepia.ceph.com

fail 7332691 2023-07-10 18:50:45 2023-07-10 22:28:30 2023-07-10 23:09:31 0:41:01 0:29:15 0:11:46 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

Found coredumps on ubuntu@smithi044.front.sepia.ceph.com

pass 7332692 2023-07-10 18:50:46 2023-07-10 22:28:30 2023-07-11 00:14:46 1:46:16 1:33:59 0:12:17 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} 2
pass 7332693 2023-07-10 18:50:47 2023-07-10 22:29:31 2023-07-10 22:55:26 0:25:55 0:16:31 0:09:24 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
pass 7332694 2023-07-10 18:50:48 2023-07-10 22:29:31 2023-07-10 22:52:46 0:23:15 0:13:54 0:09:21 smithi main centos 8.stream fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} 1
pass 7332695 2023-07-10 18:50:49 2023-07-10 22:29:42 2023-07-10 23:33:45 1:04:03 0:46:56 0:17:07 smithi main centos 8.stream fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} 2
fail 7332696 2023-07-10 18:50:50 2023-07-10 22:30:22 2023-07-10 23:14:27 0:44:05 0:28:19 0:15:46 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Found coredumps on ubuntu@smithi160.front.sepia.ceph.com

pass 7332697 2023-07-10 18:50:51 2023-07-10 22:30:53 2023-07-10 23:12:37 0:41:44 0:31:15 0:10:29 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} 3
fail 7332698 2023-07-10 18:50:51 2023-07-10 22:31:23 2023-07-10 23:06:07 0:34:44 0:22:17 0:12:27 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Found coredumps on ubuntu@smithi016.front.sepia.ceph.com

fail 7332699 2023-07-10 18:50:52 2023-07-10 22:32:54 2023-07-10 22:57:45 0:24:51 0:08:40 0:16:11 smithi main fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=0040a8bda2cc6ebfdc4c726c9b2054572d16417d

pass 7332700 2023-07-10 18:50:53 2023-07-10 22:35:45 2023-07-10 23:13:05 0:37:20 0:27:20 0:10:00 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} 2
fail 7332701 2023-07-10 18:50:54 2023-07-10 22:35:55 2023-07-10 23:13:48 0:37:53 0:25:32 0:12:21 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

Found coredumps on ubuntu@smithi040.front.sepia.ceph.com

fail 7332702 2023-07-10 18:50:55 2023-07-10 22:36:26 2023-07-10 23:08:20 0:31:54 0:23:13 0:08:41 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} 2
Failure Reason:

"2023-07-10T22:55:27.058995+0000 osd.7 (osd.7) 3 : cluster [WRN] OSD bench result of 237158.199697 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log

pass 7332703 2023-07-10 18:50:55 2023-07-10 22:37:16 2023-07-10 22:59:44 0:22:28 0:11:56 0:10:32 smithi main ubuntu 20.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} 2
pass 7332704 2023-07-10 18:50:56 2023-07-10 22:37:27 2023-07-10 23:04:36 0:27:09 0:17:05 0:10:04 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} 2
pass 7332705 2023-07-10 18:50:57 2023-07-10 22:37:37 2023-07-10 23:13:51 0:36:14 0:25:58 0:10:16 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} 3
pass 7332706 2023-07-10 18:50:58 2023-07-10 22:38:08 2023-07-10 23:20:07 0:41:59 0:32:01 0:09:58 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} 2
fail 7332707 2023-07-10 18:50:59 2023-07-10 22:38:48 2023-07-10 23:25:19 0:46:31 0:35:04 0:11:27 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2023-07-10T23:16:43.500842+0000 mon.a (mon.0) 245 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log

pass 7332708 2023-07-10 18:51:00 2023-07-10 22:40:09 2023-07-10 23:13:33 0:33:24 0:21:03 0:12:21 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 5