Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7066767 2022-10-13 18:24:25 2022-10-15 04:55:02 2022-10-15 08:25:04 3:30:02 3:22:37 0:07:25 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi186 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

fail 7066768 2022-10-13 18:24:26 2022-10-15 04:55:43 2022-10-15 05:16:59 0:21:16 0:09:59 0:11:17 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} 2
Failure Reason:

Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)

fail 7066769 2022-10-13 18:24:27 2022-10-15 04:55:53 2022-10-15 07:48:07 2:52:14 2:40:28 0:11:46 smithi main centos 8.stream fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} 2
Failure Reason:

"2022-10-15T05:13:35.935279+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066770 2022-10-13 18:24:28 2022-10-15 04:56:44 2022-10-15 05:12:36 0:15:52 0:08:32 0:07:20 smithi main rhel 8.4 fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
Failure Reason:

Test failure: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)

fail 7066771 2022-10-13 18:24:30 2022-10-15 04:57:34 2022-10-15 05:21:09 0:23:35 0:11:41 0:11:54 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} 2
Failure Reason:

Test failure: test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins)

fail 7066772 2022-10-13 18:24:31 2022-10-15 04:59:35 2022-10-15 05:33:06 0:33:31 0:26:01 0:07:30 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} 3
Failure Reason:

"2022-10-15T05:12:40.656086+0000 mgr.y (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066773 2022-10-13 18:24:32 2022-10-15 05:00:05 2022-10-15 05:31:12 0:31:07 0:17:57 0:13:10 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} 2
Failure Reason:

"2022-10-15T05:21:06.966186+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066774 2022-10-13 18:24:33 2022-10-15 05:01:16 2022-10-15 05:20:40 0:19:24 0:09:21 0:10:03 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} 2
Failure Reason:

Test failure: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub)

fail 7066775 2022-10-13 18:24:34 2022-10-15 05:01:17 2022-10-15 05:39:52 0:38:35 0:30:17 0:08:18 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2022-10-15T05:25:33.741925+0000 mgr.x (mgr.24100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066776 2022-10-13 18:24:35 2022-10-15 05:02:37 2022-10-15 05:21:13 0:18:36 0:11:18 0:07:18 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} 2
Failure Reason:

Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)

fail 7066777 2022-10-13 18:24:36 2022-10-15 05:03:28 2022-10-15 05:56:41 0:53:13 0:44:18 0:08:55 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
Failure Reason:

"2022-10-15T05:21:20.481685+0000 mgr.y (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066778 2022-10-13 18:24:38 2022-10-15 05:03:28 2022-10-15 05:33:02 0:29:34 0:18:37 0:10:57 smithi main ubuntu 20.04 fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2022-10-15T05:20:34.213663+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066779 2022-10-13 18:24:39 2022-10-15 05:03:48 2022-10-15 05:36:45 0:32:57 0:09:43 0:23:14 smithi main ubuntu 20.04 fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
Failure Reason:

Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity)

fail 7066780 2022-10-13 18:24:40 2022-10-15 05:05:09 2022-10-15 05:28:10 0:23:01 0:15:31 0:07:30 smithi main centos 8.stream fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2022-10-15T05:18:06.254681+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066781 2022-10-13 18:24:41 2022-10-15 05:05:19 2022-10-15 06:13:19 1:08:00 0:45:09 0:22:51 smithi main ubuntu 20.04 fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

"2022-10-15T05:34:04.555667+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066782 2022-10-13 18:24:42 2022-10-15 05:05:30 2022-10-15 06:15:48 1:10:18 1:01:48 0:08:30 smithi main rhel 8.4 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} 2
Failure Reason:

"2022-10-15T05:20:19.468839+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066783 2022-10-13 18:24:43 2022-10-15 05:06:00 2022-10-15 06:09:48 1:03:48 0:55:54 0:07:54 smithi main rhel 8.4 fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/valgrind} 2
Failure Reason:

"2022-10-15T05:34:40.183525+0000 mgr.x (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066784 2022-10-13 18:24:45 2022-10-15 05:06:21 2022-10-15 05:33:56 0:27:35 0:10:06 0:17:29 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} 2
Failure Reason:

Test failure: test_inject_to_empty (tasks.cephfs.test_journal_repair.TestJournalRepair)

fail 7066785 2022-10-13 18:24:46 2022-10-15 05:07:01 2022-10-15 05:56:21 0:49:20 0:30:15 0:19:05 smithi main centos 8.stream fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} 3
Failure Reason:

"2022-10-15T05:33:12.971654+0000 mgr.y (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066786 2022-10-13 18:24:47 2022-10-15 05:08:32 2022-10-15 06:32:05 1:23:33 1:03:05 0:20:28 smithi main ubuntu 20.04 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

"2022-10-15T05:34:34.262034+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066787 2022-10-13 18:24:48 2022-10-15 05:08:52 2022-10-15 05:28:04 0:19:12 0:10:09 0:09:03 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

Command failed on smithi192 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bfe5ffc-4c49-11ed-8437-001a4aab830c -- ceph orch daemon add osd smithi192:vg_nvme/lv_4'

fail 7066788 2022-10-13 18:24:49 2022-10-15 05:10:53 2022-10-15 05:38:44 0:27:51 0:10:40 0:17:11 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} 2
Failure Reason:

Test failure: test_flush (tasks.cephfs.test_flush.TestFlush)

fail 7066789 2022-10-13 18:24:51 2022-10-15 05:10:53 2022-10-15 05:56:22 0:45:29 0:29:53 0:15:36 smithi main centos 8.stream fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

"2022-10-15T05:32:29.804138+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066790 2022-10-13 18:24:52 2022-10-15 05:11:04 2022-10-15 05:32:55 0:21:51 0:09:00 0:12:51 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 5
Failure Reason:

"2022-10-15T05:28:32.647495+0000 mgr.y (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066791 2022-10-13 18:24:53 2022-10-15 05:12:44 2022-10-15 05:45:08 0:32:24 0:20:56 0:11:28 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} 2
Failure Reason:

"2022-10-15T05:28:26.291686+0000 mgr.x (mgr.4137) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066792 2022-10-13 18:24:54 2022-10-15 05:13:45 2022-10-15 05:28:44 0:14:59 0:07:16 0:07:43 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} 2
Failure Reason:

Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull)

fail 7066793 2022-10-13 18:24:55 2022-10-15 05:14:05 2022-10-15 05:36:41 0:22:36 0:10:07 0:12:29 smithi main centos 8.stream fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} 2
Failure Reason:

Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc)

fail 7066794 2022-10-13 18:24:56 2022-10-15 05:15:16 2022-10-15 05:40:28 0:25:12 0:19:12 0:06:00 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

"2022-10-15T05:27:08.813351+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

pass 7066795 2022-10-13 18:24:58 2022-10-15 05:15:36 2022-10-15 05:48:44 0:33:08 0:26:36 0:06:32 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066796 2022-10-13 18:24:59 2022-10-15 05:15:47 2022-10-15 05:46:39 0:30:52 0:15:29 0:15:23 smithi main ubuntu 20.04 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} 3
Failure Reason:

"2022-10-15T05:39:17.201422+0000 mgr.z (mgr.4112) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066797 2022-10-13 18:25:00 2022-10-15 05:20:42 2022-10-15 05:40:08 0:19:26 0:12:08 0:07:18 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} 2
Failure Reason:

"2022-10-15T05:33:10.371176+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066798 2022-10-13 18:25:01 2022-10-15 05:21:12 2022-10-15 05:39:05 0:17:53 0:10:15 0:07:38 smithi main rhel 8.4 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} 2
Failure Reason:

"2022-10-15T05:34:06.590574+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066799 2022-10-13 18:25:02 2022-10-15 05:21:23 2022-10-15 05:42:38 0:21:15 0:10:24 0:10:51 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} 2
Failure Reason:

Test failure: test_client_metrics_and_metadata (tasks.cephfs.test_mds_metrics.TestMDSMetrics)

fail 7066800 2022-10-13 18:25:03 2022-10-15 05:22:23 2022-10-15 05:42:23 0:20:00 0:13:00 0:07:00 smithi main rhel 8.4 fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T05:35:43.973352+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066801 2022-10-13 18:25:05 2022-10-15 05:23:26 2022-10-15 05:43:49 0:20:23 0:08:44 0:11:39 smithi main ubuntu 20.04 fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} 2
Failure Reason:

"2022-10-15T05:38:40.074744+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066802 2022-10-13 18:25:06 2022-10-15 05:23:36 2022-10-15 05:43:43 0:20:07 0:12:39 0:07:28 smithi main rhel 8.4 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T05:36:38.582589+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066803 2022-10-13 18:25:07 2022-10-15 05:24:47 2022-10-15 05:46:26 0:21:39 0:10:23 0:11:16 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} 2
Failure Reason:

Test failure: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2)

fail 7066804 2022-10-13 18:25:08 2022-10-15 05:24:58 2022-10-15 05:51:03 0:26:05 0:16:46 0:09:19 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2022-10-15T05:41:32.053690+0000 mgr.y (mgr.14102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066805 2022-10-13 18:25:09 2022-10-15 05:28:09 2022-10-15 05:49:01 0:20:52 0:15:08 0:05:44 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2022-10-15T05:40:29.483780+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066806 2022-10-13 18:25:10 2022-10-15 05:28:09 2022-10-15 05:47:32 0:19:23 0:12:59 0:06:24 smithi main centos 8.stream fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} 3
Failure Reason:

"2022-10-15T05:40:50.988456+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066807 2022-10-13 18:25:12 2022-10-15 05:28:50 2022-10-15 05:43:51 0:15:01 0:08:20 0:06:41 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} 2
Failure Reason:

Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable)

fail 7066808 2022-10-13 18:25:13 2022-10-15 05:29:00 2022-10-15 06:04:13 0:35:13 0:21:21 0:13:52 smithi main centos 8.stream fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} 2
Failure Reason:

"2022-10-15T05:49:22.648858+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066809 2022-10-13 18:25:14 2022-10-15 05:31:11 2022-10-15 05:46:33 0:15:22 0:08:25 0:06:57 smithi main rhel 8.4 fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)

fail 7066810 2022-10-13 18:25:15 2022-10-15 05:31:21 2022-10-15 06:04:26 0:33:05 0:20:45 0:12:20 smithi main ubuntu 20.04 fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi164 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh'

fail 7066811 2022-10-13 18:25:16 2022-10-15 05:33:02 2022-10-15 05:57:13 0:24:11 0:14:03 0:10:08 smithi main ubuntu 20.04 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} 2
Failure Reason:

"2022-10-15T05:48:18.024594+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066812 2022-10-13 18:25:17 2022-10-15 05:33:02 2022-10-15 05:48:00 0:14:58 0:08:35 0:06:23 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} 2
Failure Reason:

Test failure: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm)

pass 7066813 2022-10-13 18:25:19 2022-10-15 05:33:03 2022-10-15 06:05:56 0:32:53 0:26:04 0:06:49 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066814 2022-10-13 18:25:20 2022-10-15 05:33:13 2022-10-15 11:58:43 6:25:30 6:14:53 0:10:37 smithi main ubuntu 20.04 fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi099 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

fail 7066815 2022-10-13 18:25:21 2022-10-15 05:33:13 2022-10-15 05:58:46 0:25:33 0:13:40 0:11:53 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} 2
Failure Reason:

"2022-10-15T05:51:06.070989+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066816 2022-10-13 18:25:22 2022-10-15 05:34:04 2022-10-15 07:28:47 1:54:43 1:40:41 0:14:02 smithi main ubuntu 20.04 fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2022-10-15T05:53:50.750348+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066817 2022-10-13 18:25:23 2022-10-15 05:34:54 2022-10-15 05:58:34 0:23:40 0:11:30 0:12:10 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 4
Failure Reason:

"2022-10-15T05:52:04.659453+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066818 2022-10-13 18:25:24 2022-10-15 05:36:45 2022-10-15 05:58:26 0:21:41 0:09:54 0:11:47 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} 2
Failure Reason:

Test failure: test_remote_update_df (tasks.cephfs.test_quota.TestQuota)

fail 7066819 2022-10-13 18:25:25 2022-10-15 05:36:55 2022-10-15 05:55:59 0:19:04 0:11:48 0:07:16 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} 3
Failure Reason:

"2022-10-15T05:49:42.610964+0000 mgr.z (mgr.4115) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066820 2022-10-13 18:25:27 2022-10-15 05:36:56 2022-10-15 06:00:32 0:23:36 0:10:04 0:13:32 smithi main ubuntu 20.04 fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} 2
Failure Reason:

Test failure: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots)

fail 7066821 2022-10-13 18:25:28 2022-10-15 05:38:47 2022-10-15 06:07:16 0:28:29 0:16:27 0:12:02 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
Failure Reason:

"2022-10-15T05:57:08.927211+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066822 2022-10-13 18:25:29 2022-10-15 05:39:07 2022-10-15 06:01:32 0:22:25 0:10:34 0:11:51 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} 2
Failure Reason:

Test failure: test_recover_fs_after_fsmap_removal (tasks.cephfs.test_recovery_fs.TestFSRecovery)

fail 7066823 2022-10-13 18:25:30 2022-10-15 05:39:57 2022-10-15 06:00:50 0:20:53 0:14:16 0:06:37 smithi main centos 8.stream fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2022-10-15T05:51:20.073517+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066824 2022-10-13 18:25:31 2022-10-15 05:39:58 2022-10-15 05:59:25 0:19:27 0:12:16 0:07:11 smithi main rhel 8.4 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2022-10-15T05:52:21.780228+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066825 2022-10-13 18:25:32 2022-10-15 05:40:18 2022-10-15 06:07:37 0:27:19 0:19:27 0:07:52 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2022-10-15T05:58:29.881780+0000 mgr.x (mgr.14098) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066826 2022-10-13 18:25:33 2022-10-15 05:40:39 2022-10-15 05:55:36 0:14:57 0:08:05 0:06:52 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} 2
Failure Reason:

Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks)

fail 7066827 2022-10-13 18:25:35 2022-10-15 05:40:39 2022-10-15 06:17:40 0:37:01 0:25:51 0:11:10 smithi main ubuntu 20.04 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} 2
Failure Reason:

"2022-10-15T05:58:58.682029+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066828 2022-10-13 18:25:36 2022-10-15 05:42:30 2022-10-15 06:39:35 0:57:05 0:46:36 0:10:29 smithi main centos 8.stream fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} 3
Failure Reason:

Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi005 with status 23: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh'

fail 7066829 2022-10-13 18:25:37 2022-10-15 05:42:40 2022-10-15 05:57:33 0:14:53 0:08:46 0:06:07 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} 2
Failure Reason:

Test failure: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap)

fail 7066830 2022-10-13 18:25:38 2022-10-15 05:42:40 2022-10-15 07:30:35 1:47:55 1:37:12 0:10:43 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} 2
Failure Reason:

"2022-10-15T05:57:39.387964+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

pass 7066831 2022-10-13 18:25:39 2022-10-15 05:43:01 2022-10-15 06:19:13 0:36:12 0:27:59 0:08:13 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066832 2022-10-13 18:25:40 2022-10-15 05:43:51 2022-10-15 06:04:40 0:20:49 0:10:01 0:10:48 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} 2
Failure Reason:

Test failure: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules)

fail 7066833 2022-10-13 18:25:41 2022-10-15 05:43:52 2022-10-15 06:04:49 0:20:57 0:14:19 0:06:38 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} 2
Failure Reason:

"2022-10-15T05:55:56.561853+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066834 2022-10-13 18:25:43 2022-10-15 05:43:52 2022-10-15 06:06:27 0:22:35 0:10:27 0:12:08 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} 2
Failure Reason:

Test failure: test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots)

fail 7066835 2022-10-13 18:25:44 2022-10-15 05:45:13 2022-10-15 06:34:43 0:49:30 0:36:56 0:12:34 smithi main ubuntu 20.04 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} 3
Failure Reason:

"2022-10-15T06:04:42.986807+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066836 2022-10-13 18:25:45 2022-10-15 05:46:43 2022-10-15 06:13:56 0:27:13 0:21:09 0:06:04 smithi main rhel 8.4 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} 2
Failure Reason:

"2022-10-15T05:59:21.957310+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066837 2022-10-13 18:25:46 2022-10-15 05:46:44 2022-10-15 06:01:41 0:14:57 0:08:40 0:06:17 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} 2
Failure Reason:

Test failure: test_dir_deletion (tasks.cephfs.test_strays.TestStrays)

fail 7066838 2022-10-13 18:25:47 2022-10-15 05:46:44 2022-10-15 06:06:17 0:19:33 0:12:37 0:06:56 smithi main centos 8.stream fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2022-10-15T05:59:15.567023+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

pass 7066839 2022-10-13 18:25:48 2022-10-15 05:47:35 2022-10-15 06:04:02 0:16:27 0:09:53 0:06:34 smithi main centos 8.stream fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} 1
pass 7066840 2022-10-13 18:25:50 2022-10-15 05:47:35 2022-10-15 06:39:02 0:51:27 0:43:44 0:07:43 smithi main centos 8.stream fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} 2
pass 7066841 2022-10-13 18:25:51 2022-10-15 05:48:06 2022-10-15 06:13:59 0:25:53 0:16:34 0:09:19 smithi main ubuntu 20.04 fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} 1
fail 7066842 2022-10-13 18:25:52 2022-10-15 05:48:46 2022-10-15 06:05:58 0:17:12 0:10:24 0:06:48 smithi main rhel 8.4 fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} 2
Failure Reason:

"2022-10-15T06:01:09.962053+0000 mgr.y (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066843 2022-10-13 18:25:53 2022-10-15 05:49:06 2022-10-15 06:04:06 0:15:00 0:07:47 0:07:13 smithi main rhel 8.4 fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} 1
Failure Reason:

Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring)

pass 7066844 2022-10-13 18:25:54 2022-10-15 05:49:07 2022-10-15 08:30:32 2:41:25 2:34:09 0:07:16 smithi main rhel 8.4 fs/mirror-ha/{begin cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} 1
fail 7066845 2022-10-13 18:25:55 2022-10-15 05:49:17 2022-10-15 06:30:32 0:41:15 0:28:55 0:12:20 smithi main rhel 8.4 fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

"2022-10-15T06:07:25.578909+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066846 2022-10-13 18:25:56 2022-10-15 05:55:39 2022-10-15 06:14:43 0:19:04 0:08:02 0:11:02 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 5
Failure Reason:

Test failure: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop)

fail 7066847 2022-10-13 18:25:57 2022-10-15 05:56:29 2022-10-15 06:13:51 0:17:22 0:07:09 0:10:13 smithi main ubuntu 20.04 fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)

fail 7066848 2022-10-13 18:25:59 2022-10-15 05:56:29 2022-10-15 06:17:39 0:21:10 0:10:42 0:10:28 smithi main ubuntu 20.04 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2022-10-15T06:10:42.982307+0000 mgr.x (mgr.4124) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066849 2022-10-13 18:26:00 2022-10-15 05:56:50 2022-10-15 06:15:28 0:18:38 0:07:03 0:11:35 smithi main ubuntu 20.04 fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} 2
Failure Reason:

Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)

fail 7066850 2022-10-13 18:26:01 2022-10-15 05:57:20 2022-10-15 06:44:27 0:47:07 0:39:32 0:07:35 smithi main rhel 8.4 fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

"2022-10-15T06:09:44.539665+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066851 2022-10-13 18:26:02 2022-10-15 05:57:41 2022-10-15 06:10:04 0:12:23 0:06:37 0:05:46 smithi main centos 8.stream fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop} 1
Failure Reason:

Test failure: test_fstop (tasks.cephfs.test_fstop.TestFSTop)

fail 7066852 2022-10-13 18:26:03 2022-10-15 05:57:41 2022-10-15 06:26:26 0:28:45 0:17:50 0:10:55 smithi main ubuntu 20.04 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} 2
Failure Reason:

"2022-10-15T06:13:46.860680+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066853 2022-10-13 18:26:04 2022-10-15 05:58:32 2022-10-15 06:34:01 0:35:29 0:27:42 0:07:47 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2022-10-15T06:17:07.568210+0000 mgr.x (mgr.14108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066854 2022-10-13 18:26:05 2022-10-15 05:58:42 2022-10-15 06:22:58 0:24:16 0:19:01 0:05:15 smithi main centos 8.stream fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} 1
Failure Reason:

Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring)

fail 7066855 2022-10-13 18:26:06 2022-10-15 05:58:42 2022-10-15 06:13:38 0:14:56 0:07:43 0:07:13 smithi main centos 8.stream fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volume-client} 2
Failure Reason:

Test failure: test_15303 (tasks.cephfs.test_volume_client.TestVolumeClient)

fail 7066856 2022-10-13 18:26:07 2022-10-15 05:58:53 2022-10-15 07:50:41 1:51:48 1:43:30 0:08:18 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

"2022-10-15T06:11:14.038801+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066857 2022-10-13 18:26:09 2022-10-15 05:59:33 2022-10-15 06:15:51 0:16:18 0:08:35 0:07:43 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} 2
Failure Reason:

Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration)

fail 7066858 2022-10-13 18:26:10 2022-10-15 06:00:34 2022-10-15 07:06:23 1:05:49 0:55:27 0:10:22 smithi main ubuntu 20.04 fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} 2
Failure Reason:

"2022-10-15T06:31:08.134915+0000 mgr.x (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066859 2022-10-13 18:26:11 2022-10-15 06:00:54 2022-10-15 06:28:47 0:27:53 0:20:49 0:07:04 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} 3
Failure Reason:

"2022-10-15T06:14:25.169911+0000 mgr.z (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066860 2022-10-13 18:26:12 2022-10-15 06:01:45 2022-10-15 06:26:47 0:25:02 0:11:59 0:13:03 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} 2
Failure Reason:

"2022-10-15T06:21:31.527275+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066861 2022-10-13 18:26:13 2022-10-15 06:04:05 2022-10-15 07:28:18 1:24:13 1:12:08 0:12:05 smithi main centos 8.stream fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi121 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

pass 7066862 2022-10-13 18:26:14 2022-10-15 06:04:16 2022-10-15 06:37:38 0:33:22 0:26:44 0:06:38 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066863 2022-10-13 18:26:16 2022-10-15 06:04:36 2022-10-15 06:30:00 0:25:24 0:14:43 0:10:41 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} 2
Failure Reason:

"2022-10-15T06:21:31.429667+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066864 2022-10-13 18:26:17 2022-10-15 06:04:46 2022-10-15 06:27:51 0:23:05 0:16:33 0:06:32 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

"2022-10-15T06:16:48.179782+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066865 2022-10-13 18:26:18 2022-10-15 06:04:57 2022-10-15 06:24:57 0:20:00 0:12:18 0:07:42 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} 2
Failure Reason:

"2022-10-15T06:17:59.771985+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066866 2022-10-13 18:26:19 2022-10-15 06:05:57 2022-10-15 07:37:08 1:31:11 1:24:39 0:06:32 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} 3
Failure Reason:

"2022-10-15T06:18:44.466887+0000 mgr.z (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066867 2022-10-13 18:26:20 2022-10-15 06:06:08 2022-10-15 06:27:45 0:21:37 0:12:36 0:09:01 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} 2
Failure Reason:

"2022-10-15T06:22:44.690400+0000 mgr.y (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066868 2022-10-13 18:26:21 2022-10-15 06:06:28 2022-10-15 06:21:11 0:14:43 0:08:14 0:06:29 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} 1
Failure Reason:

"2022-10-15T06:19:01.287268+0000 mgr.x (mgr.14101) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066869 2022-10-13 18:26:22 2022-10-15 06:06:29 2022-10-15 06:23:01 0:16:32 0:08:37 0:07:55 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} 2
Failure Reason:

Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)

fail 7066870 2022-10-13 18:26:23 2022-10-15 06:07:19 2022-10-15 06:26:31 0:19:12 0:12:50 0:06:22 smithi main rhel 8.4 fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T06:19:38.532630+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066871 2022-10-13 18:26:25 2022-10-15 06:07:40 2022-10-15 06:28:34 0:20:54 0:12:22 0:08:32 smithi main ubuntu 20.04 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T06:21:26.859705+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066872 2022-10-13 18:26:26 2022-10-15 06:07:40 2022-10-15 06:36:57 0:29:17 0:17:11 0:12:06 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2022-10-15T06:27:06.897515+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066873 2022-10-13 18:26:27 2022-10-15 06:09:51 2022-10-15 06:36:33 0:26:42 0:10:17 0:16:25 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} 2
Failure Reason:

Test failure: test_dump_subtrees (tasks.cephfs.test_admin.TestAdminCommandDumpTree)

fail 7066874 2022-10-13 18:26:28 2022-10-15 06:13:22 2022-10-15 06:28:29 0:15:07 0:08:23 0:06:44 smithi main rhel 8.4 fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} 2
Failure Reason:

Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)

fail 7066875 2022-10-13 18:26:29 2022-10-15 06:13:42 2022-10-15 08:00:48 1:47:06 1:35:29 0:11:37 smithi main centos 8.stream fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2022-10-15T06:32:37.352068+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066876 2022-10-13 18:26:30 2022-10-15 06:13:53 2022-10-15 06:35:50 0:21:57 0:11:49 0:10:08 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 4
Failure Reason:

"2022-10-15T06:30:16.378003+0000 mgr.y (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066877 2022-10-13 18:26:31 2022-10-14 12:33:03 2022-10-14 13:00:25 0:27:22 0:16:51 0:10:31 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} 2
Failure Reason:

"2022-10-14T12:47:38.261270+0000 mgr.y (mgr.4124) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066878 2022-10-13 18:26:33 2022-10-15 06:14:03 2022-10-15 07:06:04 0:52:01 0:40:34 0:11:27 smithi main centos 8.stream fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} 3
Failure Reason:

"2022-10-15T06:32:55.773373+0000 mgr.z (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066879 2022-10-13 18:26:34 2022-10-15 06:14:54 2022-10-15 06:37:00 0:22:06 0:10:45 0:11:21 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} 2
Failure Reason:

Test failure: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool)

fail 7066880 2022-10-13 18:26:35 2022-10-15 06:14:54 2022-10-15 06:31:23 0:16:29 0:08:38 0:07:51 smithi main centos 8.stream fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} 2
Failure Reason:

Test failure: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps)

fail 7066881 2022-10-13 18:26:36 2022-10-15 06:15:34 2022-10-15 06:31:22 0:15:48 0:08:21 0:07:27 smithi main rhel 8.4 fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
Failure Reason:

Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity)

fail 7066882 2022-10-13 18:26:37 2022-10-15 06:15:55 2022-10-15 07:05:43 0:49:48 0:43:06 0:06:42 smithi main rhel 8.4 fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

"2022-10-15T06:28:49.946165+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066883 2022-10-13 18:26:38 2022-10-15 06:15:55 2022-10-15 07:20:23 1:04:28 0:52:42 0:11:46 smithi main ubuntu 20.04 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} 2
Failure Reason:

"2022-10-15T06:31:16.877492+0000 mgr.y (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066884 2022-10-13 18:26:39 2022-10-15 06:16:56 2022-10-15 06:32:37 0:15:41 0:08:52 0:06:49 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree} 2
Failure Reason:

Test failure: test_basic (tasks.cephfs.test_dump_tree.TestDumpTree)

fail 7066885 2022-10-13 18:26:40 2022-10-15 06:17:46 2022-10-15 06:42:36 0:24:50 0:14:50 0:10:00 smithi main ubuntu 20.04 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} 2
Failure Reason:

"2022-10-15T06:34:06.806955+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066886 2022-10-13 18:26:42 2022-10-15 06:17:47 2022-10-15 12:45:29 6:27:42 6:15:23 0:12:19 smithi main centos 8.stream fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi012 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

fail 7066887 2022-10-13 18:26:43 2022-10-15 06:19:17 2022-10-15 06:38:49 0:19:32 0:08:33 0:10:59 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair} 2
Failure Reason:

Test failure: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair)

fail 7066888 2022-10-13 18:26:44 2022-10-15 06:23:08 2022-10-15 07:42:15 1:19:07 1:10:54 0:08:13 smithi main ubuntu 18.04 fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} 2
Failure Reason:

"2022-10-15T06:40:50.728493+0000 mgr.y (mgr.14109) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066889 2022-10-13 18:26:45 2022-10-15 06:23:09 2022-10-15 06:43:51 0:20:42 0:13:18 0:07:24 smithi main rhel 8.4 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} 2
Failure Reason:

"2022-10-15T06:36:57.372125+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066890 2022-10-13 18:26:46 2022-10-15 06:24:59 2022-10-15 06:50:37 0:25:38 0:14:35 0:11:03 smithi main ubuntu 20.04 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} 3
Failure Reason:

"2022-10-15T06:43:18.963125+0000 mgr.z (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066891 2022-10-13 18:26:47 2022-10-15 06:26:40 2022-10-15 06:47:25 0:20:45 0:13:17 0:07:28 smithi main centos 8.stream fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2022-10-15T06:37:54.908982+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066892 2022-10-13 18:26:48 2022-10-15 06:26:51 2022-10-15 06:47:06 0:20:15 0:09:04 0:11:11 smithi main ubuntu 20.04 fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} 2
Failure Reason:

"2022-10-15T06:42:03.178648+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066893 2022-10-13 18:26:49 2022-10-15 06:27:51 2022-10-15 06:46:59 0:19:08 0:12:14 0:06:54 smithi main rhel 8.4 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2022-10-15T06:40:02.752483+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066894 2022-10-13 18:26:50 2022-10-15 06:28:02 2022-10-15 06:49:20 0:21:18 0:10:04 0:11:14 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace} 2
Failure Reason:

Test failure: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace)

fail 7066895 2022-10-13 18:26:51 2022-10-15 06:28:32 2022-10-15 07:07:12 0:38:40 0:33:05 0:05:35 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} 2
Failure Reason:

"2022-10-15T06:40:22.147134+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066896 2022-10-13 18:26:53 2022-10-15 06:28:43 2022-10-15 06:50:13 0:21:30 0:10:31 0:10:59 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush} 2
Failure Reason:

Test failure: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush)

pass 7066897 2022-10-13 18:26:54 2022-10-15 06:28:53 2022-10-15 07:01:48 0:32:55 0:27:10 0:05:45 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066898 2022-10-13 18:26:55 2022-10-15 06:28:53 2022-10-15 08:36:52 2:07:59 1:56:37 0:11:22 smithi main centos 8.stream fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} 2
Failure Reason:

"2022-10-15T06:46:46.391738+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066899 2022-10-13 18:26:56 2022-10-15 06:30:04 2022-10-15 06:51:23 0:21:19 0:09:53 0:11:26 smithi main centos 8.stream fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
Failure Reason:

Test failure: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)

fail 7066900 2022-10-13 18:26:57 2022-10-15 06:30:34 2022-10-15 06:47:01 0:16:27 0:07:20 0:09:07 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} 2
Failure Reason:

Test failure: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits)

fail 7066901 2022-10-13 18:26:58 2022-10-15 06:31:25 2022-10-15 06:51:13 0:19:48 0:13:20 0:06:28 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}} 3
Failure Reason:

"2022-10-15T06:44:39.950462+0000 mgr.x (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066902 2022-10-13 18:27:00 2022-10-15 06:32:15 2022-10-15 06:47:43 0:15:28 0:08:55 0:06:33 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead} 2
Failure Reason:

Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)

fail 7066903 2022-10-13 18:27:01 2022-10-15 06:32:46 2022-10-15 06:55:37 0:22:51 0:13:34 0:09:17 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} 2
Failure Reason:

"2022-10-15T06:46:35.760833+0000 mgr.y (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066904 2022-10-13 18:27:02 2022-10-15 06:34:06 2022-10-15 07:15:59 0:41:53 0:30:46 0:11:07 smithi main ubuntu 20.04 fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

"2022-10-15T06:50:58.587637+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066905 2022-10-13 18:27:03 2022-10-15 06:34:07 2022-10-15 07:01:42 0:27:35 0:14:59 0:12:36 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 5
Failure Reason:

"2022-10-15T06:51:18.325018+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066906 2022-10-13 18:27:04 2022-10-15 06:35:58 2022-10-15 07:00:09 0:24:11 0:16:50 0:07:21 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2022-10-15T06:50:08.162932+0000 mgr.x (mgr.14098) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066907 2022-10-13 18:27:05 2022-10-15 06:36:38 2022-10-15 06:58:05 0:21:27 0:09:55 0:11:32 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} 2
Failure Reason:

Test failure: test_abort_conn (tasks.cephfs.test_client_recovery.TestClientRecovery)

fail 7066908 2022-10-13 18:27:06 2022-10-15 06:36:59 2022-10-15 07:29:18 0:52:19 0:41:38 0:10:41 smithi main ubuntu 20.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} 2
Failure Reason:

"2022-10-15T06:54:37.848213+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066909 2022-10-13 18:27:07 2022-10-15 06:37:09 2022-10-15 07:06:57 0:29:48 0:23:16 0:06:32 smithi main rhel 8.4 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} 3
Failure Reason:

"2022-10-15T06:50:29.412670+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066910 2022-10-13 18:27:09 2022-10-15 06:37:40 2022-10-15 07:03:09 0:25:29 0:13:26 0:12:03 smithi main ubuntu 20.04 fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T06:54:10.233600+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066911 2022-10-13 18:27:10 2022-10-15 06:39:00 2022-10-15 07:00:29 0:21:29 0:09:42 0:11:47 smithi main centos 8.stream fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)

fail 7066912 2022-10-13 18:27:11 2022-10-15 06:39:11 2022-10-15 07:02:54 0:23:43 0:14:30 0:09:13 smithi main ubuntu 20.04 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T06:53:46.315521+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066913 2022-10-13 18:27:12 2022-10-15 06:39:41 2022-10-15 07:38:44 0:59:03 0:48:47 0:10:16 smithi main centos 8.stream fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

"2022-10-15T06:56:18.395191+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066914 2022-10-13 18:27:13 2022-10-15 06:39:41 2022-10-15 07:49:34 1:09:53 0:57:53 0:12:00 smithi main ubuntu 20.04 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} 2
Failure Reason:

"2022-10-15T06:56:00.187116+0000 mgr.y (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066915 2022-10-13 18:27:14 2022-10-15 06:42:42 2022-10-15 07:05:04 0:22:22 0:10:55 0:11:27 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} 2
Failure Reason:

Test failure: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage)

fail 7066916 2022-10-13 18:27:15 2022-10-15 06:43:53 2022-10-15 08:15:30 1:31:37 1:21:05 0:10:32 smithi main centos 8.stream fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

"2022-10-15T07:01:45.555032+0000 mgr.y (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066917 2022-10-13 18:27:16 2022-10-15 06:44:33 2022-10-15 07:05:14 0:20:41 0:08:11 0:12:30 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} 2
Failure Reason:

Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)

pass 7066918 2022-10-13 18:27:17 2022-10-15 06:47:04 2022-10-15 07:19:55 0:32:51 0:26:58 0:05:53 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7066919 2022-10-13 18:27:19 2022-10-15 06:47:04 2022-10-15 07:48:09 1:01:05 0:54:16 0:06:49 smithi main centos 8.stream fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} 2
Failure Reason:

"2022-10-15T07:14:39.170858+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066920 2022-10-13 18:27:20 2022-10-15 06:47:15 2022-10-15 14:40:43 7:53:28 7:46:14 0:07:14 smithi main rhel 8.4 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi017 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=60eace5a7cfe4ca229a3196708e80fc826f2cbf6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

fail 7066921 2022-10-13 18:27:21 2022-10-15 06:47:35 2022-10-15 07:04:28 0:16:53 0:09:38 0:07:15 smithi main rhel 8.4 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} 2
Failure Reason:

Test failure: test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins)

fail 7066922 2022-10-13 18:27:22 2022-10-15 06:47:46 2022-10-15 07:17:20 0:29:34 0:12:52 0:16:42 smithi main centos 8.stream fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} 3
Failure Reason:

"2022-10-15T07:12:05.652845+0000 mgr.z (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066923 2022-10-13 18:27:23 2022-10-15 06:50:17 2022-10-15 07:18:41 0:28:24 0:09:51 0:18:33 smithi main ubuntu 20.04 fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} 2
Failure Reason:

Test failure: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc)

fail 7066924 2022-10-13 18:27:24 2022-10-15 06:50:47 2022-10-15 07:27:37 0:36:50 0:18:02 0:18:48 smithi main ubuntu 20.04 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

"2022-10-15T07:15:15.916582+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066926 2022-10-13 18:27:26 2022-10-15 06:50:47 2022-10-15 07:16:06 0:25:19 0:10:04 0:15:15 smithi main centos 8.stream fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} 2
Failure Reason:

Test failure: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub)

fail 7066927 2022-10-13 18:27:27 2022-10-15 06:51:18 2022-10-15 07:18:23 0:27:05 0:21:03 0:06:02 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2022-10-15T07:09:13.116910+0000 mgr.x (mgr.24103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066928 2022-10-13 18:27:28 2022-10-15 06:51:28 2022-10-15 07:19:26 0:27:58 0:10:28 0:17:30 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} 2
Failure Reason:

Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation)

fail 7066929 2022-10-13 18:27:29 2022-10-15 06:55:39 2022-10-15 07:15:25 0:19:46 0:10:05 0:09:41 smithi main rhel 8.4 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} 2
Failure Reason:

"2022-10-15T07:10:37.663823+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066930 2022-10-13 18:27:30 2022-10-15 06:58:10 2022-10-15 07:19:36 0:21:26 0:11:22 0:10:04 smithi main centos 8.stream fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T07:12:02.776737+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066931 2022-10-13 18:27:31 2022-10-15 07:00:31 2022-10-15 07:23:19 0:22:48 0:12:08 0:10:40 smithi main ubuntu 20.04 fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} 2
Failure Reason:

"2022-10-15T07:16:22.148288+0000 mgr.y (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066932 2022-10-13 18:27:32 2022-10-15 07:01:51 2022-10-15 08:43:29 1:41:38 1:35:36 0:06:02 smithi main rhel 8.4 fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2022-10-15T07:14:27.109314+0000 mgr.x (mgr.4117) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066933 2022-10-13 18:27:33 2022-10-15 07:01:52 2022-10-15 07:20:21 0:18:29 0:07:37 0:10:52 smithi main ubuntu 20.04 fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 4
Failure Reason:

Test failure: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop)

fail 7066934 2022-10-13 18:27:35 2022-10-15 07:03:02 2022-10-15 07:25:22 0:22:20 0:11:40 0:10:40 smithi main ubuntu 20.04 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2022-10-15T07:18:59.964556+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066935 2022-10-13 18:27:36 2022-10-15 07:03:13 2022-10-15 07:37:48 0:34:35 0:22:11 0:12:24 smithi main ubuntu 20.04 fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}} 3
Failure Reason:

"2022-10-15T07:21:24.075229+0000 mgr.z (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log

fail 7066936 2022-10-13 18:27:37 2022-10-15 07:04:33 2022-10-15 07:22:23 0:17:50 0:07:26 0:10:24 smithi main ubuntu 20.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} 2
Failure Reason:

Test failure: test_inject_to_empty (tasks.cephfs.test_journal_repair.TestJournalRepair)

fail 7066937 2022-10-13 18:27:38 2022-10-15 07:05:14 2022-10-15 07:29:13 0:23:59 0:15:44 0:08:15 smithi main rhel 8.4 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2022-10-15T07:18:22.447646+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: snap_schedule" in cluster log