Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7135423 2023-01-24 16:04:07 2023-01-24 16:07:40 2023-01-24 16:31:56 0:24:16 0:12:24 0:11:52 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 5
fail 7135424 2023-01-24 16:04:08 2023-01-24 16:07:41 2023-01-24 16:37:00 0:29:19 0:21:46 0:07:33 smithi main rhel 8.4 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

"2023-01-24T16:22:50.966743+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135425 2023-01-24 16:04:09 2023-01-24 16:08:01 2023-01-24 16:32:04 0:24:03 0:15:59 0:08:04 smithi main rhel 8.4 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi103 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135426 2023-01-24 16:04:10 2023-01-24 16:08:31 2023-01-24 17:38:52 1:30:21 1:23:16 0:07:05 smithi main rhel 8.4 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} 2
Failure Reason:

"2023-01-24T16:24:49.973955+0000 mgr.x (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135427 2023-01-24 16:04:11 2023-01-24 16:09:02 2023-01-24 16:44:17 0:35:15 0:25:49 0:09:26 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{no}} 3
fail 7135428 2023-01-24 16:04:12 2023-01-24 16:11:02 2023-01-24 16:40:25 0:29:23 0:21:59 0:07:24 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

"2023-01-24T16:26:34.726906+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135429 2023-01-24 16:04:13 2023-01-24 16:11:23 2023-01-24 16:53:18 0:41:55 0:28:55 0:13:00 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} 2
Failure Reason:

"2023-01-24T16:37:02.957525+0000 mgr.x (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135430 2023-01-24 16:04:15 2023-01-24 16:11:33 2023-01-24 16:54:50 0:43:17 0:33:01 0:10:16 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135431 2023-01-24 16:04:16 2023-01-24 16:11:43 2023-01-24 16:49:29 0:37:46 0:29:36 0:08:10 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} 2
Failure Reason:

"2023-01-24T16:29:53.226249+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135432 2023-01-24 16:04:17 2023-01-24 16:14:44 2023-01-24 16:54:49 0:40:05 0:19:23 0:20:42 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} 2
fail 7135433 2023-01-24 16:04:18 2023-01-24 16:14:44 2023-01-24 16:46:41 0:31:57 0:22:22 0:09:35 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2023-01-24T16:38:37.554721+0000 mgr.x (mgr.14103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135434 2023-01-24 16:04:19 2023-01-24 16:15:35 2023-01-24 16:38:54 0:23:19 0:17:25 0:05:54 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} 2
Failure Reason:

"2023-01-24T16:31:07.738966+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135435 2023-01-24 16:04:21 2023-01-24 16:15:55 2023-01-24 17:29:34 1:13:39 0:58:55 0:14:44 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}} 3
fail 7135436 2023-01-24 16:04:22 2023-01-24 16:18:16 2023-01-24 16:56:44 0:38:28 0:23:01 0:15:27 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2023-01-24T16:46:17.714743+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135437 2023-01-24 16:04:23 2023-01-24 16:19:16 2023-01-24 16:44:41 0:25:25 0:12:55 0:12:30 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
pass 7135438 2023-01-24 16:04:24 2023-01-24 16:19:17 2023-01-24 16:42:21 0:23:04 0:11:37 0:11:27 smithi main ubuntu 20.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} 2
fail 7135440 2023-01-24 16:04:25 2023-01-24 16:20:08 2023-01-24 16:41:03 0:20:55 0:13:49 0:07:06 smithi main rhel 8.4 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2023-01-24T16:34:47.919969+0000 mgr.y (mgr.4117) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135442 2023-01-24 16:04:26 2023-01-24 16:22:24 2023-01-24 16:53:10 0:30:46 0:15:57 0:14:49 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} 2
pass 7135444 2023-01-24 16:04:28 2023-01-24 16:22:24 2023-01-24 17:05:40 0:43:16 0:33:20 0:09:56 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135446 2023-01-24 16:04:29 2023-01-24 16:22:25 2023-01-24 16:44:29 0:22:04 0:15:09 0:06:55 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
Failure Reason:

"2023-01-24T16:38:02.519911+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135448 2023-01-24 16:04:30 2023-01-24 16:22:46 2023-01-24 16:48:18 0:25:32 0:14:27 0:11:05 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} 2
pass 7135450 2023-01-24 16:04:31 2023-01-24 16:23:06 2023-01-24 16:47:45 0:24:39 0:14:07 0:10:32 smithi main ubuntu 20.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} 2
fail 7135452 2023-01-24 16:04:32 2023-01-24 16:25:47 2023-01-24 16:48:37 0:22:50 0:13:37 0:09:13 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} 2
Failure Reason:

"2023-01-24T16:42:21.412578+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135454 2023-01-24 16:04:34 2023-01-24 16:27:48 2023-01-24 18:35:22 2:07:34 2:01:07 0:06:27 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} 3
fail 7135456 2023-01-24 16:04:35 2023-01-24 16:28:59 2023-01-24 18:07:37 1:38:38 1:30:15 0:08:23 smithi main rhel 8.4 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2023-01-24T16:45:20.260012+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135458 2023-01-24 16:04:36 2023-01-24 16:30:30 2023-01-24 16:56:31 0:26:01 0:16:12 0:09:49 smithi main centos 8.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} 2
Failure Reason:

"2023-01-24T16:49:24.716488+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135460 2023-01-24 16:04:37 2023-01-24 16:32:01 2023-01-24 17:09:11 0:37:10 0:27:42 0:09:28 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2023-01-24T16:55:31.400699+0000 mgr.x (mgr.24103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135462 2023-01-24 16:04:38 2023-01-24 16:32:11 2023-01-24 17:06:32 0:34:21 0:20:21 0:14:00 smithi main rhel 8.4 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep} 2
Failure Reason:

"2023-01-24T16:58:22.865826+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135464 2023-01-24 16:04:39 2023-01-24 16:36:12 2023-01-24 17:08:36 0:32:24 0:18:27 0:13:57 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} 2
Failure Reason:

Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)

pass 7135466 2023-01-24 16:04:41 2023-01-24 16:39:03 2023-01-24 17:05:11 0:26:08 0:13:29 0:12:39 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 4
fail 7135468 2023-01-24 16:04:42 2023-01-24 16:41:54 2023-01-24 17:49:24 1:07:30 0:55:40 0:11:50 smithi main rhel 8.4 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
Failure Reason:

"2023-01-24T17:06:16.148871+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135470 2023-01-24 16:04:43 2023-01-24 16:42:15 2023-01-24 17:12:40 0:30:25 0:18:05 0:12:20 smithi main ubuntu 20.04 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi008 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135472 2023-01-24 16:04:44 2023-01-24 16:42:26 2023-01-24 18:01:47 1:19:21 1:07:06 0:12:15 smithi main centos 8.stream fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
Failure Reason:

"2023-01-24T17:08:49.486599+0000 mgr.y (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135474 2023-01-24 16:04:45 2023-01-24 16:44:36 2023-01-24 17:09:10 0:24:34 0:16:41 0:07:53 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi066 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

pass 7135476 2023-01-24 16:04:47 2023-01-24 16:45:07 2023-01-24 17:27:59 0:42:52 0:33:19 0:09:33 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7135478 2023-01-24 16:04:48 2023-01-24 16:46:48 2023-01-24 17:48:33 1:01:45 0:51:53 0:09:52 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} 2
fail 7135480 2023-01-24 16:04:49 2023-01-24 16:46:49 2023-01-24 17:10:03 0:23:14 0:14:15 0:08:59 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi006 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

pass 7135482 2023-01-24 16:04:50 2023-01-24 16:48:20 2023-01-24 17:16:42 0:28:22 0:17:00 0:11:22 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
fail 7135484 2023-01-24 16:04:51 2023-01-24 16:49:30 2023-01-24 17:10:17 0:20:47 0:15:20 0:05:27 smithi main rhel 8.4 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2023-01-24T17:04:05.587845+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135486 2023-01-24 16:04:52 2023-01-24 16:50:21 2023-01-24 17:44:59 0:54:38 0:44:22 0:10:16 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}} 3
fail 7135488 2023-01-24 16:04:54 2023-01-24 16:53:22 2023-01-24 17:12:22 0:19:00 0:12:26 0:06:34 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} 2
Failure Reason:

"2023-01-24T17:08:10.386845+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135491 2023-01-24 16:04:55 2023-01-24 16:54:53 2023-01-24 17:26:00 0:31:07 0:20:00 0:11:07 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2023-01-24T17:17:01.641144+0000 mgr.x (mgr.14106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135493 2023-01-24 16:04:56 2023-01-24 17:31:14 967 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi039 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 7135495 2023-01-24 16:04:57 2023-01-24 17:46:31 1831 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree} 2
Failure Reason:

"2023-01-24T17:29:47.941731+0000 mgr.x (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

dead 7135496 2023-01-24 16:04:58 2023-01-25 05:14:29 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

hit max job timeout

fail 7135498 2023-01-24 16:04:59 2023-01-24 17:05:47 2023-01-24 17:29:34 0:23:47 0:12:59 0:10:48 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair} 2
Failure Reason:

"2023-01-24T17:24:05.338421+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135500 2023-01-24 16:05:01 2023-01-24 17:06:37 2023-01-24 17:41:44 0:35:07 0:23:35 0:11:32 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} 2
Failure Reason:

"2023-01-24T17:31:00.851540+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135502 2023-01-24 16:05:02 2023-01-24 17:07:48 2023-01-24 19:13:13 2:05:25 1:55:07 0:10:18 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} 3
fail 7135505 2023-01-24 16:05:03 2023-01-24 17:08:39 2023-01-24 17:26:16 0:17:37 0:11:24 0:06:13 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace} 2
Failure Reason:

"2023-01-24T17:23:50.985928+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135507 2023-01-24 16:05:04 2023-01-24 17:09:20 2023-01-24 17:48:30 0:39:10 0:26:55 0:12:15 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2023-01-24T17:34:55.400159+0000 mgr.x (mgr.14103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135509 2023-01-24 16:05:05 2023-01-24 17:10:21 2023-01-24 17:43:37 0:33:16 0:23:37 0:09:39 smithi main centos 8.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} 2
Failure Reason:

"2023-01-24T17:28:19.874638+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135511 2023-01-24 16:05:07 2023-01-24 17:12:21 2023-01-24 17:38:09 0:25:48 0:15:39 0:10:09 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush} 2
fail 7135513 2023-01-24 16:05:08 2023-01-24 17:12:42 2023-01-24 17:48:10 0:35:28 0:25:10 0:10:18 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi149 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135515 2023-01-24 16:05:09 2023-01-24 17:13:13 2023-01-24 17:39:38 0:26:25 0:15:36 0:10:49 smithi main centos 8.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2023-01-24T17:32:18.263433+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135517 2023-01-24 16:05:10 2023-01-24 17:14:44 2023-01-24 17:37:21 0:22:37 0:13:20 0:09:17 smithi main centos 8.stream fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} 1
fail 7135519 2023-01-24 16:05:11 2023-01-24 17:15:24 2023-01-24 17:49:50 0:34:26 0:22:58 0:11:28 smithi main centos 8.stream fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} 2
Failure Reason:

timeout expired in wait_until_healthy

fail 7135521 2023-01-24 16:05:12 2023-01-24 17:16:47 2023-01-24 17:43:06 0:26:19 0:19:17 0:07:02 smithi main rhel 8.4 fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} 1
Failure Reason:

"2023-01-24T17:41:06.705364+0000 mon.a (mon.0) 346 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7135522 2023-01-24 16:05:14 2023-01-24 17:18:07 2023-01-24 17:49:03 0:30:56 0:20:10 0:10:46 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} 2
pass 7135523 2023-01-24 16:05:15 2023-01-24 17:18:27 2023-01-24 17:41:18 0:22:51 0:10:57 0:11:54 smithi main ubuntu 20.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} 2
fail 7135524 2023-01-24 16:05:16 2023-01-24 17:19:38 2023-01-24 19:24:42 2:05:04 1:54:50 0:10:14 smithi main centos 8.stream fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{centos_8} tasks/mirror} 1
Failure Reason:

"2023-01-24T17:38:53.033807+0000 mgr.x (mgr.4667) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135525 2023-01-24 16:05:17 2023-01-24 17:19:38 2023-01-24 17:52:04 0:32:26 0:22:20 0:10:06 smithi main centos 8.stream fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} 1
Failure Reason:

"2023-01-24T17:46:32.722340+0000 mon.a (mon.0) 325 : cluster [WRN] Health check failed: 2 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

fail 7135526 2023-01-24 16:05:18 2023-01-24 17:20:59 2023-01-24 18:13:23 0:52:24 0:41:05 0:11:19 smithi main rhel 8.4 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2023-01-24T17:40:29.980374+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135527 2023-01-24 16:05:19 2023-01-24 17:26:20 2023-01-24 18:26:16 0:59:56 0:48:03 0:11:53 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 5
pass 7135528 2023-01-24 16:05:21 2023-01-24 17:28:00 2023-01-24 18:07:32 0:39:32 0:27:55 0:11:37 smithi main ubuntu 20.04 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
pass 7135529 2023-01-24 16:05:22 2023-01-24 17:29:41 2023-01-24 17:53:51 0:24:10 0:12:59 0:11:11 smithi main ubuntu 20.04 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
pass 7135530 2023-01-24 16:05:23 2023-01-24 17:29:41 2023-01-24 18:15:55 0:46:14 0:35:32 0:10:42 smithi main ubuntu 20.04 fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} 2
fail 7135531 2023-01-24 16:05:24 2023-01-24 17:29:52 2023-01-24 17:51:32 0:21:40 0:13:44 0:07:56 smithi main rhel 8.4 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi050 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135532 2023-01-24 16:05:25 2023-01-24 17:30:12 2023-01-24 17:51:04 0:20:52 0:11:23 0:09:29 smithi main centos 8.stream fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop} 1
Failure Reason:

"2023-01-24T17:47:55.270362+0000 mgr.x (mgr.4511) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135533 2023-01-24 16:05:27 2023-01-24 17:30:12 2023-01-24 18:00:12 0:30:00 0:22:08 0:07:52 smithi main rhel 8.4 fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} 2
Failure Reason:

"2023-01-24T17:45:46.069135+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135534 2023-01-24 16:05:28 2023-01-24 17:31:13 2023-01-24 18:08:38 0:37:25 0:26:33 0:10:52 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2023-01-24T17:54:31.737717+0000 mgr.y (mgr.14098) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135535 2023-01-24 16:05:29 2023-01-24 17:31:43 2023-01-24 19:49:23 2:17:40 2:06:38 0:11:02 smithi main centos 8.stream fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} 1
Failure Reason:

"2023-01-24T17:59:17.726125+0000 mgr.x (mgr.4593) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135536 2023-01-24 16:05:30 2023-01-24 17:31:43 2023-01-24 18:03:22 0:31:39 0:19:15 0:12:24 smithi main centos 8.stream fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} 2
Failure Reason:

"2023-01-24T17:52:50.536700+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135537 2023-01-24 16:05:31 2023-01-24 17:33:44 2023-01-24 18:07:59 0:34:15 0:23:37 0:10:38 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} 3
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi100 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 7135538 2023-01-24 16:05:33 2023-01-24 17:36:45 2023-01-24 17:54:30 0:17:45 0:11:30 0:06:15 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead} 2
Failure Reason:

"2023-01-24T17:52:06.611512+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135539 2023-01-24 16:05:34 2023-01-24 17:37:25 2023-01-24 18:41:33 1:04:08 0:53:41 0:10:27 smithi main ubuntu 20.04 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} 2
fail 7135540 2023-01-24 16:05:35 2023-01-24 17:37:25 2023-01-24 17:59:00 0:21:35 0:13:40 0:07:55 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} 2
Failure Reason:

"2023-01-24T17:52:40.301236+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135541 2023-01-24 16:05:36 2023-01-24 17:38:06 2023-01-24 18:21:18 0:43:12 0:33:28 0:09:44 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135542 2023-01-24 16:05:37 2023-01-24 17:38:16 2023-01-24 18:30:32 0:52:16 0:40:13 0:12:03 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} 2
Failure Reason:

"2023-01-24T18:04:17.403485+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135543 2023-01-24 16:05:39 2023-01-24 17:38:57 2023-01-24 18:07:29 0:28:32 0:18:18 0:10:14 smithi main ubuntu 20.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} 2
fail 7135544 2023-01-24 16:05:40 2023-01-24 17:39:27 2023-01-24 18:10:44 0:31:17 0:20:41 0:10:36 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} 2
Failure Reason:

"2023-01-24T17:57:14.318416+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135545 2023-01-24 16:05:41 2023-01-24 17:39:47 2023-01-24 18:02:23 0:22:36 0:11:33 0:11:03 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} 1
Failure Reason:

"2023-01-24T18:00:12.454194+0000 mgr.y (mgr.14125) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135546 2023-01-24 16:05:42 2023-01-24 17:41:28 2023-01-24 18:14:50 0:33:22 0:26:19 0:07:03 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} 3
pass 7135547 2023-01-24 16:05:43 2023-01-24 17:41:48 2023-01-24 18:19:18 0:37:30 0:25:58 0:11:32 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} 2
fail 7135548 2023-01-24 16:05:45 2023-01-24 17:43:39 2023-01-24 18:18:10 0:34:31 0:26:38 0:07:53 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} 2
Failure Reason:

"2023-01-24T17:59:40.488450+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135549 2023-01-24 16:05:46 2023-01-24 17:45:09 2023-01-24 18:06:53 0:21:44 0:12:20 0:09:24 smithi main centos 8.stream fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} 2
fail 7135550 2023-01-24 16:05:47 2023-01-24 17:45:10 2023-01-24 18:53:53 1:08:43 0:55:37 0:13:06 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} 2
Failure Reason:

"2023-01-24T18:11:36.061337+0000 mgr.x (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135551 2023-01-24 16:05:48 2023-01-24 17:46:20 2023-01-24 18:09:42 0:23:22 0:15:37 0:07:45 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2023-01-24T18:01:00.338291+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135552 2023-01-24 16:05:49 2023-01-24 17:46:40 2023-01-24 18:10:35 0:23:55 0:13:16 0:10:39 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7135553 2023-01-24 16:05:50 2023-01-24 17:46:51 2023-01-24 18:11:03 0:24:12 0:13:53 0:10:19 smithi main centos 8.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2023-01-24T18:05:33.304289+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135554 2023-01-24 16:05:52 2023-01-24 17:48:12 2023-01-24 18:30:44 0:42:32 0:30:38 0:11:54 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} 3
fail 7135555 2023-01-24 16:05:53 2023-01-24 17:48:32 2023-01-24 18:09:34 0:21:02 0:15:25 0:05:37 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} 2
Failure Reason:

"2023-01-24T18:02:55.662256+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135556 2023-01-24 16:05:54 2023-01-24 17:48:42 2023-01-24 18:19:07 0:30:25 0:20:36 0:09:49 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2023-01-24T18:09:52.202812+0000 mgr.x (mgr.14103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135557 2023-01-24 16:05:55 2023-01-24 17:49:13 2023-01-24 18:13:30 0:24:17 0:13:25 0:10:52 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} 2
fail 7135558 2023-01-24 16:05:57 2023-01-24 17:49:33 2023-01-24 18:12:59 0:23:26 0:17:18 0:06:08 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} 2
Failure Reason:

"2023-01-24T18:04:28.222677+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135559 2023-01-24 16:05:58 2023-01-24 17:49:53 2023-01-24 18:13:15 0:23:22 0:11:35 0:11:47 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 4
fail 7135560 2023-01-24 16:05:59 2023-01-24 17:51:34 2023-01-24 18:52:20 1:00:46 0:52:28 0:08:18 smithi main rhel 8.4 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
Failure Reason:

"2023-01-24T18:07:54.268940+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135561 2023-01-24 16:06:00 2023-01-24 17:52:14 2023-01-24 18:15:24 0:23:10 0:14:15 0:08:55 smithi main rhel 8.4 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi154 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135562 2023-01-24 16:06:01 2023-01-24 17:53:55 2023-01-24 18:29:30 0:35:35 0:29:08 0:06:27 smithi main rhel 8.4 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} 2
Failure Reason:

"2023-01-24T18:09:00.365280+0000 mgr.x (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135563 2023-01-24 16:06:03 2023-01-24 17:54:35 2023-01-24 18:26:20 0:31:45 0:19:59 0:11:46 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} 2
Failure Reason:

"2023-01-24T18:18:24.952486+0000 mgr.x (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135564 2023-01-24 16:06:04 2023-01-24 17:54:46 2023-01-24 18:18:18 0:23:32 0:08:43 0:14:49 smithi main fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=596208b4445cdfc3f826189613d9bc0d095a1bb7

fail 7135565 2023-01-24 16:06:05 2023-01-24 17:59:07 2023-01-24 19:14:31 1:15:24 1:05:21 0:10:03 smithi main centos 8.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} 2
Failure Reason:

"2023-01-24T18:16:46.311136+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135566 2023-01-24 16:06:06 2023-01-24 18:00:17 2023-01-24 18:43:04 0:42:47 0:31:24 0:11:23 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135567 2023-01-24 16:06:08 2023-01-24 18:01:58 2023-01-24 18:38:58 0:37:00 0:24:38 0:12:22 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} 2
Failure Reason:

"2023-01-24T18:27:30.275417+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135568 2023-01-24 16:06:09 2023-01-24 18:03:28 2023-01-24 18:38:44 0:35:16 0:25:06 0:10:10 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} 3
Failure Reason:

Command failed (workunit test suites/fsx.sh) on smithi090 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh'

fail 7135569 2023-01-24 16:06:10 2023-01-24 18:06:59 2023-01-24 18:26:43 0:19:44 0:11:49 0:07:55 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} 2
Failure Reason:

"2023-01-24T18:22:06.618433+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135570 2023-01-24 16:06:11 2023-01-24 18:07:29 2023-01-24 18:30:45 0:23:16 0:16:57 0:06:19 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi067 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135571 2023-01-24 16:06:12 2023-01-24 18:07:40 2023-01-24 18:43:07 0:35:27 0:25:29 0:09:58 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2023-01-24T18:30:54.497388+0000 mgr.x (mgr.14101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135572 2023-01-24 16:06:13 2023-01-24 18:08:00 2023-01-24 18:37:56 0:29:56 0:19:32 0:10:24 smithi main ubuntu 20.04 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep} 2
fail 7135573 2023-01-24 16:06:15 2023-01-24 18:08:00 2023-01-24 18:35:37 0:27:37 0:21:20 0:06:17 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} 2
Failure Reason:

"2023-01-24T18:22:37.459753+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135574 2023-01-24 16:06:16 2023-01-24 18:08:41 2023-01-24 18:33:34 0:24:53 0:13:07 0:11:46 smithi main ubuntu 20.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi135 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh'

pass 7135575 2023-01-24 16:06:17 2023-01-24 18:09:41 2023-01-24 18:35:48 0:26:07 0:15:24 0:10:43 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
fail 7135576 2023-01-24 16:06:18 2023-01-24 18:09:51 2023-01-24 18:29:55 0:20:04 0:12:23 0:07:41 smithi main rhel 8.4 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} 2
Failure Reason:

"2023-01-24T18:25:31.775751+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135578 2023-01-24 16:06:19 2023-01-24 18:10:53 2023-01-24 18:31:49 0:20:56 0:14:11 0:06:45 smithi main rhel 8.4 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2023-01-24T18:25:10.255787+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135580 2023-01-24 16:06:20 2023-01-24 18:10:54 2023-01-24 18:40:42 0:29:48 0:18:06 0:11:42 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} 2
Failure Reason:

"2023-01-24T18:35:49.412274+0000 mgr.x (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135582 2023-01-24 16:06:22 2023-01-24 18:13:05 2023-01-24 18:56:28 0:43:23 0:34:12 0:09:11 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7135584 2023-01-24 16:06:23 2023-01-24 18:13:25 2023-01-24 18:44:22 0:30:57 0:25:36 0:05:21 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{yes}} 3
fail 7135586 2023-01-24 16:06:24 2023-01-24 18:13:26 2023-01-24 18:41:51 0:28:25 0:16:14 0:12:11 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi019 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

pass 7135588 2023-01-24 16:06:25 2023-01-24 18:14:57 2023-01-24 18:55:33 0:40:36 0:29:08 0:11:28 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} 2
fail 7135590 2023-01-24 16:06:26 2023-01-24 18:15:58 2023-01-24 18:54:51 0:38:53 0:27:38 0:11:15 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2023-01-24T18:40:57.493039+0000 mgr.y (mgr.14100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135592 2023-01-24 16:06:27 2023-01-24 18:18:19 2023-01-24 18:39:19 0:21:00 0:15:32 0:05:28 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} 2
Failure Reason:

"2023-01-24T18:33:11.939144+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135593 2023-01-24 16:06:28 2023-01-24 18:18:29 2023-01-24 18:44:24 0:25:55 0:18:21 0:07:34 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} 2
Failure Reason:

"2023-01-24T18:33:54.982521+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135595 2023-01-24 16:06:30 2023-01-24 18:21:20 2023-01-24 18:56:03 0:34:43 0:19:10 0:15:33 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} 2
Failure Reason:

"2023-01-24T18:51:16.520629+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135597 2023-01-24 16:06:31 2023-01-24 18:26:22 2023-01-24 18:56:00 0:29:38 0:17:27 0:12:11 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 5
fail 7135600 2023-01-24 16:06:32 2023-01-24 18:29:44 2023-01-24 19:10:54 0:41:10 0:31:06 0:10:04 smithi main centos 8.stream fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

"2023-01-24T18:54:11.679308+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135602 2023-01-24 16:06:33 2023-01-24 18:30:34 2023-01-24 19:03:16 0:32:42 0:20:22 0:12:20 smithi main centos 8.stream fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi066 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135604 2023-01-24 16:06:34 2023-01-24 18:30:45 2023-01-24 20:32:04 2:01:19 1:51:26 0:09:53 smithi main centos 8.stream fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} 2
Failure Reason:

"2023-01-24T18:55:21.389356+0000 mgr.y (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135606 2023-01-24 16:06:36 2023-01-24 18:31:56 2023-01-24 19:14:23 0:42:27 0:32:00 0:10:27 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135608 2023-01-24 16:06:37 2023-01-24 18:35:27 2023-01-24 19:08:41 0:33:14 0:26:52 0:06:22 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} 2
Failure Reason:

"2023-01-24T18:50:01.215723+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135610 2023-01-24 16:06:38 2023-01-24 18:35:47 2023-01-24 19:25:14 0:49:27 0:38:25 0:11:02 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} 3
fail 7135612 2023-01-24 16:06:39 2023-01-24 18:38:48 2023-01-24 18:58:27 0:19:39 0:12:11 0:07:28 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} 2
Failure Reason:

"2023-01-24T18:53:47.545244+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135613 2023-01-24 16:06:40 2023-01-24 18:38:49 2023-01-24 19:16:35 0:37:46 0:25:11 0:12:35 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi178 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135616 2023-01-24 16:06:41 2023-01-24 19:02:45 845 smithi main rhel 8.4 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2023-01-24T18:56:17.489639+0000 mgr.x (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135618 2023-01-24 16:06:42 2023-01-24 20:03:07 4466 smithi main rhel 8.4 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
Failure Reason:

"2023-01-24T18:56:44.649193+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135620 2023-01-24 16:06:44 2023-01-24 18:43:11 2023-01-24 19:04:02 0:20:51 0:14:57 0:05:54 smithi main rhel 8.4 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2023-01-24T18:57:22.430179+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135622 2023-01-24 16:06:45 2023-01-24 18:43:12 2023-01-24 19:03:25 0:20:13 0:11:58 0:08:15 smithi main rhel 8.4 fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi112 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 7135624 2023-01-24 16:06:46 2023-01-24 18:44:33 2023-01-24 19:16:38 0:32:05 0:20:19 0:11:46 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2023-01-24T19:07:45.874250+0000 mgr.x (mgr.14103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135626 2023-01-24 16:06:47 2023-01-24 18:49:14 2023-01-24 19:07:52 0:18:38 0:12:37 0:06:01 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} 2
Failure Reason:

"2023-01-24T19:03:45.252943+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135628 2023-01-24 16:06:48 2023-01-24 18:49:35 2023-01-24 19:12:33 0:22:58 0:12:38 0:10:20 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} 2
Failure Reason:

"2023-01-24T19:08:14.119186+0000 mgr.y (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135630 2023-01-24 16:06:49 2023-01-24 18:52:46 2023-01-24 19:25:22 0:32:36 0:14:56 0:17:40 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} 2
pass 7135632 2023-01-24 16:06:51 2023-01-24 18:54:27 2023-01-24 19:39:48 0:45:21 0:38:26 0:06:55 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} 3
pass 7135634 2023-01-24 16:06:52 2023-01-24 18:54:58 2023-01-24 19:37:52 0:42:54 0:33:21 0:09:33 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135636 2023-01-24 16:06:53 2023-01-24 18:55:38 2023-01-24 19:56:57 1:01:19 0:52:05 0:09:14 smithi main centos 8.stream fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} 2
Failure Reason:

"2023-01-24T19:23:17.777177+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135638 2023-01-24 16:06:54 2023-01-24 18:55:49 2023-01-24 19:26:19 0:30:30 0:19:33 0:10:57 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} 2
Failure Reason:

"2023-01-24T19:15:14.829265+0000 mgr.y (mgr.4122) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135640 2023-01-24 16:06:55 2023-01-24 18:56:10 2023-01-24 19:25:14 0:29:04 0:19:36 0:09:28 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

"2023-01-24T19:14:49.774599+0000 mgr.y (mgr.4123) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135642 2023-01-24 16:06:56 2023-01-24 18:56:30 2023-01-24 19:36:11 0:39:41 0:27:04 0:12:37 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} 3
Failure Reason:

"2023-01-24T19:23:32.123086+0000 mgr.y (mgr.14099) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135644 2023-01-24 16:06:58 2023-01-24 18:59:41 2023-01-24 19:20:55 0:21:14 0:14:19 0:06:55 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} 2
Failure Reason:

"2023-01-24T19:14:35.221995+0000 mgr.x (mgr.4111) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135646 2023-01-24 16:06:59 2023-01-24 19:02:52 2023-01-24 19:42:36 0:39:44 0:22:41 0:17:03 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} 2
pass 7135648 2023-01-24 16:07:00 2023-01-24 19:03:33 2023-01-24 19:37:04 0:33:31 0:26:21 0:07:10 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}} 3
fail 7135650 2023-01-24 16:07:01 2023-01-24 19:05:54 2023-01-24 19:44:22 0:38:28 0:25:32 0:12:56 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} 2
Failure Reason:

"2023-01-24T19:31:31.636068+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135652 2023-01-24 16:07:02 2023-01-24 19:08:45 2023-01-24 19:50:20 0:41:35 0:32:39 0:08:56 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
fail 7135654 2023-01-24 16:07:04 2023-01-24 19:09:36 2023-01-24 19:32:45 0:23:09 0:16:29 0:06:40 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
Failure Reason:

"2023-01-24T19:24:12.790975+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135656 2023-01-24 16:07:05 2023-01-24 19:10:57 2023-01-24 19:30:07 0:19:10 0:13:22 0:05:48 smithi main rhel 8.4 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
Failure Reason:

"2023-01-24T19:25:30.429034+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135658 2023-01-24 16:07:06 2023-01-24 19:11:47 2023-01-24 19:46:35 0:34:48 0:22:54 0:11:54 smithi main centos 8.stream fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health} 2
Failure Reason:

timeout expired in wait_until_healthy

fail 7135660 2023-01-24 16:07:07 2023-01-24 19:13:19 2023-01-24 19:34:00 0:20:41 0:11:56 0:08:45 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} 2
Failure Reason:

"2023-01-24T19:30:45.024284+0000 mgr.x (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135662 2023-01-24 16:07:08 2023-01-24 19:14:29 2023-01-24 19:38:53 0:24:24 0:14:35 0:09:49 smithi main centos 8.stream fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} 2
Failure Reason:

"2023-01-24T19:31:51.728764+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135664 2023-01-24 16:07:09 2023-01-24 19:15:20 2023-01-24 20:17:42 1:02:22 0:48:30 0:13:52 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 4
pass 7135666 2023-01-24 16:07:11 2023-01-24 19:16:41 2023-01-24 20:20:54 1:04:13 0:48:48 0:15:25 smithi main ubuntu 20.04 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
pass 7135668 2023-01-24 16:07:12 2023-01-24 19:24:53 2023-01-24 19:50:59 0:26:06 0:13:02 0:13:04 smithi main ubuntu 20.04 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7135670 2023-01-24 16:07:13 2023-01-24 19:25:23 2023-01-24 19:50:18 0:24:55 0:15:08 0:09:47 smithi main centos 8.stream fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi174 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135672 2023-01-24 16:07:14 2023-01-24 19:25:24 2023-01-24 20:23:15 0:57:51 0:50:54 0:06:57 smithi main rhel 8.4 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
Failure Reason:

"2023-01-24T19:40:30.335479+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135674 2023-01-24 16:07:15 2023-01-24 19:27:53 2023-01-24 19:55:23 0:27:30 0:21:10 0:06:20 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} 2
Failure Reason:

"2023-01-24T19:42:43.144059+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135676 2023-01-24 16:07:16 2023-01-24 19:28:04 2023-01-24 20:05:32 0:37:28 0:25:55 0:11:33 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

"2023-01-24T19:53:27.035337+0000 mgr.y (mgr.14100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135678 2023-01-24 16:07:18 2023-01-24 19:30:45 2023-01-24 19:56:36 0:25:51 0:13:18 0:12:33 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} 2
pass 7135680 2023-01-24 16:07:19 2023-01-24 19:34:05 2023-01-24 20:13:09 0:39:04 0:29:44 0:09:20 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} 3
fail 7135682 2023-01-24 16:07:20 2023-01-24 19:34:06 2023-01-24 20:25:53 0:51:47 0:38:04 0:13:43 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} 2
Failure Reason:

"2023-01-24T20:01:40.610641+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135684 2023-01-24 16:07:21 2023-01-24 19:36:17 2023-01-24 20:06:00 0:29:43 0:19:01 0:10:42 smithi main rhel 8.4 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} 2
Failure Reason:

"2023-01-24T20:00:30.393897+0000 mgr.x (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135686 2023-01-24 16:07:22 2023-01-24 19:37:38 2023-01-24 19:57:29 0:19:51 0:08:41 0:11:10 smithi main fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=596208b4445cdfc3f826189613d9bc0d095a1bb7

fail 7135688 2023-01-24 16:07:23 2023-01-24 19:39:49 2023-01-24 20:06:06 0:26:17 0:17:46 0:08:31 smithi main centos 8.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} 2
Failure Reason:

"2023-01-24T19:56:25.821516+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135690 2023-01-24 16:07:25 2023-01-24 19:41:40 2023-01-24 20:02:26 0:20:46 0:11:34 0:09:12 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} 1
Failure Reason:

"2023-01-24T19:59:54.387428+0000 mgr.y (mgr.14109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135692 2023-01-24 16:07:26 2023-01-24 19:42:40 2023-01-24 20:11:31 0:28:51 0:16:49 0:12:02 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} 2
fail 7135694 2023-01-24 16:07:27 2023-01-24 19:44:41 2023-01-24 20:14:44 0:30:03 0:20:53 0:09:10 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi027 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

fail 7135696 2023-01-24 16:07:28 2023-01-24 19:46:42 2023-01-24 20:07:09 0:20:27 0:12:25 0:08:02 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} 2
Failure Reason:

"2023-01-24T20:02:42.836390+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135698 2023-01-24 16:07:29 2023-01-24 19:48:43 2023-01-24 20:09:52 0:21:09 0:12:24 0:08:45 smithi main centos 8.stream fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} 2
fail 7135700 2023-01-24 16:07:30 2023-01-24 19:49:24 2023-01-24 20:29:19 0:39:55 0:28:33 0:11:22 smithi main rhel 8.4 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}} 3
Failure Reason:

SELinux denials found on ubuntu@smithi050.front.sepia.ceph.com: ['type=AVC msg=audit(1674590315.335:195): avc: denied { node_bind } for pid=1409 comm="ping" saddr=172.21.15.50 scontext=system_u:system_r:ping_t:s0 tcontext=system_u:object_r:node_t:s0 tclass=icmp_socket permissive=1']

fail 7135702 2023-01-24 16:07:31 2023-01-24 19:50:25 2023-01-24 20:20:01 0:29:36 0:19:10 0:10:26 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} 2
Failure Reason:

"2023-01-24T20:14:25.662871+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135704 2023-01-24 16:07:33 2023-01-24 19:51:05 2023-01-24 20:15:57 0:24:52 0:12:37 0:12:15 smithi main ubuntu 20.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi047 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh'

fail 7135706 2023-01-24 16:07:34 2023-01-24 19:52:46 2023-01-24 21:36:48 1:44:02 1:29:57 0:14:05 smithi main centos 8.stream fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind} 2
Failure Reason:

"2023-01-24T20:29:18.761581+0000 mgr.y (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135708 2023-01-24 16:07:35 2023-01-24 19:56:37 2023-01-24 20:23:55 0:27:18 0:17:08 0:10:10 smithi main centos 8.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

"2023-01-24T20:14:32.056922+0000 mgr.x (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7135710 2023-01-24 16:07:36 2023-01-24 19:56:58 2023-01-24 20:18:29 0:21:31 0:15:18 0:06:13 smithi main rhel 8.4 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
Failure Reason:

"2023-01-24T20:11:56.854194+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7135712 2023-01-24 16:07:37 2023-01-24 19:57:49 2023-01-24 20:40:59 0:43:10 0:32:36 0:10:34 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7135714 2023-01-24 16:07:38 2023-01-24 19:59:49 2023-01-24 20:23:53 0:24:04 0:12:56 0:11:08 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} 2
fail 7135716 2023-01-24 16:07:40 2023-01-24 20:02:31 2023-01-24 20:32:13 0:29:42 0:18:07 0:11:35 smithi main rhel 8.4 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi019 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=596208b4445cdfc3f826189613d9bc0d095a1bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 7135718 2023-01-24 16:07:41 2023-01-24 20:05:42 2023-01-24 20:24:56 0:19:14 0:12:11 0:07:03 smithi main rhel 8.4 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} 2
Failure Reason:

Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)

pass 7135720 2023-01-24 16:07:42 2023-01-24 20:05:52 2023-01-24 20:30:14 0:24:22 0:13:08 0:11:14 smithi main ubuntu 20.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 5