Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 5761555 2021-01-07 02:27:40 2021-01-07 08:26:49 2021-01-07 08:50:49 0:24:00 0:14:10 0:09:50 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/direct_io} 3
fail 5761556 2021-01-07 02:27:41 2021-01-07 08:27:35 2021-01-07 09:15:36 0:48:01 0:36:20 0:11:41 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/fs/misc} 3
Failure Reason:

"2021-01-07T08:54:17.318375+0000 mds.b (mds.0) 69 : cluster [WRN] Scrub error on inode 0x1000000cd18 (/client.0/tmp/payload.1/multiple_rsync_payload.126683/python2.7/dist-packages/matplotlib) see mds.b log and `damage ls` output for details" in cluster log

pass 5761557 2021-01-07 02:27:42 2021-01-07 08:28:20 2021-01-07 10:08:22 1:40:02 1:21:50 0:18:12 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/kernel_untar_build} 3
pass 5761558 2021-01-07 02:27:42 2021-01-07 08:29:40 2021-01-07 09:09:39 0:39:59 0:18:31 0:21:28 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/blogbench} 3
pass 5761559 2021-01-07 02:27:43 2021-01-07 08:29:40 2021-01-07 09:25:40 0:56:00 0:43:05 0:12:55 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/dbench} 3
pass 5761560 2021-01-07 02:27:44 2021-01-07 08:29:40 2021-01-07 09:15:40 0:46:00 0:28:33 0:17:27 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/ffsb} 3
pass 5761561 2021-01-07 02:27:45 2021-01-07 08:29:40 2021-01-07 08:57:39 0:27:59 0:12:41 0:15:18 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/norstats} 3
pass 5761562 2021-01-07 02:27:45 2021-01-07 08:29:57 2021-01-07 08:55:57 0:26:00 0:13:09 0:12:51 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsstress} 3
pass 5761563 2021-01-07 02:27:46 2021-01-07 08:31:35 2021-01-07 10:17:36 1:46:01 0:27:33 1:18:28 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/fsx} 3
pass 5761564 2021-01-07 02:27:47 2021-01-07 08:33:49 2021-01-07 08:59:48 0:25:59 0:11:10 0:14:49 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsync-tester} 3
pass 5761565 2021-01-07 02:27:48 2021-01-07 08:33:49 2021-01-07 09:11:49 0:38:00 0:20:38 0:17:22 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/test_o_trunc} 3
pass 5761566 2021-01-07 02:27:49 2021-01-07 08:34:15 2021-01-07 09:48:16 1:14:01 1:00:01 0:14:00 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/iogen} 3
pass 5761567 2021-01-07 02:27:49 2021-01-07 08:35:32 2021-01-07 09:33:32 0:58:00 0:18:34 0:39:26 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/iozone} 3
pass 5761568 2021-01-07 02:27:50 2021-01-07 08:37:39 2021-01-07 09:13:39 0:36:00 0:13:10 0:22:50 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/pjd} 3
pass 5761569 2021-01-07 02:27:51 2021-01-07 08:37:39 2021-01-07 09:21:39 0:44:00 0:12:15 0:31:45 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/direct_io} 3
pass 5761570 2021-01-07 02:27:52 2021-01-07 08:37:39 2021-01-07 09:21:39 0:44:00 0:32:01 0:11:59 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/fs/misc} 3
pass 5761571 2021-01-07 02:27:53 2021-01-07 08:40:02 2021-01-07 09:44:03 1:04:01 0:38:40 0:25:21 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/kernel_untar_build} 3
pass 5761572 2021-01-07 02:27:53 2021-01-07 08:40:02 2021-01-07 09:42:03 1:02:01 0:22:54 0:39:07 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/blogbench} 3
pass 5761573 2021-01-07 02:27:54 2021-01-07 08:40:02 2021-01-07 10:14:03 1:34:01 0:51:29 0:42:32 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/dbench} 3
pass 5761574 2021-01-07 02:27:55 2021-01-07 08:40:02 2021-01-07 09:24:02 0:44:00 0:21:04 0:22:56 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/ffsb} 3
pass 5761575 2021-01-07 02:27:56 2021-01-07 08:41:33 2021-01-07 09:25:33 0:44:00 0:15:54 0:28:06 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/norstats} 3
pass 5761576 2021-01-07 02:27:56 2021-01-07 08:41:33 2021-01-07 09:27:33 0:46:00 0:16:11 0:29:49 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsstress} 3
pass 5761577 2021-01-07 02:27:57 2021-01-07 08:41:46 2021-01-07 09:27:46 0:46:00 0:25:50 0:20:10 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/fsx} 3
pass 5761578 2021-01-07 02:27:58 2021-01-07 08:41:49 2021-01-07 08:59:48 0:17:59 0:10:49 0:07:10 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsync-tester} 3
pass 5761579 2021-01-07 02:27:59 2021-01-07 08:43:28 2021-01-07 09:31:28 0:48:00 0:22:54 0:25:06 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/test_o_trunc} 3
fail 5761580 2021-01-07 02:28:00 2021-01-07 08:43:28 2021-01-07 09:43:29 1:00:01 0:45:37 0:14:24 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/iogen} 3
Failure Reason:

"2021-01-07T09:18:23.699210+0000 mon.c (mon.0) 343 : cluster [WRN] Replacing daemon mds.b as rank 0 with standby daemon mds.c" in cluster log

pass 5761581 2021-01-07 02:28:00 2021-01-07 08:43:28 2021-01-07 09:09:28 0:26:00 0:14:40 0:11:20 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/iozone} 3
pass 5761582 2021-01-07 02:28:01 2021-01-07 08:43:31 2021-01-07 09:01:30 0:17:59 0:11:20 0:06:39 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/pjd} 3
pass 5761583 2021-01-07 02:28:02 2021-01-07 08:43:32 2021-01-07 09:17:32 0:34:00 0:13:52 0:20:08 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/direct_io} 3
fail 5761584 2021-01-07 02:28:03 2021-01-07 08:43:51 2021-01-07 09:27:52 0:44:01 0:30:50 0:13:11 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/fs/misc} 3
Failure Reason:

"2021-01-07T09:12:24.394130+0000 mds.c (mds.0) 69 : cluster [WRN] Scrub error on inode 0x1000000d84b (/client.0/tmp/payload.1/multiple_rsync_payload.119689/python2.7/dist-packages/samba/tests) see mds.c log and `damage ls` output for details" in cluster log

pass 5761585 2021-01-07 02:28:04 2021-01-07 08:44:16 2021-01-07 10:32:18 1:48:02 1:21:10 0:26:52 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/kernel_untar_build} 3
pass 5761586 2021-01-07 02:28:04 2021-01-07 08:45:46 2021-01-07 09:53:47 1:08:01 0:22:13 0:45:48 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/blogbench} 3
pass 5761587 2021-01-07 02:28:05 2021-01-07 08:45:46 2021-01-07 10:17:47 1:32:01 0:51:21 0:40:40 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/dbench} 3
pass 5761588 2021-01-07 02:28:06 2021-01-07 08:46:13 2021-01-07 09:44:14 0:58:01 0:36:11 0:21:50 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/ffsb} 3
pass 5761589 2021-01-07 02:28:07 2021-01-07 08:48:14 2021-01-07 09:24:14 0:36:00 0:12:21 0:23:39 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/norstats} 3
pass 5761590 2021-01-07 02:28:08 2021-01-07 08:48:14 2021-01-07 09:12:13 0:23:59 0:12:28 0:11:31 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsstress} 3
pass 5761591 2021-01-07 02:28:08 2021-01-07 08:49:03 2021-01-07 09:29:03 0:40:00 0:25:35 0:14:25 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/fsx} 3
pass 5761592 2021-01-07 02:28:09 2021-01-07 08:51:19 2021-01-07 09:13:18 0:21:59 0:12:04 0:09:55 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsync-tester} 3
pass 5761593 2021-01-07 02:28:10 2021-01-07 08:51:51 2021-01-07 09:23:50 0:31:59 0:20:38 0:11:21 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/test_o_trunc} 3
pass 5761594 2021-01-07 02:28:11 2021-01-07 08:53:56 2021-01-07 09:27:56 0:34:00 0:20:41 0:13:19 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/iogen} 3
pass 5761595 2021-01-07 02:28:12 2021-01-07 08:53:56 2021-01-07 09:39:56 0:46:00 0:15:04 0:30:56 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/iozone} 3
pass 5761596 2021-01-07 02:28:12 2021-01-07 08:53:56 2021-01-07 09:29:56 0:36:00 0:13:46 0:22:14 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/pjd} 3
pass 5761597 2021-01-07 02:28:13 2021-01-07 08:56:14 2021-01-07 09:22:13 0:25:59 0:12:33 0:13:26 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/direct_io} 3
fail 5761598 2021-01-07 02:28:14 2021-01-07 08:56:14 2021-01-07 09:42:14 0:46:00 0:31:48 0:14:12 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/fs/misc} 3
Failure Reason:

"2021-01-07T09:26:22.025893+0000 mds.c (mds.0) 84 : cluster [WRN] Scrub error on inode 0x10000015b48 (/client.0/tmp/payload.2/multiple_rsync_payload.144661/modules/4.18.0-240.el8.x86_64/kernel/drivers/hwmon) see mds.c log and `damage ls` output for details" in cluster log

pass 5761599 2021-01-07 02:28:15 2021-01-07 08:56:14 2021-01-07 09:42:14 0:46:00 0:31:28 0:14:32 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/kernel_untar_build} 3
pass 5761600 2021-01-07 02:28:15 2021-01-07 08:57:58 2021-01-07 10:09:59 1:12:01 0:23:22 0:48:39 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/blogbench} 3
pass 5761601 2021-01-07 02:28:16 2021-01-07 08:57:58 2021-01-07 10:37:59 1:40:01 0:43:59 0:56:02 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/dbench} 3
pass 5761602 2021-01-07 02:28:17 2021-01-07 08:57:58 2021-01-07 10:15:59 1:18:01 0:36:59 0:41:02 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/ffsb} 3
pass 5761603 2021-01-07 02:28:18 2021-01-07 08:59:40 2021-01-07 09:39:40 0:40:00 0:15:54 0:24:06 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/norstats} 3
pass 5761604 2021-01-07 02:28:18 2021-01-07 08:59:49 2021-01-07 09:35:49 0:36:00 0:13:47 0:22:13 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsstress} 3
pass 5761605 2021-01-07 02:28:19 2021-01-07 08:59:49 2021-01-07 09:37:49 0:38:00 0:25:38 0:12:22 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/fsx} 3
pass 5761606 2021-01-07 02:28:20 2021-01-07 09:01:39 2021-01-07 10:11:40 1:10:01 0:10:41 0:59:20 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsync-tester} 3
pass 5761607 2021-01-07 02:28:21 2021-01-07 09:01:39 2021-01-07 10:03:40 1:02:01 0:23:17 0:38:44 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/test_o_trunc} 3
pass 5761608 2021-01-07 02:28:22 2021-01-07 09:01:40 2021-01-07 10:33:41 1:32:01 0:43:21 0:48:40 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/iogen} 3
pass 5761609 2021-01-07 02:28:22 2021-01-07 09:01:40 2021-01-07 09:55:40 0:54:00 0:18:06 0:35:54 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/iozone} 3
pass 5761610 2021-01-07 02:28:23 2021-01-07 09:01:49 2021-01-07 09:55:49 0:54:00 0:11:10 0:42:50 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/pjd} 3
pass 5761611 2021-01-07 02:28:24 2021-01-07 09:01:59 2021-01-07 10:09:59 1:08:00 0:14:05 0:53:55 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/direct_io} 3
fail 5761612 2021-01-07 02:28:25 2021-01-07 09:02:13 2021-01-07 09:54:13 0:52:00 0:35:31 0:16:29 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/fs/misc} 3
Failure Reason:

"2021-01-07T09:36:14.810877+0000 mds.c (mds.0) 89 : cluster [WRN] Scrub error on inode 0x10000011791 (/client.0/tmp/payload.1/multiple_rsync_payload.126527/x86_64-linux-gnu/perl5/5.26/Tk/pTk) see mds.c log and `damage ls` output for details" in cluster log

pass 5761613 2021-01-07 02:28:26 2021-01-07 09:05:39 2021-01-07 11:21:41 2:16:02 1:20:21 0:55:41 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/kernel_untar_build} 3
pass 5761614 2021-01-07 02:28:26 2021-01-07 09:05:39 2021-01-07 09:55:40 0:50:01 0:20:10 0:29:51 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/blogbench} 3
pass 5761615 2021-01-07 02:28:27 2021-01-07 09:09:18 2021-01-07 10:31:19 1:22:01 0:58:03 0:23:58 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/dbench} 3
pass 5761616 2021-01-07 02:28:28 2021-01-07 09:09:29 2021-01-07 10:21:30 1:12:01 0:37:44 0:34:17 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/ffsb} 3
pass 5761617 2021-01-07 02:28:29 2021-01-07 09:09:41 2021-01-07 10:13:41 1:04:00 0:12:48 0:51:12 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/norstats} 3
pass 5761618 2021-01-07 02:28:29 2021-01-07 09:12:17 2021-01-07 09:32:16 0:19:59 0:12:19 0:07:40 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsstress} 3
pass 5761619 2021-01-07 02:28:30 2021-01-07 09:12:17 2021-01-07 09:52:17 0:40:00 0:21:57 0:18:03 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/fsx} 3
pass 5761620 2021-01-07 02:28:31 2021-01-07 09:12:17 2021-01-07 09:36:16 0:23:59 0:12:29 0:11:30 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/fsync-tester} 3
pass 5761621 2021-01-07 02:28:32 2021-01-07 09:13:46 2021-01-07 09:41:45 0:27:59 0:20:14 0:07:45 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/fs/test_o_trunc} 3
pass 5761622 2021-01-07 02:28:33 2021-01-07 09:13:46 2021-01-07 10:15:46 1:02:00 0:53:16 0:08:44 smithi master rhel 8.3 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/iogen} 3
pass 5761623 2021-01-07 02:28:33 2021-01-07 09:15:59 2021-01-07 09:43:59 0:28:00 0:15:11 0:12:49 smithi master centos 8.2 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/no tasks/workunit/suites/iozone} 3
pass 5761624 2021-01-07 02:28:34 2021-01-07 09:15:59 2021-01-07 09:43:59 0:28:00 0:13:10 0:14:50 smithi master ubuntu 18.04 fs:workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} scrub/yes tasks/workunit/suites/pjd} 3