Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 5815391 2021-01-22 12:30:55 2021-01-22 12:31:19 2021-01-22 12:51:18 0:19:59 0:13:07 0:06:52 smithi master rhel 8.3 fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
pass 5815392 2021-01-22 12:30:55 2021-01-22 12:31:47 2021-01-22 12:47:47 0:16:00 0:07:21 0:08:39 smithi master fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073} 1
pass 5815393 2021-01-22 12:30:56 2021-01-22 12:32:20 2021-01-22 12:48:19 0:15:59 0:09:54 0:06:05 smithi master rhel 8.3 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} 2
pass 5815394 2021-01-22 12:30:57 2021-01-22 12:32:42 2021-01-22 12:48:41 0:15:59 0:09:02 0:06:57 smithi master rhel 8.3 fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} 2
pass 5815395 2021-01-22 12:30:58 2021-01-22 12:33:38 2021-01-22 13:31:39 0:58:01 0:48:15 0:09:46 smithi master centos 8.2 fs/mirror/{begin cephfs-mirror/{one-per-cluster} clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{centos_8} tasks/mirror} 1
pass 5815396 2021-01-22 12:30:58 2021-01-22 12:33:39 2021-01-22 14:03:40 1:30:01 1:23:05 0:06:56 smithi master rhel 8.3 fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} 2
fail 5815397 2021-01-22 12:30:59 2021-01-22 12:33:39 2021-01-22 13:01:38 0:27:59 0:15:53 0:12:06 smithi master ubuntu 18.04 fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} 4
Failure Reason:

Test failure: test_dispatch_queue_throttle_message (tasks.cephfs.test_misc.TestMisc)

pass 5815398 2021-01-22 12:31:00 2021-01-22 12:33:54 2021-01-22 13:23:54 0:50:00 0:39:39 0:10:21 smithi master centos 8.2 fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} 2
pass 5815399 2021-01-22 12:31:01 2021-01-22 12:34:19 2021-01-22 12:50:19 0:16:00 0:10:13 0:05:47 smithi master rhel 8.3 fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
pass 5815400 2021-01-22 12:31:02 2021-01-22 12:35:09 2021-01-22 13:17:09 0:42:00 0:31:46 0:10:14 smithi master ubuntu 18.04 fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} 2
pass 5815401 2021-01-22 12:31:02 2021-01-22 12:35:09 2021-01-22 13:45:10 1:10:01 1:00:27 0:09:34 smithi master centos 8.2 fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} 2
pass 5815402 2021-01-22 12:31:03 2021-01-22 12:35:10 2021-01-22 16:21:14 3:46:04 3:37:31 0:08:33 smithi master ubuntu 18.04 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} 2
pass 5815403 2021-01-22 12:31:04 2021-01-22 12:35:15 2021-01-22 12:53:14 0:17:59 0:09:52 0:08:07 smithi master centos 8.2 fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{centos_8} tasks/fstop} 1
pass 5815404 2021-01-22 12:31:05 2021-01-22 12:37:15 2021-01-22 13:03:15 0:26:00 0:15:55 0:10:05 smithi master ubuntu 18.04 fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} 2
pass 5815405 2021-01-22 12:31:06 2021-01-22 12:37:15 2021-01-22 13:19:15 0:42:00 0:30:07 0:11:53 smithi master fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} 3
pass 5815406 2021-01-22 12:31:06 2021-01-22 12:37:21 2021-01-22 13:41:22 1:04:01 0:54:12 0:09:49 smithi master centos 8.2 fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/lockdep} 2
pass 5815407 2021-01-22 12:31:07 2021-01-22 12:39:07 2021-01-22 13:33:07 0:54:00 0:45:06 0:08:54 smithi master centos 8.2 fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} 2
pass 5815408 2021-01-22 12:31:08 2021-01-22 12:39:07 2021-01-22 13:03:07 0:24:00 0:12:42 0:11:18 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} 3
fail 5815409 2021-01-22 12:31:09 2021-01-22 12:39:07 2021-01-22 13:23:07 0:44:00 0:37:18 0:06:42 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} 3
Failure Reason:

"2021-01-22T12:55:18.884356+0000 mds.f (mds.0) 39 : cluster [WRN] Scrub error on inode 0x10000000263 (/client.0/tmp/testdir/dir1) see mds.f log and `damage ls` output for details" in cluster log

pass 5815410 2021-01-22 12:31:10 2021-01-22 12:39:10 2021-01-22 14:13:11 1:34:01 1:23:59 0:10:02 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} 2
pass 5815411 2021-01-22 12:31:10 2021-01-22 12:39:21 2021-01-22 13:27:21 0:48:00 0:38:07 0:09:53 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} 3
pass 5815412 2021-01-22 12:31:11 2021-01-22 12:41:13 2021-01-22 13:25:13 0:44:00 0:38:32 0:05:28 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} 2
pass 5815413 2021-01-22 12:31:12 2021-01-22 12:41:13 2021-01-22 13:19:13 0:38:00 0:28:24 0:09:36 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} 3
pass 5815414 2021-01-22 12:31:13 2021-01-22 12:42:05 2021-01-22 13:52:05 1:10:00 0:59:16 0:10:44 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} 3
pass 5815415 2021-01-22 12:31:13 2021-01-22 12:43:15 2021-01-22 13:15:15 0:32:00 0:22:09 0:09:51 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} 2
fail 5815416 2021-01-22 12:31:14 2021-01-22 12:43:26 2021-01-22 13:39:27 0:56:01 0:49:28 0:06:33 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} 3
Failure Reason:

"2021-01-22T13:02:24.788263+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log

pass 5815417 2021-01-22 12:31:15 2021-01-22 12:44:46 2021-01-22 13:02:46 0:18:00 0:09:47 0:08:13 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} 2
pass 5815418 2021-01-22 12:31:16 2021-01-22 12:44:46 2021-01-22 13:10:46 0:26:00 0:16:22 0:09:38 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/norstats}} 3
fail 5815419 2021-01-22 12:31:17 2021-01-22 12:44:46 2021-01-22 13:08:46 0:24:00 0:15:18 0:08:42 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} 3
Failure Reason:

"2021-01-22T13:02:20.404339+0000 mds.a (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000c24 (/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_hpi_testsuite) see mds.a log and `damage ls` output for details" in cluster log

pass 5815420 2021-01-22 12:31:18 2021-01-22 12:44:52 2021-01-22 13:06:51 0:21:59 0:13:13 0:08:46 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} 2
pass 5815421 2021-01-22 12:31:18 2021-01-22 12:46:31 2021-01-22 13:32:31 0:46:00 0:39:50 0:06:10 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/fsx}} 3
pass 5815422 2021-01-22 12:31:19 2021-01-22 12:46:31 2021-01-22 13:06:31 0:20:00 0:12:35 0:07:25 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} 2
pass 5815423 2021-01-22 12:31:20 2021-01-22 12:47:08 2021-01-22 13:05:07 0:17:59 0:11:33 0:06:26 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} 3
pass 5815424 2021-01-22 12:31:21 2021-01-22 12:47:11 2021-01-22 13:21:11 0:34:00 0:23:34 0:10:26 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} 3
pass 5815425 2021-01-22 12:31:22 2021-01-22 12:47:59 2021-01-22 14:22:00 1:34:01 1:24:14 0:09:47 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} 2
pass 5815426 2021-01-22 12:31:22 2021-01-22 12:47:59 2021-01-22 13:17:58 0:29:59 0:24:01 0:05:58 smithi master rhel 8.3 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} 2
pass 5815427 2021-01-22 12:31:23 2021-01-22 12:48:20 2021-01-22 13:56:20 1:08:00 0:57:45 0:10:15 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} 3
pass 5815428 2021-01-22 12:31:24 2021-01-22 12:48:20 2021-01-22 13:38:20 0:50:00 0:40:15 0:09:45 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
pass 5815429 2021-01-22 12:31:25 2021-01-22 12:48:22 2021-01-22 13:10:22 0:22:00 0:10:12 0:11:48 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/iozone}} 3
fail 5815430 2021-01-22 12:31:26 2021-01-22 12:48:42 2021-01-22 13:38:43 0:50:01 0:11:31 0:38:30 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} 3
Failure Reason:

"2021-01-22T13:34:27.607739+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000002cd (/client.0/tmp/tmp) see mds.f log and `damage ls` output for details" in cluster log

pass 5815431 2021-01-22 12:31:26 2021-01-22 12:48:50 2021-01-22 13:08:49 0:19:59 0:13:34 0:06:25 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} 2
pass 5815432 2021-01-22 12:31:27 2021-01-22 12:49:38 2021-01-22 13:15:37 0:25:59 0:13:48 0:12:11 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/direct_io}} 3
pass 5815433 2021-01-22 12:31:28 2021-01-22 12:49:38 2021-01-22 13:05:37 0:15:59 0:10:31 0:05:28 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} 2
fail 5815434 2021-01-22 12:31:29 2021-01-22 12:50:20 2021-01-22 13:48:20 0:58:00 0:47:35 0:10:25 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/fs/misc}} 3
Failure Reason:

"2021-01-22T13:19:15.597764+0000 mds.d (mds.0) 94 : cluster [WRN] Scrub error on inode 0x100000124d9 (/client.0/tmp/payload.2/multiple_rsync_payload.125785/python2.7/dist-packages) see mds.d log and `damage ls` output for details" in cluster log

pass 5815435 2021-01-22 12:31:30 2021-01-22 12:51:46 2021-01-22 14:33:47 1:42:01 1:34:15 0:07:46 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} 3
pass 5815436 2021-01-22 12:31:30 2021-01-22 12:51:46 2021-01-22 13:19:45 0:27:59 0:18:44 0:09:15 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} 2
fail 5815437 2021-01-22 12:31:31 2021-01-22 12:51:46 2021-01-22 13:43:46 0:52:00 0:36:17 0:15:43 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} 3
Failure Reason:

"2021-01-22T13:14:51.100617+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fe (/client.0/tmp/blogbench-1.0/src) see mds.f log and `damage ls` output for details" in cluster log

pass 5815438 2021-01-22 12:31:32 2021-01-22 12:51:46 2021-01-22 13:15:45 0:23:59 0:13:17 0:10:42 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
pass 5815439 2021-01-22 12:31:33 2021-01-22 12:53:12 2021-01-22 13:59:13 1:06:01 0:54:19 0:11:42 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/dbench}} 3
fail 5815440 2021-01-22 12:31:34 2021-01-22 12:53:16 2021-01-22 13:55:16 1:02:00 0:48:31 0:13:29 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} 3
Failure Reason:

"2021-01-22T13:14:57.795421+0000 mds.d (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000348 (/client.0/tmp/tmp) see mds.d log and `damage ls` output for details" in cluster log

pass 5815441 2021-01-22 12:31:34 2021-01-22 12:53:46 2021-01-22 14:43:48 1:50:02 1:36:53 0:13:09 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} 2
pass 5815442 2021-01-22 12:31:35 2021-01-22 12:53:55 2021-01-22 13:33:55 0:40:00 0:14:13 0:25:47 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/norstats}} 3
pass 5815443 2021-01-22 12:31:36 2021-01-22 12:55:12 2021-01-22 13:45:12 0:50:00 0:37:57 0:12:03 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} 2
pass 5815444 2021-01-22 12:31:37 2021-01-22 12:57:07 2021-01-22 13:29:07 0:32:00 0:15:44 0:16:16 smithi master centos 8.2 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} 2
pass 5815445 2021-01-22 12:31:38 2021-01-22 12:57:07 2021-01-22 13:23:07 0:26:00 0:14:59 0:11:01 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} 3
pass 5815446 2021-01-22 12:31:38 2021-01-22 12:58:42 2021-01-22 13:40:42 0:42:00 0:27:10 0:14:50 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/fsx}} 3
pass 5815447 2021-01-22 12:31:39 2021-01-22 12:58:42 2021-01-22 13:30:42 0:32:00 0:18:37 0:13:23 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} 2
pass 5815448 2021-01-22 12:31:40 2021-01-22 13:01:13 2021-01-22 13:39:13 0:38:00 0:11:31 0:26:29 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} 3
pass 5815449 2021-01-22 12:31:41 2021-01-22 13:01:39 2021-01-22 13:21:39 0:20:00 0:09:35 0:10:25 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} 2
pass 5815450 2021-01-22 12:31:42 2021-01-22 13:03:13 2021-01-22 13:37:13 0:34:00 0:20:27 0:13:33 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} 3
pass 5815451 2021-01-22 12:31:42 2021-01-22 13:03:13 2021-01-22 14:15:14 1:12:01 1:01:22 0:10:39 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} 3
pass 5815452 2021-01-22 12:31:43 2021-01-22 13:03:13 2021-01-22 13:29:13 0:26:00 0:15:19 0:10:41 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} 2
pass 5815453 2021-01-22 12:31:44 2021-01-22 13:03:17 2021-01-22 13:33:17 0:30:00 0:18:29 0:11:31 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} 3
pass 5815454 2021-01-22 12:31:45 2021-01-22 13:04:52 2021-01-22 13:22:52 0:18:00 0:12:53 0:05:07 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} 2
pass 5815455 2021-01-22 12:31:46 2021-01-22 13:05:06 2021-01-22 13:35:06 0:30:00 0:13:11 0:16:49 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} 3
fail 5815456 2021-01-22 12:31:46 2021-01-22 13:05:08 2021-01-22 15:45:11 2:40:03 2:33:34 0:06:29 smithi master rhel 8.3 fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi001 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e6265640e37f1af7c1911eb062f1d62b6f0d4986 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

fail 5815457 2021-01-22 12:31:47 2021-01-22 13:05:38 2021-01-22 14:19:39 1:14:01 1:04:30 0:09:31 smithi master centos 8.2 fs/verify/{begin centos_latest clusters/1a5s-mds-1c-client conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/valgrind} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi195 with status 135: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e6265640e37f1af7c1911eb062f1d62b6f0d4986 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

pass 5815458 2021-01-22 12:31:48 2021-01-22 13:06:58 2021-01-22 13:32:57 0:25:59 0:13:12 0:12:47 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} 3
fail 5815459 2021-01-22 12:31:49 2021-01-22 13:06:58 2021-01-22 14:24:59 1:18:01 0:41:02 0:36:59 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} 3
Failure Reason:

Command failed on smithi005 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp'

fail 5815460 2021-01-22 12:31:50 2021-01-22 13:06:58 2021-01-22 13:46:58 0:40:00 0:27:50 0:12:10 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} 2
Failure Reason:

Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi178 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e6265640e37f1af7c1911eb062f1d62b6f0d4986 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh'

pass 5815461 2021-01-22 12:31:50 2021-01-22 13:06:58 2021-01-22 14:26:59 1:20:01 0:56:36 0:23:25 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} 3
pass 5815462 2021-01-22 12:31:51 2021-01-22 13:09:12 2021-01-22 13:57:13 0:48:01 0:35:38 0:12:23 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} 2
fail 5815463 2021-01-22 12:31:52 2021-01-22 13:09:13 2021-01-22 13:57:13 0:48:00 0:28:41 0:19:19 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} 3
Failure Reason:

"2021-01-22T13:41:02.654985+0000 mds.c (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000002171 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-47) see mds.c log and `damage ls` output for details" in cluster log

pass 5815464 2021-01-22 12:31:53 2021-01-22 13:10:49 2021-01-22 13:44:48 0:33:59 0:21:18 0:12:41 smithi master ubuntu 18.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} 2
pass 5815465 2021-01-22 12:31:54 2021-01-22 13:10:49 2021-01-22 14:38:50 1:28:01 1:10:24 0:17:37 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} 3
pass 5815466 2021-01-22 12:31:55 2021-01-22 13:10:49 2021-01-22 13:38:48 0:27:59 0:18:19 0:09:40 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} 2
pass 5815467 2021-01-22 12:31:56 2021-01-22 13:15:12 2021-01-22 13:53:12 0:38:00 0:26:22 0:11:38 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} 3
pass 5815468 2021-01-22 12:31:56 2021-01-22 13:15:16 2021-01-22 13:31:15 0:15:59 0:09:43 0:06:16 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} 2
pass 5815469 2021-01-22 12:31:57 2021-01-22 13:15:38 2021-01-22 13:45:38 0:30:00 0:16:52 0:13:08 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} 3
fail 5815470 2021-01-22 12:31:58 2021-01-22 13:15:47 2021-01-22 13:41:46 0:25:59 0:13:58 0:12:01 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} 3
Failure Reason:

"2021-01-22T13:35:18.396090+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x1000000301b (/client.0/tmp/tmp.I5LlEfrRET) see mds.f log and `damage ls` output for details" in cluster log

pass 5815471 2021-01-22 12:31:59 2021-01-22 13:17:37 2021-01-22 13:51:37 0:34:00 0:19:52 0:14:08 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} 2
pass 5815472 2021-01-22 12:32:00 2021-01-22 13:18:00 2021-01-22 14:06:00 0:48:00 0:25:47 0:22:13 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} 3
pass 5815473 2021-01-22 12:32:00 2021-01-22 13:18:21 2021-01-22 13:58:21 0:40:00 0:13:00 0:27:00 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} 2
pass 5815474 2021-01-22 12:32:01 2021-01-22 13:19:40 2021-01-22 13:57:40 0:38:00 0:11:02 0:26:58 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} 3
pass 5815475 2021-01-22 12:32:02 2021-01-22 13:19:40 2021-01-22 14:07:40 0:48:00 0:24:30 0:23:30 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} 3
pass 5815476 2021-01-22 12:32:03 2021-01-22 13:19:46 2021-01-22 16:05:53 2:46:07 2:34:53 0:11:14 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} 2
pass 5815477 2021-01-22 12:32:04 2021-01-22 13:21:10 2021-01-22 14:23:11 1:02:01 0:32:56 0:29:05 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} 3
pass 5815478 2021-01-22 12:32:04 2021-01-22 13:21:11 2021-01-22 14:13:11 0:52:00 0:34:04 0:17:56 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
pass 5815479 2021-01-22 12:32:05 2021-01-22 13:21:12 2021-01-22 13:43:11 0:21:59 0:11:29 0:10:30 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone}} 3
fail 5815480 2021-01-22 12:32:06 2021-01-22 13:21:41 2021-01-22 14:03:41 0:42:00 0:11:19 0:30:41 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} 3
Failure Reason:

"2021-01-22T13:59:56.088446+0000 mds.e (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000002cd (/client.0/tmp/tmp) see mds.e log and `damage ls` output for details" in cluster log

pass 5815481 2021-01-22 12:32:07 2021-01-22 13:23:18 2021-01-22 13:53:18 0:30:00 0:18:00 0:12:00 smithi master centos 8.2 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} 2
pass 5815482 2021-01-22 12:32:08 2021-01-22 13:23:18 2021-01-22 13:43:18 0:20:00 0:08:39 0:11:21 smithi master ubuntu 18.04 fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} 2
pass 5815483 2021-01-22 12:32:08 2021-01-22 13:23:18 2021-01-22 13:51:18 0:28:00 0:15:12 0:12:48 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} 3
pass 5815484 2021-01-22 12:32:09 2021-01-22 13:23:58 2021-01-22 13:45:58 0:22:00 0:10:47 0:11:13 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} 2
fail 5815485 2021-01-22 12:32:10 2021-01-22 13:25:06 2021-01-22 14:09:06 0:44:00 0:31:32 0:12:28 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} 3
Failure Reason:

"2021-01-22T13:49:15.382029+0000 mds.d (mds.2) 6 : cluster [WRN] Scrub error on inode 0x10000000264 (/client.0/tmp/testdir/dir1/dir2) see mds.d log and `damage ls` output for details" in cluster log

pass 5815486 2021-01-22 12:32:11 2021-01-22 13:25:14 2021-01-22 16:09:17 2:44:03 2:31:29 0:12:34 smithi master ubuntu 18.04 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} 3
pass 5815487 2021-01-22 12:32:12 2021-01-22 13:26:53 2021-01-22 13:48:52 0:21:59 0:10:57 0:11:02 smithi master ubuntu 18.04 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} 2
pass 5815488 2021-01-22 12:32:12 2021-01-22 13:27:22 2021-01-22 14:05:22 0:38:00 0:21:59 0:16:01 smithi master rhel 8.3 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} 3
pass 5815489 2021-01-22 12:32:13 2021-01-22 13:28:21 2021-01-22 13:48:21 0:20:00 0:13:35 0:06:25 smithi master rhel 8.3 fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
pass 5815490 2021-01-22 12:32:14 2021-01-22 13:29:08 2021-01-22 14:55:09 1:26:01 1:12:27 0:13:34 smithi master centos 8.2 fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} 3