Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 4413578 2019-10-14 19:04:36 2019-10-14 19:05:08 2019-10-14 20:41:09 1:36:01 0:19:34 1:16:27 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cephfs_misc_tests.yaml} 4
fail 4413579 2019-10-14 19:04:37 2019-10-14 19:06:23 2019-10-14 20:28:23 1:22:00 0:25:04 0:56:56 smithi master fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}} 3
Failure Reason:

"2019-10-14 20:07:00.997788 mon.a mon.0 172.21.15.158:6789/0 58 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

fail 4413580 2019-10-14 19:04:38 2019-10-14 19:07:42 2019-10-14 20:29:43 1:22:01 1:06:50 0:15:11 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_misc.yaml} 2
Failure Reason:

"2019-10-14T19:44:26.892638+0000 mon.b (mon.0) 171 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571082265 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

dead 4413581 2019-10-14 19:04:39 2019-10-14 19:10:54 2019-10-15 07:13:16 12:02:22 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
fail 4413582 2019-10-14 19:04:39 2019-10-14 19:11:30 2019-10-14 19:47:30 0:36:00 0:26:32 0:09:28 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsx.yaml} 2
Failure Reason:

"2019-10-14T19:33:58.721997+0000 mon.b (mon.0) 142 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571081636 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413583 2019-10-14 19:04:40 2019-10-14 19:12:25 2019-10-14 19:56:24 0:43:59 0:32:13 0:11:46 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T19:41:16.647446+0000 mon.a (mon.0) 147 : cluster [ERR] Health check failed: mon a is very low on available space (MON_DISK_CRIT)" in cluster log

pass 4413584 2019-10-14 19:04:41 2019-10-14 19:12:49 2019-10-14 21:04:50 1:52:01 0:10:52 1:41:09 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-3-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/ior-shared-file.yaml} 5
fail 4413585 2019-10-14 19:04:42 2019-10-14 19:13:09 2019-10-14 20:35:09 1:22:00 0:32:21 0:49:39 smithi master ubuntu 18.04 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cephfs-shell.yaml} 2
Failure Reason:

Test failure: test_exceed_file_limit (tasks.cephfs.test_cephfs_shell.TestQuota)

fail 4413586 2019-10-14 19:04:43 2019-10-14 19:14:54 2019-10-14 20:48:54 1:34:00 1:24:03 0:09:57 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_kernel_untar_build.yaml} 2
Failure Reason:

"2019-10-14T19:39:38.687536+0000 mon.b (mon.0) 144 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571081976 sec, osd.1 has slow ops (SLOW_OPS)" in cluster log

fail 4413587 2019-10-14 19:04:44 2019-10-14 19:16:16 2019-10-14 20:30:16 1:14:00 0:54:09 0:19:51 smithi master centos 7.6 fs/traceless/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_dbench.yaml traceless/50pc.yaml} 2
Failure Reason:

"2019-10-14T19:53:44.001594+0000 mon.a (mon.0) 156 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571082822 sec, osd.5 has slow ops (SLOW_OPS)" in cluster log

fail 4413588 2019-10-14 19:04:44 2019-10-14 19:19:06 2019-10-14 22:47:11 3:28:05 3:11:45 0:16:20 smithi master centos fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_fsstress.yaml validater/valgrind.yaml} 2
Failure Reason:

"2019-10-14T21:11:24.271891+0000 mon.a (mon.0) 2692 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571087481 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413589 2019-10-14 19:04:45 2019-10-14 19:19:06 2019-10-14 20:33:06 1:14:00 0:53:54 0:20:06 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_dbench.yaml} 2
Failure Reason:

"2019-10-14T20:02:44.521813+0000 mon.b (mon.0) 604 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083361 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log

dead 4413590 2019-10-14 19:04:46 2019-10-14 19:20:52 2019-10-15 07:23:18 12:02:26 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
fail 4413591 2019-10-14 19:04:47 2019-10-14 19:22:43 2019-10-14 19:56:43 0:34:00 0:17:46 0:16:14 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsstress.yaml} 2
Failure Reason:

"2019-10-14T19:50:34.994284+0000 mon.a (mon.0) 282 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571082633 sec, osd.5 has slow ops (SLOW_OPS)" in cluster log

fail 4413592 2019-10-14 19:04:48 2019-10-14 19:26:10 2019-10-14 20:06:09 0:39:59 0:26:46 0:13:13 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T19:54:51.618052+0000 mon.a (mon.0) 157 : cluster [WRN] Health check failed: 2 slow ops, oldest one blocked for 1571082889 sec, osd.1 has slow ops (SLOW_OPS)" in cluster log

fail 4413593 2019-10-14 19:04:48 2019-10-14 19:26:10 2019-10-14 20:30:10 1:04:00 0:24:55 0:39:05 smithi master fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}} 3
Failure Reason:

"2019-10-14 20:09:32.775053 mon.a mon.0 172.21.15.192:6789/0 63 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

pass 4413594 2019-10-14 19:04:49 2019-10-14 19:29:16 2019-10-14 21:01:16 1:32:00 0:20:56 1:11:04 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-3-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cephfs_misc_tests.yaml} 5
fail 4413595 2019-10-14 19:04:50 2019-10-14 19:32:50 2019-10-14 21:04:50 1:32:00 1:17:03 0:14:57 smithi master ubuntu 18.04 fs/snaps/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/snaptests.yaml} 2
Failure Reason:

"2019-10-14T20:40:30.584412+0000 mon.a (mon.0) 338 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571085628 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log

fail 4413596 2019-10-14 19:04:51 2019-10-14 19:33:00 2019-10-14 21:17:05 1:44:05 1:33:58 0:10:07 smithi master rhel 7.7 fs/thrash/{begin.yaml ceph-thrash/mds.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/none.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_snaptests.yaml} 2
Failure Reason:

"2019-10-14T20:23:36.550951+0000 mon.a (mon.0) 781 : cluster [WRN] Health check failed: 2 slow ops, oldest one blocked for 1571084614 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413597 2019-10-14 19:04:52 2019-10-14 19:33:03 2019-10-14 20:21:03 0:48:00 0:33:33 0:14:27 smithi master centos 7.6 fs/traceless/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml traceless/50pc.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi199 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413598 2019-10-14 19:04:52 2019-10-14 19:35:58 2019-10-14 20:53:58 1:18:00 1:07:13 0:10:47 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_misc.yaml} 2
Failure Reason:

"2019-10-14T20:08:45.859655+0000 mon.b (mon.0) 170 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083724 sec, osd.5 has slow ops (SLOW_OPS)" in cluster log

pass 4413599 2019-10-14 19:04:53 2019-10-14 19:37:28 2019-10-14 20:25:28 0:48:00 0:30:05 0:17:55 smithi master ubuntu 18.04 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/data-scan.yaml} 2
dead 4413600 2019-10-14 19:04:54 2019-10-14 19:37:28 2019-10-15 07:39:51 12:02:23 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
fail 4413601 2019-10-14 19:04:55 2019-10-14 19:39:21 2019-10-14 20:23:21 0:44:00 0:16:56 0:27:04 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_fsync.yaml} 2
Failure Reason:

"2019-10-14T20:12:53.585554+0000 mon.a (mon.0) 263 : cluster [WRN] Health check failed: 14 slow ops, oldest one blocked for 1571083971 sec, daemons [osd.0,osd.1,osd.3] have slow ops. (SLOW_OPS)" in cluster log

fail 4413602 2019-10-14 19:04:56 2019-10-14 19:41:50 2019-10-14 21:25:51 1:44:01 1:26:37 0:17:24 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_kernel_untar_build.yaml} 2
Failure Reason:

"2019-10-14T20:11:08.207940+0000 mon.b (mon.0) 138 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083865 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413603 2019-10-14 19:04:56 2019-10-14 19:45:25 2019-10-14 20:13:25 0:28:00 0:15:47 0:12:13 smithi master ubuntu 18.04 fs/traceless/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_fsstress.yaml traceless/50pc.yaml} 2
Failure Reason:

"2019-10-14T20:05:15.636908+0000 mon.b (mon.0) 140 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083513 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413604 2019-10-14 19:04:57 2019-10-14 19:47:42 2019-10-14 23:03:45 3:16:03 3:00:14 0:15:49 smithi master centos fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_fsstress.yaml validater/valgrind.yaml} 2
Failure Reason:

"2019-10-14T21:38:36.455119+0000 mon.b (mon.0) 2185 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571089115 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413605 2019-10-14 19:04:58 2019-10-14 19:47:42 2019-10-14 20:21:42 0:34:00 0:14:50 0:19:10 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_norstats.yaml} 2
Failure Reason:

"2019-10-14T20:13:11.690757+0000 mon.a (mon.0) 158 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083990 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413606 2019-10-14 19:04:59 2019-10-14 19:48:17 2019-10-14 20:50:17 1:02:00 0:54:08 0:07:52 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_dbench.yaml} 2
Failure Reason:

"2019-10-14T20:09:43.849436+0000 mon.b (mon.0) 149 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571083782 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413607 2019-10-14 19:05:00 2019-10-14 19:48:33 2019-10-14 20:40:33 0:52:00 0:24:51 0:27:09 smithi master fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}} 3
Failure Reason:

"2019-10-14 20:19:11.316553 mon.a mon.0 172.21.15.26:6789/0 97 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

fail 4413608 2019-10-14 19:05:00 2019-10-14 19:48:34 2019-10-14 20:38:34 0:50:00 0:20:29 0:29:31 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi149 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413609 2019-10-14 19:05:01 2019-10-14 19:49:08 2019-10-14 20:39:08 0:50:00 0:24:54 0:25:06 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_fsx.yaml} 2
Failure Reason:

"2019-10-14T20:22:28.320487+0000 mon.b (mon.0) 302 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571084546 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413610 2019-10-14 19:05:02 2019-10-14 19:53:04 2019-10-14 20:51:04 0:58:00 0:26:55 0:31:05 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T20:37:14.395002+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571085431 sec, osd.1 has slow ops (SLOW_OPS)" in cluster log

fail 4413611 2019-10-14 19:05:03 2019-10-14 19:54:28 2019-10-14 23:26:31 3:32:03 3:15:37 0:16:26 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_iozone.yaml} 2
Failure Reason:

Command failed (workunit test suites/iozone.sh) on smithi019 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh'

pass 4413612 2019-10-14 19:05:04 2019-10-14 19:55:45 2019-10-14 22:49:46 2:54:01 0:20:08 2:33:53 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cephfs_misc_tests.yaml} 4
fail 4413613 2019-10-14 19:05:04 2019-10-14 19:56:26 2019-10-14 22:04:27 2:08:01 1:39:13 0:28:48 smithi master centos 7.6 fs/thrash/{begin.yaml ceph-thrash/mds.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/none.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_snaptests.yaml} 2
Failure Reason:

"2019-10-14T21:09:04.249995+0000 mon.b (mon.0) 777 : cluster [WRN] Health check failed: 4 slow ops, oldest one blocked for 1571087342 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413614 2019-10-14 19:05:05 2019-10-14 19:57:00 2019-10-14 20:33:00 0:36:00 0:17:46 0:18:14 smithi master centos 7.6 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/strays.yaml} 2
Failure Reason:

Test failure: test_files_throttle (tasks.cephfs.test_strays.TestStrays)

fail 4413615 2019-10-14 19:05:06 2019-10-14 20:03:47 2019-10-14 20:49:46 0:45:59 0:29:44 0:16:15 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413616 2019-10-14 19:05:07 2019-10-14 20:03:53 2019-10-14 20:35:53 0:32:00 0:15:57 0:16:03 smithi master centos 7.6 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/volume-client/{py/2.yaml task/test/{test.yaml}}} 2
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

fail 4413617 2019-10-14 19:05:07 2019-10-14 20:04:25 2019-10-14 20:34:24 0:29:59 0:19:07 0:10:52 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_fsync.yaml} 2
Failure Reason:

"2019-10-14T20:21:50.927128+0000 mon.b (mon.0) 146 : cluster [WRN] Health check failed: 12 slow ops, oldest one blocked for 1571084508 sec, daemons [osd.2,osd.4,osd.5,osd.6] have slow ops. (SLOW_OPS)" in cluster log

fail 4413618 2019-10-14 19:05:08 2019-10-14 20:05:11 2019-10-14 20:39:10 0:33:59 0:21:02 0:12:57 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T20:30:37.818971+0000 mon.b (mon.0) 372 : cluster [ERR] Health check failed: mon b is very low on available space (MON_DISK_CRIT)" in cluster log

fail 4413619 2019-10-14 19:05:09 2019-10-14 20:06:27 2019-10-14 20:56:27 0:50:00 0:15:45 0:34:15 smithi master centos 7.6 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/volume-client/{py/3.yaml task/test/{test.yaml}}} 2
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

fail 4413620 2019-10-14 19:05:10 2019-10-14 20:06:27 2019-10-14 20:50:27 0:44:00 0:24:56 0:19:04 smithi master fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}} 3
Failure Reason:

"2019-10-14 20:28:45.183706 mon.a mon.0 172.21.15.156:6789/0 65 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

fail 4413621 2019-10-14 19:05:10 2019-10-14 20:08:53 2019-10-14 21:24:53 1:16:00 0:56:55 0:19:05 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_misc.yaml} 2
Failure Reason:

"2019-10-14T20:50:55.167841+0000 mon.a (mon.0) 181 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571086254 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log

pass 4413622 2019-10-14 19:05:11 2019-10-14 20:09:12 2019-10-14 20:37:12 0:28:00 0:14:34 0:13:26 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_norstats.yaml} 2
pass 4413623 2019-10-14 19:05:12 2019-10-14 20:13:41 2019-10-14 20:55:41 0:42:00 0:18:54 0:23:06 smithi master centos 7.6 fs/thrash/{begin.yaml ceph-thrash/mon.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/none.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsstress.yaml} 2
dead 4413624 2019-10-14 19:05:13 2019-10-14 20:13:45 2019-10-15 08:16:07 12:02:22 11:45:52 0:16:30 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi100 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413625 2019-10-14 19:05:13 2019-10-14 20:13:57 2019-10-14 20:59:57 0:46:00 0:25:33 0:20:27 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsx.yaml} 2
Failure Reason:

"2019-10-14T20:45:49.217325+0000 mon.b (mon.0) 139 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571085947 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413626 2019-10-14 19:05:14 2019-10-14 20:15:01 2019-10-14 21:11:00 0:55:59 0:29:33 0:26:26 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T20:57:13.396522+0000 mon.a (mon.0) 486 : cluster [ERR] Health check failed: mon a is very low on available space (MON_DISK_CRIT)" in cluster log

fail 4413627 2019-10-14 19:05:15 2019-10-14 20:21:08 2019-10-14 21:05:08 0:44:00 0:31:51 0:12:09 smithi master ubuntu 18.04 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cephfs-shell.yaml} 2
Failure Reason:

Test failure: test_exceed_file_limit (tasks.cephfs.test_cephfs_shell.TestQuota)

pass 4413628 2019-10-14 19:05:16 2019-10-14 20:21:08 2019-10-14 21:15:08 0:54:00 0:17:52 0:36:08 smithi master centos 7.6 fs/32bits/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{faked-ino.yaml frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsstress.yaml} 2
fail 4413629 2019-10-14 19:05:17 2019-10-14 20:21:43 2019-10-14 22:05:44 1:44:01 1:23:15 0:20:46 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_kernel_untar_build.yaml} 2
Failure Reason:

"2019-10-14T20:54:19.624910+0000 mon.b (mon.0) 142 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571086458 sec, osd.5 has slow ops (SLOW_OPS)" in cluster log

dead 4413630 2019-10-14 19:05:17 2019-10-14 20:23:38 2019-10-15 08:25:50 12:02:12 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-3-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cephfs_misc_tests.yaml}
fail 4413631 2019-10-14 19:05:18 2019-10-14 20:24:33 2019-10-14 22:00:34 1:36:01 1:21:47 0:14:14 smithi master centos 7.6 fs/snaps/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/snaptests.yaml} 2
Failure Reason:

"2019-10-14T21:29:37.920083+0000 mon.b (mon.0) 1796 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571088577 sec, osd.2 has slow ops (SLOW_OPS)" in cluster log

dead 4413632 2019-10-14 19:05:19 2019-10-14 20:25:48 2019-10-15 08:28:14 12:02:26 smithi master ubuntu 18.04 fs/traceless/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml traceless/50pc.yaml} 2
fail 4413633 2019-10-14 19:05:20 2019-10-14 20:27:55 2019-10-14 21:29:55 1:02:00 0:46:25 0:15:35 smithi master fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_dbench.yaml validater/lockdep.yaml} 2
Failure Reason:

"2019-10-14T20:50:10.775214+0000 mon.b (mon.0) 142 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571086208 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413634 2019-10-14 19:05:21 2019-10-14 20:28:25 2019-10-14 21:32:25 1:04:00 0:53:28 0:10:32 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_dbench.yaml} 2
Failure Reason:

"2019-10-14T21:03:21.180173+0000 mon.a (mon.0) 728 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571086999 sec, osd.6 has slow ops (SLOW_OPS)" in cluster log

fail 4413635 2019-10-14 19:05:21 2019-10-14 20:30:00 2019-10-14 21:22:00 0:52:00 0:32:45 0:19:15 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413636 2019-10-14 19:05:22 2019-10-14 20:30:11 2019-10-14 21:22:11 0:52:00 0:31:14 0:20:46 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T21:06:14.292856+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571087171 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413637 2019-10-14 19:05:23 2019-10-14 20:30:17 2019-10-14 21:08:17 0:38:00 0:24:53 0:13:07 smithi master fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}} 3
Failure Reason:

"2019-10-14 20:48:37.176758 mon.a mon.0 172.21.15.198:6789/0 67 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

fail 4413638 2019-10-14 19:05:24 2019-10-14 20:33:17 2019-10-15 00:29:20 3:56:03 3:44:27 0:11:36 smithi master ubuntu 18.04 fs/thrash/{begin.yaml ceph-thrash/mds.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/osd-mds-delay.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_snaptests.yaml} 2
Failure Reason:

"2019-10-14T22:45:07.493459+0000 mon.b (mon.0) 1873 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571093106 sec, osd.6 has slow ops (SLOW_OPS)" in cluster log

pass 4413639 2019-10-14 19:05:24 2019-10-14 20:33:17 2019-10-15 00:07:19 3:34:02 3:16:18 0:17:44 smithi master centos fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_fsstress.yaml validater/valgrind.yaml} 2
fail 4413640 2019-10-14 19:05:25 2019-10-14 20:34:42 2019-10-14 21:34:42 1:00:00 0:48:33 0:11:27 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_misc.yaml} 2
Failure Reason:

"2019-10-14T21:01:26.476754+0000 mon.b (mon.0) 595 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571086883 sec, osd.5 has slow ops (SLOW_OPS)" in cluster log

pass 4413641 2019-10-14 19:05:26 2019-10-14 20:35:10 2019-10-14 21:33:10 0:58:00 0:36:13 0:21:47 smithi master rhel 7.7 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/data-scan.yaml} 2
dead 4413642 2019-10-14 19:05:27 2019-10-14 20:36:10 2019-10-15 08:38:32 12:02:22 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
fail 4413643 2019-10-14 19:05:28 2019-10-14 20:37:29 2019-10-14 21:03:29 0:26:00 0:18:00 0:08:00 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_fsync.yaml} 2
Failure Reason:

"2019-10-14T20:57:52.084020+0000 mon.a (mon.0) 157 : cluster [WRN] Health check failed: 7 slow ops, oldest one blocked for 1571086670 sec, daemons [osd.1,osd.3,osd.4] have slow ops. (SLOW_OPS)" in cluster log

fail 4413644 2019-10-14 19:05:28 2019-10-14 20:38:51 2019-10-14 22:32:52 1:54:01 1:27:36 0:26:25 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_kernel_untar_build.yaml} 2
Failure Reason:

"2019-10-14T21:22:23.414357+0000 mon.a (mon.0) 162 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571088141 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log

pass 4413645 2019-10-14 19:05:29 2019-10-14 20:39:10 2019-10-14 23:51:13 3:12:03 0:19:39 2:52:24 smithi master ubuntu 18.04 fs/multiclient/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distros/ubuntu_latest.yaml mount/fuse.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cephfs_misc_tests.yaml} 4
fail 4413646 2019-10-14 19:05:30 2019-10-14 20:39:12 2019-10-14 21:45:17 1:06:05 0:47:56 0:18:09 smithi master fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_dbench.yaml validater/lockdep.yaml} 2
Failure Reason:

"2019-10-14T21:10:09.500480+0000 mon.b (mon.0) 146 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571087407 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413647 2019-10-14 19:05:31 2019-10-14 20:40:52 2019-10-14 21:50:52 1:10:00 0:53:28 0:16:32 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_dbench.yaml} 2
Failure Reason:

"2019-10-14T21:10:46.164512+0000 mon.b (mon.0) 147 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571087444 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log

fail 4413648 2019-10-14 19:05:31 2019-10-14 20:41:10 2019-10-14 21:25:10 0:44:00 0:25:04 0:18:56 smithi master fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-mimic.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}} 3
Failure Reason:

"2019-10-14 21:03:29.030731 mon.a mon.0 172.21.15.156:6789/0 99 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

fail 4413649 2019-10-14 19:05:32 2019-10-14 20:49:11 2019-10-14 21:19:10 0:29:59 0:22:13 0:07:46 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi140 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4413650 2019-10-14 19:05:33 2019-10-14 20:49:48 2019-10-14 21:29:47 0:39:59 0:28:52 0:11:07 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_fsx.yaml} 2
Failure Reason:

"2019-10-14T21:15:12.133320+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571087710 sec, osd.4 has slow ops (SLOW_OPS)" in cluster log

fail 4413651 2019-10-14 19:05:34 2019-10-14 20:50:03 2019-10-14 21:40:03 0:50:00 0:26:32 0:23:28 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T21:28:26.135917+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571088503 sec, osd.1 has slow ops (SLOW_OPS)" in cluster log

fail 4413652 2019-10-14 19:05:34 2019-10-14 20:50:34 2019-10-15 00:22:38 3:32:04 3:16:53 0:15:11 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_iozone.yaml} 2
Failure Reason:

Command failed (workunit test suites/iozone.sh) on smithi071 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh'

dead 4413653 2019-10-14 19:05:35 2019-10-14 20:50:35 2019-10-14 21:18:34 0:27:59 smithi master rhel 7.7 fs/thrash/{begin.yaml ceph-thrash/mds.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/osd-mds-delay.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_snaptests.yaml} 2
Failure Reason:

SSH connection to smithi072 was lost: 'rpm -q kernel --last | head -n 1'

pass 4413654 2019-10-14 19:05:36 2019-10-14 20:51:06 2019-10-15 00:29:09 3:38:03 3:04:12 0:33:51 smithi master centos fs/verify/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/cfuse_workunit_suites_fsstress.yaml validater/valgrind.yaml} 2
fail 4413655 2019-10-14 19:05:37 2019-10-14 20:52:04 2019-10-14 21:48:04 0:56:00 0:38:02 0:17:58 smithi master rhel 7.7 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/strays.yaml} 2
Failure Reason:

Test failure: test_ops_throttle (tasks.cephfs.test_strays.TestStrays)

fail 4413656 2019-10-14 19:05:37 2019-10-14 20:54:15 2019-10-14 21:52:15 0:58:00 0:30:26 0:27:34 smithi master ubuntu 18.04 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_suites_ffsb.yaml} 2
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi016 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=96e5565376f419bd097c227b15f8088a545840f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

pass 4413657 2019-10-14 19:05:38 2019-10-14 20:55:58 2019-10-14 21:37:57 0:41:59 0:18:12 0:23:47 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_fsstress.yaml} 2
fail 4413658 2019-10-14 19:05:39 2019-10-14 20:56:28 2019-10-14 21:46:28 0:50:00 0:15:51 0:34:09 smithi master centos 7.6 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/volume-client/{py/2.yaml task/test/{test.yaml}}} 2
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

fail 4413659 2019-10-14 19:05:40 2019-10-14 20:59:05 2019-10-14 21:29:05 0:30:00 0:19:07 0:10:53 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_fsync.yaml} 2
Failure Reason:

"2019-10-14T21:22:52.981504+0000 mon.a (mon.0) 163 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1571088170 sec, osd.0 has slow ops (SLOW_OPS)" in cluster log

fail 4413660 2019-10-14 19:05:40 2019-10-14 20:59:58 2019-10-14 21:49:58 0:50:00 0:25:49 0:24:11 smithi master rhel 7.7 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml omap_limit/10000.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/cfuse_workunit_suites_iogen.yaml} 2
Failure Reason:

"2019-10-14T21:37:52.736698+0000 mon.a (mon.0) 164 : cluster [ERR] Health check failed: mon a is very low on available space (MON_DISK_CRIT)" in cluster log

fail 4413661 2019-10-14 19:05:41 2019-10-14 21:00:37 2019-10-14 21:22:36 0:21:59 smithi master centos 7.6 fs/basic_workload/{begin.yaml clusters/fixed-2-ucephfs.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml omap_limit/10.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_7.yaml} tasks/cfuse_workunit_suites_iozone.yaml}
Failure Reason:

Could not reconnect to ubuntu@smithi072.front.sepia.ceph.com

fail 4413662 2019-10-14 19:05:42 2019-10-14 21:01:18 2019-10-14 21:27:17 0:25:59 0:18:11 0:07:48 smithi master rhel 7.7 fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{rhel_7.yaml} tasks/volume-client/{py/3.yaml task/test/{test.yaml}}} 2
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)