Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 2116081 2018-01-27 03:25:37 2018-01-29 04:47:30 2018-01-29 05:07:32 0:20:02 0:09:29 0:10:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116084 2018-01-27 03:25:38 2018-01-29 04:49:30 2018-01-29 05:53:30 1:04:00 0:50:52 0:13:08 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
pass 2116087 2018-01-27 03:25:39 2018-01-29 04:51:35 2018-01-29 05:13:35 0:22:00 0:10:30 0:11:30 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
pass 2116090 2018-01-27 03:25:39 2018-01-29 04:53:31 2018-01-29 06:39:34 1:46:03 1:04:13 0:41:50 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 2116093 2018-01-27 03:25:40 2018-01-29 04:53:31 2018-01-29 06:45:34 1:52:03 0:22:46 1:29:17 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116096 2018-01-27 03:25:41 2018-01-29 04:53:45 2018-01-29 06:29:46 1:36:01 0:09:19 1:26:42 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 2116097 2018-01-27 03:25:41 2018-01-29 04:55:34 2018-01-29 05:43:34 0:48:00 0:30:21 0:17:39 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116098 2018-01-27 03:25:42 2018-01-29 04:55:34 2018-01-29 06:21:35 1:26:01 0:18:45 1:07:16 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116099 2018-01-27 03:25:42 2018-01-29 04:55:34 2018-01-29 06:55:36 2:00:02 0:16:09 1:43:53 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
pass 2116100 2018-01-27 03:25:43 2018-01-29 04:55:34 2018-01-29 06:11:35 1:16:01 0:37:28 0:38:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116101 2018-01-27 03:25:44 2018-01-29 04:55:34 2018-01-29 05:19:34 0:24:00 0:11:37 0:12:23 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116102 2018-01-27 03:25:44 2018-01-29 04:57:24 2018-01-29 07:59:27 3:02:03 0:33:05 2:28:58 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 2116103 2018-01-27 03:25:45 2018-01-29 04:59:29 2018-01-29 06:31:30 1:32:01 0:43:06 0:48:55 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116104 2018-01-27 03:25:46 2018-01-29 04:59:29 2018-01-29 08:07:32 3:08:03 0:19:39 2:48:24 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 07:54:34.822225 mon.b mon.0 172.21.15.89:6789/0 98 : cluster [WRN] Health check failed: 4610 slow ops, oldest one blocked for 33 sec (SLOW_OPS)" in cluster log

pass 2116105 2018-01-27 03:25:46 2018-01-29 05:21:20 2018-01-29 06:01:20 0:40:00 0:09:52 0:30:08 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 2116106 2018-01-27 03:25:47 2018-01-29 05:21:26 2018-01-29 06:23:27 1:02:01 0:10:09 0:51:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116107 2018-01-27 03:25:47 2018-01-29 05:25:47 2018-01-29 06:15:47 0:50:00 0:18:13 0:31:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116108 2018-01-27 03:25:48 2018-01-29 05:27:30 2018-01-29 06:11:30 0:44:00 0:18:58 0:25:02 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/damage.yaml whitelist_health.yaml} 4
fail 2116109 2018-01-27 03:25:49 2018-01-29 05:27:34 2018-01-29 07:07:36 1:40:02 1:10:59 0:29:03 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 06:02:21.439189 mon.b mon.0 172.21.15.93:6789/0 127 : cluster [WRN] Health check update: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log

pass 2116110 2018-01-27 03:25:49 2018-01-29 05:29:47 2018-01-29 06:21:48 0:52:01 0:09:18 0:42:43 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116111 2018-01-27 03:25:50 2018-01-29 05:31:34 2018-01-29 07:31:36 2:00:02 0:22:05 1:37:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
pass 2116112 2018-01-27 03:25:51 2018-01-29 05:31:34 2018-01-29 06:15:34 0:44:00 0:12:06 0:31:54 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116113 2018-01-27 03:25:51 2018-01-29 05:33:36 2018-01-29 06:41:36 1:08:00 0:10:29 0:57:31 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 2116114 2018-01-27 03:25:52 2018-01-29 05:33:36 2018-01-29 11:37:46 6:04:10 0:18:57 5:45:13 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/failover.yaml whitelist_health.yaml} 4
pass 2116115 2018-01-27 03:25:53 2018-01-29 05:35:34 2018-01-29 06:31:34 0:56:00 0:08:13 0:47:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116116 2018-01-27 03:25:53 2018-01-29 05:37:31 2018-01-29 06:33:31 0:56:00 0:08:57 0:47:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116117 2018-01-27 03:25:54 2018-01-29 05:37:37 2018-01-29 06:59:38 1:22:01 0:30:37 0:51:24 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-comp.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 2116118 2018-01-27 03:25:55 2018-01-29 05:39:47 2018-01-29 07:27:49 1:48:02 0:12:29 1:35:33 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 2116119 2018-01-27 03:25:55 2018-01-29 05:40:41 2018-01-29 06:12:41 0:32:00 0:10:52 0:21:08 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116120 2018-01-27 03:25:56 2018-01-29 05:41:39 2018-01-29 06:53:40 1:12:01 0:21:07 0:50:54 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116121 2018-01-27 03:25:57 2018-01-29 05:43:35 2018-01-29 07:29:37 1:46:02 0:15:57 1:30:05 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 2116122 2018-01-27 03:25:57 2018-01-29 05:43:35 2018-01-29 07:43:37 2:00:02 0:29:31 1:30:31 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116123 2018-01-27 03:25:58 2018-01-29 05:47:25 2018-01-29 06:17:25 0:30:00 0:18:50 0:11:10 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116124 2018-01-27 03:25:58 2018-01-29 05:47:25 2018-01-29 07:15:26 1:28:01 0:10:12 1:17:49 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 2116125 2018-01-27 03:25:59 2018-01-29 05:47:26 2018-01-29 07:11:27 1:24:01 0:46:38 0:37:23 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116126 2018-01-27 03:26:00 2018-01-29 05:47:38 2018-01-29 06:29:38 0:42:00 0:24:00 0:18:00 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
dead 2116127 2018-01-27 03:26:01 2018-01-29 05:49:36 2018-01-29 17:53:17 12:03:41 7:50:22 4:13:19 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_full_different_file (tasks.cephfs.test_full.TestClusterFull), test_full_different_file (tasks.cephfs.test_full.TestClusterFull)

pass 2116128 2018-01-27 03:26:01 2018-01-29 05:49:36 2018-01-29 08:23:39 2:34:03 0:38:29 1:55:34 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
pass 2116129 2018-01-27 03:26:02 2018-01-29 05:49:48 2018-01-29 07:37:49 1:48:01 0:24:17 1:23:44 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
pass 2116130 2018-01-27 03:26:03 2018-01-29 05:52:00 2018-01-29 08:24:02 2:32:02 0:10:16 2:21:46 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 2116131 2018-01-27 03:26:03 2018-01-29 05:52:00 2018-01-29 06:46:00 0:54:00 0:10:14 0:43:46 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116132 2018-01-27 03:26:04 2018-01-29 05:53:26 2018-01-29 07:23:27 1:30:01 0:18:02 1:11:59 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116133 2018-01-27 03:26:05 2018-01-29 05:53:26 2018-01-29 09:47:31 3:54:05 0:12:23 3:41:42 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
fail 2116134 2018-01-27 03:26:05 2018-01-29 05:53:31 2018-01-29 06:29:31 0:36:00 0:12:06 0:23:54 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 06:22:34.691408 mon.a mon.1 172.21.15.97:6789/0 42 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 2116135 2018-01-27 03:26:06 2018-01-29 05:55:16 2018-01-29 06:29:16 0:34:00 0:09:30 0:24:30 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
fail 2116136 2018-01-27 03:26:07 2018-01-29 05:57:44 2018-01-29 10:15:49 4:18:05 0:28:43 3:49:22 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/strays.yaml whitelist_health.yaml} 4
Failure Reason:

"2018-01-29 10:03:21.475611 mon.a mon.0 172.21.15.175:6789/0 1808 : cluster [WRN] Health check failed: 53 slow ops, oldest one blocked for 33 sec (SLOW_OPS)" in cluster log

pass 2116137 2018-01-27 03:26:07 2018-01-29 05:59:51 2018-01-29 06:21:50 0:21:59 0:10:28 0:11:31 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116138 2018-01-27 03:26:08 2018-01-29 06:01:34 2018-01-29 06:23:34 0:22:00 0:10:22 0:11:38 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 2116139 2018-01-27 03:26:09 2018-01-29 06:55:51 2018-01-29 11:47:56 4:52:05 0:13:33 4:38:32 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

pass 2116140 2018-01-27 03:26:09 2018-01-29 06:55:51 2018-01-29 07:17:50 0:21:59 0:08:56 0:13:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116141 2018-01-27 03:26:10 2018-01-29 06:55:51 2018-01-29 07:51:51 0:56:00 0:09:22 0:46:38 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116142 2018-01-27 03:26:10 2018-01-29 06:55:51 2018-01-29 09:43:54 2:48:03 0:52:26 1:55:37 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
pass 2116143 2018-01-27 03:26:11 2018-01-29 06:55:53 2018-01-29 16:16:05 9:20:12 0:11:13 9:08:59 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
pass 2116144 2018-01-27 03:26:12 2018-01-29 06:57:50 2018-01-29 07:35:50 0:38:00 0:21:00 0:17:00 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116145 2018-01-27 03:26:12 2018-01-29 06:57:50 2018-01-29 07:43:51 0:46:01 0:27:13 0:18:48 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
Failure Reason:

"2018-01-29 07:23:03.544052 mon.b mon.0 172.21.15.93:6789/0 117 : cluster [WRN] Health check failed: 5914 slow ops, oldest one blocked for 32 sec (SLOW_OPS)" in cluster log

pass 2116146 2018-01-27 03:26:13 2018-01-29 06:57:50 2018-01-29 10:19:54 3:22:04 0:09:23 3:12:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 2116147 2018-01-27 03:26:14 2018-01-29 06:59:43 2018-01-29 08:01:44 1:02:01 0:26:12 0:35:49 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116148 2018-01-27 03:26:14 2018-01-29 06:59:44 2018-01-29 07:51:44 0:52:00 0:18:41 0:33:19 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116149 2018-01-27 03:26:15 2018-01-29 06:59:44 2018-01-29 08:23:44 1:24:00 0:15:38 1:08:22 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
fail 2116150 2018-01-27 03:26:16 2018-01-29 06:59:51 2018-01-29 08:21:52 1:22:01 0:46:04 0:35:57 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_snaps.yaml} 3
Failure Reason:

"2018-01-29 07:59:33.420175 mon.b mon.0 172.21.15.107:6789/0 303 : cluster [WRN] Health check failed: 8862 slow ops, oldest one blocked for 33 sec (SLOW_OPS)" in cluster log

pass 2116151 2018-01-27 03:26:16 2018-01-29 07:03:45 2018-01-29 07:45:45 0:42:00 0:12:45 0:29:15 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116152 2018-01-27 03:26:17 2018-01-29 07:03:45 2018-01-29 09:47:48 2:44:03 0:33:13 2:10:50 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 2116153 2018-01-27 03:26:17 2018-01-29 07:03:45 2018-01-29 08:57:47 1:54:02 0:38:44 1:15:18 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116154 2018-01-27 03:26:18 2018-01-29 07:05:26 2018-01-29 08:09:27 1:04:01 0:49:28 0:14:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 07:27:40.634497 mon.b mon.0 172.21.15.76:6789/0 113 : cluster [WRN] Health check failed: 2052 slow ops, oldest one blocked for 31 sec (SLOW_OPS)" in cluster log

pass 2116155 2018-01-27 03:26:19 2018-01-29 07:05:28 2018-01-29 12:21:34 5:16:06 0:10:17 5:05:49 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 2116156 2018-01-27 03:26:19 2018-01-29 07:05:36 2018-01-29 07:25:35 0:19:59 0:09:06 0:10:53 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116157 2018-01-27 03:26:20 2018-01-29 07:05:38 2018-01-29 07:57:38 0:52:00 0:15:16 0:36:44 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116158 2018-01-27 03:26:21 2018-01-29 07:07:52 2018-01-29 10:11:55 3:04:03 0:18:37 2:45:26 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/damage.yaml whitelist_health.yaml} 4
fail 2116159 2018-01-27 03:26:21 2018-01-29 07:07:52 2018-01-29 07:37:51 0:29:59 0:18:58 0:11:01 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 07:24:12.115971 mon.b mon.0 172.21.15.1:6789/0 152 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log

pass 2116160 2018-01-27 03:26:22 2018-01-29 07:10:03 2018-01-29 07:34:03 0:24:00 0:10:01 0:13:59 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116161 2018-01-27 03:26:23 2018-01-29 07:10:56 2018-01-29 09:30:58 2:20:02 0:21:34 1:58:28 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
pass 2116162 2018-01-27 03:26:23 2018-01-29 07:11:42 2018-01-29 07:37:41 0:25:59 0:11:06 0:14:53 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116163 2018-01-27 03:26:24 2018-01-29 07:11:42 2018-01-29 07:37:41 0:25:59 0:10:19 0:15:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 2116164 2018-01-27 03:26:24 2018-01-29 07:11:42 2018-01-29 08:21:42 1:10:00 0:19:10 0:50:50 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/failover.yaml whitelist_health.yaml} 4
pass 2116165 2018-01-27 03:26:25 2018-01-29 07:11:49 2018-01-29 07:33:49 0:22:00 0:08:34 0:13:26 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116166 2018-01-27 03:26:26 2018-01-29 07:15:33 2018-01-29 07:47:33 0:32:00 0:08:54 0:23:06 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116167 2018-01-27 03:26:26 2018-01-29 07:15:33 2018-01-29 13:11:40 5:56:07 0:31:25 5:24:42 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 2116168 2018-01-27 03:26:27 2018-01-29 07:15:40 2018-01-29 10:13:43 2:58:03 0:12:08 2:45:55 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 2116169 2018-01-27 03:26:28 2018-01-29 07:15:45 2018-01-29 08:15:46 1:00:01 0:22:56 0:37:05 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116170 2018-01-27 03:26:28 2018-01-29 07:15:49 2018-01-29 08:09:49 0:54:00 0:27:16 0:26:44 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116171 2018-01-27 03:26:29 2018-01-29 07:15:49 2018-01-29 10:41:53 3:26:04 0:17:31 3:08:33 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 2116172 2018-01-27 03:26:30 2018-01-29 07:17:46 2018-01-29 08:59:48 1:42:02 0:25:46 1:16:16 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116173 2018-01-27 03:26:30 2018-01-29 07:17:46 2018-01-29 07:53:46 0:36:00 0:18:51 0:17:09 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116174 2018-01-27 03:26:31 2018-01-29 07:17:51 2018-01-29 16:10:02 8:52:11 0:10:10 8:42:01 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 2116175 2018-01-27 03:26:32 2018-01-29 07:17:59 2018-01-29 08:44:01 1:26:02 0:39:58 0:46:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116176 2018-01-27 03:26:33 2018-01-29 07:22:00 2018-01-29 09:20:02 1:58:02 1:15:18 0:42:44 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116177 2018-01-27 03:26:33 2018-01-29 07:23:41 2018-01-29 09:15:42 1:52:01 0:07:41 1:44:20 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Command failed on smithi201 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true'

pass 2116178 2018-01-27 03:26:34 2018-01-29 07:23:41 2018-01-29 08:13:41 0:50:00 0:37:43 0:12:17 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116179 2018-01-27 03:26:35 2018-01-29 07:25:42 2018-01-29 08:49:42 1:24:00 1:13:53 0:10:07 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 07:42:50.494844 mon.a mon.0 172.21.15.40:6789/0 122 : cluster [WRN] Health check failed: 16308 slow ops, oldest one blocked for 75 sec (SLOW_OPS)" in cluster log

pass 2116180 2018-01-27 03:26:35 2018-01-29 07:25:45 2018-01-29 09:05:46 1:40:01 0:10:08 1:29:53 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 2116181 2018-01-27 03:26:36 2018-01-29 07:27:46 2018-01-29 08:11:46 0:44:00 0:09:23 0:34:37 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
dead 2116182 2018-01-27 03:26:37 2018-01-29 07:27:46 2018-01-29 19:31:23 12:03:37 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116183 2018-01-27 03:26:37 2018-01-29 07:27:46 2018-01-29 10:49:49 3:22:03 0:12:00 3:10:03 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
fail 2116184 2018-01-27 03:26:38 2018-01-29 07:27:46 2018-01-29 07:53:45 0:25:59 0:11:18 0:14:41 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 07:48:26.393815 mon.a mon.0 172.21.15.51:6789/0 122 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

pass 2116185 2018-01-27 03:26:39 2018-01-29 07:27:50 2018-01-29 08:23:50 0:56:00 0:09:20 0:46:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116186 2018-01-27 03:26:39 2018-01-29 07:29:48 2018-01-29 10:25:51 2:56:03 0:27:10 2:28:53 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/strays.yaml whitelist_health.yaml} 4
fail 2116187 2018-01-27 03:26:40 2018-01-29 07:29:49 2018-01-29 08:41:50 1:12:01 0:20:54 0:51:07 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 08:29:39.276944 mon.b mon.0 172.21.15.4:6789/0 153 : cluster [WRN] Health check failed: 17 slow ops, oldest one blocked for 31 sec (SLOW_OPS)" in cluster log

pass 2116188 2018-01-27 03:26:41 2018-01-29 07:31:47 2018-01-29 08:03:46 0:31:59 0:10:30 0:21:29 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 2116189 2018-01-27 03:26:41 2018-01-29 07:33:41 2018-01-29 10:13:44 2:40:03 0:13:14 2:26:49 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

pass 2116190 2018-01-27 03:26:42 2018-01-29 07:33:41 2018-01-29 08:11:41 0:38:00 0:08:21 0:29:39 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116191 2018-01-27 03:26:42 2018-01-29 07:33:50 2018-01-29 07:59:50 0:26:00 0:11:14 0:14:46 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116192 2018-01-27 03:26:43 2018-01-29 07:34:04 2018-01-29 08:42:05 1:08:01 0:47:23 0:20:38 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/filestore-xfs.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
pass 2116193 2018-01-27 03:26:44 2018-01-29 07:36:05 2018-01-29 09:06:06 1:30:01 0:10:46 1:19:15 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
pass 2116194 2018-01-27 03:26:44 2018-01-29 07:37:55 2018-01-29 08:57:56 1:20:01 1:07:31 0:12:30 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 2116195 2018-01-27 03:26:45 2018-01-29 07:37:55 2018-01-29 08:15:55 0:38:00 0:22:20 0:15:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116196 2018-01-27 03:26:46 2018-01-29 07:37:55 2018-01-29 14:56:04 7:18:09 0:09:41 7:08:28 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 2116197 2018-01-27 03:26:46 2018-01-29 07:37:55 2018-01-29 08:15:55 0:38:00 0:23:29 0:14:31 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116198 2018-01-27 03:26:47 2018-01-29 07:39:53 2018-01-29 08:27:53 0:48:00 0:18:56 0:29:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116199 2018-01-27 03:26:48 2018-01-29 07:40:03 2018-01-29 09:42:05 2:02:02 0:15:20 1:46:42 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
pass 2116200 2018-01-27 03:26:48 2018-01-29 07:41:57 2018-01-29 08:33:58 0:52:01 0:40:32 0:11:29 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116201 2018-01-27 03:26:49 2018-01-29 07:43:53 2018-01-29 08:13:53 0:30:00 0:11:11 0:18:49 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116202 2018-01-27 03:26:50 2018-01-29 07:43:53 2018-01-29 09:51:55 2:08:02 0:35:44 1:32:18 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 2116203 2018-01-27 03:26:50 2018-01-29 07:43:53 2018-01-29 08:47:54 1:04:01 0:40:15 0:23:46 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116204 2018-01-27 03:26:51 2018-01-29 07:43:53 2018-01-29 08:37:53 0:54:00 0:18:24 0:35:36 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 08:26:18.984978 mon.b mon.0 172.21.15.176:6789/0 100 : cluster [WRN] Health check failed: 151 slow ops, oldest one blocked for 32 sec (SLOW_OPS)" in cluster log

pass 2116205 2018-01-27 03:26:51 2018-01-29 07:43:53 2018-01-29 09:37:55 1:54:02 0:09:56 1:44:06 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 2116206 2018-01-27 03:26:52 2018-01-29 07:45:34 2018-01-29 08:05:34 0:20:00 0:09:08 0:10:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
dead 2116207 2018-01-27 03:26:53 2018-01-29 07:45:46 2018-01-29 19:49:22 12:03:36 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116208 2018-01-27 03:26:53 2018-01-29 07:47:47 2018-01-29 09:11:48 1:24:01 0:19:04 1:04:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/damage.yaml whitelist_health.yaml} 4
fail 2116209 2018-01-27 03:26:54 2018-01-29 07:51:57 2018-01-29 08:33:58 0:42:01 0:23:21 0:18:40 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 08:15:15.446796 mon.a mon.1 172.21.15.190:6789/0 79 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 2116210 2018-01-27 03:26:55 2018-01-29 07:51:58 2018-01-29 08:19:58 0:28:00 0:09:26 0:18:34 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116211 2018-01-27 03:26:55 2018-01-29 07:53:39 2018-01-29 09:29:41 1:36:02 0:22:57 1:13:05 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
pass 2116212 2018-01-27 03:26:56 2018-01-29 07:53:39 2018-01-29 08:19:39 0:26:00 0:11:24 0:14:36 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116213 2018-01-27 03:26:57 2018-01-29 07:53:47 2018-01-29 08:23:46 0:29:59 0:10:29 0:19:30 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 2116214 2018-01-27 03:26:57 2018-01-29 07:53:47 2018-01-29 09:01:48 1:08:01 0:19:14 0:48:47 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/failover.yaml whitelist_health.yaml} 4
pass 2116215 2018-01-27 03:26:58 2018-01-29 07:55:58 2018-01-29 08:15:58 0:20:00 0:07:57 0:12:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116216 2018-01-27 03:26:59 2018-01-29 07:57:44 2018-01-29 08:25:44 0:28:00 0:10:16 0:17:44 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116217 2018-01-27 03:26:59 2018-01-29 07:57:49 2018-01-29 14:05:56 6:08:07 0:35:45 5:32:22 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 2116218 2018-01-27 03:27:00 2018-01-29 07:59:40 2018-01-29 16:21:51 8:22:11 0:13:14 8:08:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 2116219 2018-01-27 03:27:01 2018-01-29 07:59:43 2018-01-29 08:35:43 0:36:00 0:16:01 0:19:59 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116220 2018-01-27 03:27:01 2018-01-29 07:59:44 2018-01-29 08:33:44 0:34:00 0:22:41 0:11:19 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116221 2018-01-27 03:27:02 2018-01-29 07:59:48 2018-01-29 09:57:49 1:58:01 0:17:49 1:40:12 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 2116222 2018-01-27 03:27:02 2018-01-29 07:59:48 2018-01-29 08:57:48 0:58:00 0:29:47 0:28:13 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116223 2018-01-27 03:27:03 2018-01-29 07:59:51 2018-01-29 08:29:50 0:29:59 0:18:31 0:11:28 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116224 2018-01-27 03:27:04 2018-01-29 08:01:46 2018-01-29 08:41:46 0:40:00 0:09:38 0:30:22 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 2116225 2018-01-27 03:27:04 2018-01-29 08:01:46 2018-01-29 09:07:47 1:06:01 0:36:40 0:29:21 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116226 2018-01-27 03:27:05 2018-01-29 08:01:46 2018-01-29 08:47:46 0:46:00 0:22:59 0:23:01 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116227 2018-01-27 03:27:06 2018-01-29 08:03:43 2018-01-29 08:45:43 0:42:00 0:10:53 0:31:07 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_full_different_file (tasks.cephfs.test_full.TestClusterFull)

pass 2116228 2018-01-27 03:27:06 2018-01-29 08:03:47 2018-01-29 09:05:48 1:02:01 0:42:11 0:19:50 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116229 2018-01-27 03:27:07 2018-01-29 08:05:48 2018-01-29 08:43:48 0:38:00 0:19:11 0:18:49 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 08:31:38.372646 mon.b mon.0 172.21.15.34:6789/0 102 : cluster [WRN] Health check failed: 4106 slow ops, oldest one blocked for 33 sec (SLOW_OPS)" in cluster log

pass 2116230 2018-01-27 03:27:08 2018-01-29 08:07:48 2018-01-29 09:49:50 1:42:02 0:09:50 1:32:12 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 2116231 2018-01-27 03:27:08 2018-01-29 08:09:41 2018-01-29 08:33:40 0:23:59 0:09:33 0:14:26 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116232 2018-01-27 03:27:09 2018-01-29 08:09:50 2018-01-29 08:41:50 0:32:00 0:17:58 0:14:02 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116233 2018-01-27 03:27:10 2018-01-29 08:11:45 2018-01-29 14:55:53 6:44:08 0:12:04 6:32:04 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
fail 2116234 2018-01-27 03:27:10 2018-01-29 08:11:45 2018-01-29 08:39:45 0:28:00 0:11:19 0:16:41 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 08:34:16.079984 mon.b mon.0 172.21.15.97:6789/0 183 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log

pass 2116235 2018-01-27 03:27:11 2018-01-29 08:11:47 2018-01-29 08:33:46 0:21:59 0:09:36 0:12:23 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116236 2018-01-27 03:27:12 2018-01-29 08:13:55 2018-01-29 11:27:59 3:14:04 0:26:19 2:47:45 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/strays.yaml whitelist_health.yaml} 4
pass 2116237 2018-01-27 03:27:12 2018-01-29 08:13:55 2018-01-29 08:35:55 0:22:00 0:11:47 0:10:13 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116238 2018-01-27 03:27:13 2018-01-29 08:16:00 2018-01-29 08:43:59 0:27:59 0:10:32 0:17:27 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 2116239 2018-01-27 03:27:14 2018-01-29 08:16:00 2018-01-29 11:48:04 3:32:04 0:12:52 3:19:12 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

pass 2116240 2018-01-27 03:27:14 2018-01-29 08:16:00 2018-01-29 08:40:00 0:24:00 0:08:31 0:15:29 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116241 2018-01-27 03:27:15 2018-01-29 08:16:00 2018-01-29 08:44:00 0:28:00 0:09:10 0:18:50 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116242 2018-01-27 03:27:16 2018-01-29 08:19:55 2018-01-29 11:37:58 3:18:03 0:48:10 2:29:53 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-comp.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
pass 2116243 2018-01-27 03:27:16 2018-01-29 08:19:59 2018-01-29 09:06:00 0:46:01 0:10:29 0:35:32 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
pass 2116244 2018-01-27 03:27:17 2018-01-29 08:21:56 2018-01-29 09:01:56 0:40:00 0:22:46 0:17:14 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 2116245 2018-01-27 03:27:18 2018-01-29 08:21:56 2018-01-29 08:59:56 0:38:00 0:20:27 0:17:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116246 2018-01-27 03:27:18 2018-01-29 08:23:53 2018-01-29 08:43:52 0:19:59 0:09:26 0:10:33 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 2116247 2018-01-27 03:27:19 2018-01-29 08:23:53 2018-01-29 09:11:53 0:48:00 0:29:09 0:18:51 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116248 2018-01-27 03:27:20 2018-01-29 08:23:53 2018-01-29 08:55:53 0:32:00 0:18:40 0:13:20 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116249 2018-01-27 03:27:20 2018-01-29 08:23:53 2018-01-29 10:53:55 2:30:02 0:15:42 2:14:20 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
pass 2116250 2018-01-27 03:27:21 2018-01-29 08:24:03 2018-01-29 09:30:04 1:06:01 0:48:03 0:17:58 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116251 2018-01-27 03:27:22 2018-01-29 08:25:58 2018-01-29 09:03:58 0:38:00 0:21:15 0:16:45 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116252 2018-01-27 03:27:22 2018-01-29 08:28:07 2018-01-29 09:30:08 1:02:01 0:34:51 0:27:10 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 2116253 2018-01-27 03:27:23 2018-01-29 08:29:55 2018-01-29 09:21:55 0:52:00 0:38:55 0:13:05 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116254 2018-01-27 03:27:24 2018-01-29 08:29:55 2018-01-29 09:07:55 0:38:00 0:23:13 0:14:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 08:50:35.982321 mon.b mon.0 172.21.15.181:6789/0 99 : cluster [WRN] Health check failed: 9 slow ops, oldest one blocked for 32 sec (SLOW_OPS)" in cluster log

pass 2116255 2018-01-27 03:27:24 2018-01-29 08:33:55 2018-01-29 09:25:55 0:52:00 0:10:35 0:41:25 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 2116256 2018-01-27 03:27:25 2018-01-29 08:33:55 2018-01-29 08:57:54 0:23:59 0:09:59 0:14:00 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116257 2018-01-27 03:27:26 2018-01-29 08:33:55 2018-01-29 09:01:54 0:27:59 0:17:46 0:10:13 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116258 2018-01-27 03:27:26 2018-01-29 08:33:59 2018-01-29 13:38:05 5:04:06 0:18:25 4:45:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/damage.yaml whitelist_health.yaml} 4
fail 2116259 2018-01-27 03:27:27 2018-01-29 08:33:59 2018-01-29 09:42:00 1:08:01 0:52:17 0:15:44 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 08:56:16.489602 mon.a mon.1 172.21.15.147:6789/0 42 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 2116260 2018-01-27 03:27:28 2018-01-29 08:35:48 2018-01-29 08:55:48 0:20:00 0:09:53 0:10:07 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116261 2018-01-27 03:27:28 2018-01-29 08:35:56 2018-01-29 09:51:57 1:16:01 0:21:23 0:54:38 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
pass 2116262 2018-01-27 03:27:29 2018-01-29 08:38:07 2018-01-29 08:58:07 0:20:00 0:10:13 0:09:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116263 2018-01-27 03:27:30 2018-01-29 08:40:00 2018-01-29 09:02:00 0:22:00 0:10:22 0:11:38 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 2116264 2018-01-27 03:27:30 2018-01-29 08:40:00 2018-01-29 09:38:01 0:58:01 0:18:56 0:39:05 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/failover.yaml whitelist_health.yaml} 4
pass 2116265 2018-01-27 03:27:31 2018-01-29 08:41:59 2018-01-29 09:01:59 0:20:00 0:08:56 0:11:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116266 2018-01-27 03:27:32 2018-01-29 08:42:00 2018-01-29 09:01:59 0:19:59 0:09:31 0:10:28 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116267 2018-01-27 03:27:32 2018-01-29 08:42:00 2018-01-29 11:12:02 2:30:02 0:37:59 1:52:03 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 2116268 2018-01-27 03:27:33 2018-01-29 08:42:06 2018-01-29 09:08:06 0:26:00 0:12:19 0:13:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 2116269 2018-01-27 03:27:34 2018-01-29 08:44:02 2018-01-29 09:24:02 0:40:00 0:12:09 0:27:51 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116270 2018-01-27 03:27:34 2018-01-29 08:44:02 2018-01-29 09:32:03 0:48:01 0:26:07 0:21:54 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116271 2018-01-27 03:27:35 2018-01-29 08:44:02 2018-01-29 16:56:12 8:12:10 0:18:14 7:53:56 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 2116272 2018-01-27 03:27:35 2018-01-29 08:44:02 2018-01-29 09:36:03 0:52:01 0:24:56 0:27:05 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116273 2018-01-27 03:27:36 2018-01-29 08:44:02 2018-01-29 09:48:03 1:04:01 0:18:08 0:45:53 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116274 2018-01-27 03:27:37 2018-01-29 08:45:57 2018-01-29 09:29:58 0:44:01 0:09:25 0:34:36 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 2116275 2018-01-27 03:27:37 2018-01-29 08:48:00 2018-01-29 09:46:01 0:58:01 0:49:20 0:08:41 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116276 2018-01-27 03:27:38 2018-01-29 08:48:00 2018-01-29 09:54:01 1:06:01 0:49:21 0:16:40 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116277 2018-01-27 03:27:39 2018-01-29 08:49:56 2018-01-29 09:21:56 0:32:00 0:10:43 0:21:17 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_full_different_file (tasks.cephfs.test_full.TestClusterFull)

pass 2116278 2018-01-27 03:27:39 2018-01-29 08:56:02 2018-01-29 09:46:03 0:50:01 0:38:30 0:11:31 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116279 2018-01-27 03:27:40 2018-01-29 08:56:02 2018-01-29 10:16:03 1:20:01 1:09:37 0:10:24 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 09:14:25.441213 mon.a mon.0 172.21.15.178:6789/0 123 : cluster [WRN] Health check failed: 16884 slow ops, oldest one blocked for 79 sec (SLOW_OPS)" in cluster log

pass 2116280 2018-01-27 03:27:41 2018-01-29 08:58:00 2018-01-29 10:30:02 1:32:02 0:09:47 1:22:15 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 2116281 2018-01-27 03:27:41 2018-01-29 08:58:01 2018-01-29 09:20:00 0:21:59 0:09:40 0:12:19 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116282 2018-01-27 03:27:42 2018-01-29 08:58:01 2018-01-29 09:24:00 0:25:59 0:14:47 0:11:12 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116283 2018-01-27 03:27:43 2018-01-29 08:58:00 2018-01-29 12:22:04 3:24:04 0:12:00 3:12:04 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
fail 2116284 2018-01-27 03:27:43 2018-01-29 08:58:08 2018-01-29 09:20:08 0:22:00 0:11:22 0:10:38 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 09:14:15.462029 mon.a mon.0 172.21.15.146:6789/0 26 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

pass 2116285 2018-01-27 03:27:44 2018-01-29 09:00:02 2018-01-29 09:22:01 0:21:59 0:09:55 0:12:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116286 2018-01-27 03:27:45 2018-01-29 09:00:02 2018-01-29 13:18:06 4:18:04 0:25:54 3:52:10 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/strays.yaml whitelist_health.yaml} 4
pass 2116287 2018-01-27 03:27:45 2018-01-29 09:02:01 2018-01-29 09:24:01 0:22:00 0:11:09 0:10:51 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116288 2018-01-27 03:27:46 2018-01-29 09:02:01 2018-01-29 09:28:01 0:26:00 0:10:31 0:15:29 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 2116289 2018-01-27 03:27:46 2018-01-29 09:02:01 2018-01-29 12:14:05 3:12:04 0:12:51 2:59:13 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

pass 2116290 2018-01-27 03:27:47 2018-01-29 09:02:01 2018-01-29 09:22:01 0:20:00 0:08:23 0:11:37 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116291 2018-01-27 03:27:48 2018-01-29 09:02:01 2018-01-29 09:26:01 0:24:00 0:08:59 0:15:01 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116292 2018-01-27 03:27:48 2018-01-29 09:02:01 2018-01-29 10:46:03 1:44:02 0:46:59 0:57:03 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/bluestore.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
pass 2116293 2018-01-27 03:27:49 2018-01-29 09:04:04 2018-01-29 10:00:05 0:56:01 0:10:05 0:45:56 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
pass 2116294 2018-01-27 03:27:50 2018-01-29 09:06:02 2018-01-29 09:50:02 0:44:00 0:23:00 0:21:00 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116295 2018-01-27 03:27:50 2018-01-29 09:06:02 2018-01-29 10:24:03 1:18:01 0:27:08 0:50:53 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
Failure Reason:

"2018-01-29 10:02:54.294510 mon.b mon.0 172.21.15.76:6789/0 120 : cluster [WRN] Health check failed: 7180 slow ops, oldest one blocked for 32 sec (SLOW_OPS)" in cluster log

pass 2116296 2018-01-27 03:27:51 2018-01-29 09:06:02 2018-01-29 14:54:09 5:48:07 0:10:07 5:38:00 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 2116297 2018-01-27 03:27:52 2018-01-29 09:06:07 2018-01-29 10:00:07 0:54:00 0:26:05 0:27:55 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116298 2018-01-27 03:27:52 2018-01-29 09:08:00 2018-01-29 09:40:00 0:32:00 0:18:58 0:13:02 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116299 2018-01-27 03:27:53 2018-01-29 09:08:00 2018-01-29 09:56:01 0:48:01 0:16:27 0:31:34 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
pass 2116300 2018-01-27 03:27:54 2018-01-29 09:08:07 2018-01-29 10:12:08 1:04:01 0:39:49 0:24:12 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116301 2018-01-27 03:27:54 2018-01-29 09:11:54 2018-01-29 09:41:54 0:30:00 0:12:03 0:17:57 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp-ec-root.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116302 2018-01-27 03:27:55 2018-01-29 09:11:54 2018-01-29 10:25:55 1:14:01 0:36:36 0:37:25 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 2116303 2018-01-27 03:27:55 2018-01-29 09:15:56 2018-01-29 10:03:56 0:48:00 0:36:36 0:11:24 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116304 2018-01-27 03:27:56 2018-01-29 09:19:53 2018-01-29 10:45:54 1:26:01 1:14:48 0:11:13 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 09:38:29.071898 mon.a mon.0 172.21.15.61:6789/0 119 : cluster [WRN] Health check failed: 15553 slow ops, oldest one blocked for 64 sec (SLOW_OPS)" in cluster log

pass 2116305 2018-01-27 03:27:57 2018-01-29 09:20:01 2018-01-29 09:50:01 0:30:00 0:10:28 0:19:32 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 2116306 2018-01-27 03:27:57 2018-01-29 09:20:03 2018-01-29 09:46:03 0:26:00 0:09:50 0:16:10 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116307 2018-01-27 03:27:58 2018-01-29 09:20:12 2018-01-29 09:56:12 0:36:00 0:23:39 0:12:21 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116308 2018-01-27 03:27:59 2018-01-29 09:21:44 2018-01-29 10:41:44 1:20:00 0:19:32 1:00:28 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/damage.yaml whitelist_health.yaml} 4
fail 2116309 2018-01-27 03:27:59 2018-01-29 09:21:56 2018-01-29 09:53:56 0:32:00 0:19:44 0:12:16 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-comp.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 09:39:08.622265 mon.b mon.0 172.21.15.17:6789/0 107 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log

pass 2116310 2018-01-27 03:28:00 2018-01-29 09:21:57 2018-01-29 09:59:57 0:38:00 0:09:20 0:28:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116311 2018-01-27 03:28:01 2018-01-29 09:22:02 2018-01-29 12:52:06 3:30:04 0:22:18 3:07:46 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
pass 2116312 2018-01-27 03:28:01 2018-01-29 09:22:02 2018-01-29 09:56:02 0:34:00 0:21:25 0:12:35 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116313 2018-01-27 03:28:02 2018-01-29 09:24:14 2018-01-29 09:50:14 0:26:00 0:10:04 0:15:56 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 2116314 2018-01-27 03:28:03 2018-01-29 09:24:14 2018-01-29 12:32:17 3:08:03 0:19:22 2:48:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/failover.yaml whitelist_health.yaml} 4
pass 2116315 2018-01-27 03:28:03 2018-01-29 09:24:14 2018-01-29 10:04:14 0:40:00 0:08:10 0:31:50 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 2116316 2018-01-27 03:28:04 2018-01-29 09:26:09 2018-01-29 09:48:09 0:22:00 0:09:24 0:12:36 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 2116317 2018-01-27 03:28:05 2018-01-29 09:26:09 2018-01-29 10:28:11 1:02:02 0:28:36 0:33:26 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore-ec/filestore-xfs.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 2116318 2018-01-27 03:28:05 2018-01-29 09:28:15 2018-01-29 13:16:19 3:48:04 0:12:59 3:35:05 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 2116319 2018-01-27 03:28:06 2018-01-29 09:29:56 2018-01-29 10:11:56 0:42:00 0:13:17 0:28:43 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore-ec-root.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 2116320 2018-01-27 03:28:07 2018-01-29 09:30:00 2018-01-29 10:12:02 0:42:02 0:22:26 0:19:36 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 2116321 2018-01-27 03:28:07 2018-01-29 09:30:05 2018-01-29 10:42:06 1:12:01 0:17:26 0:54:35 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 2116322 2018-01-27 03:28:08 2018-01-29 09:30:09 2018-01-29 10:08:10 0:38:01 0:23:21 0:14:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_misc.yaml} 3
pass 2116323 2018-01-27 03:28:08 2018-01-29 09:31:13 2018-01-29 10:11:13 0:40:00 0:18:54 0:21:06 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 2116324 2018-01-27 03:28:09 2018-01-29 09:32:04 2018-01-29 10:16:04 0:44:00 0:09:03 0:34:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 2116325 2018-01-27 03:28:10 2018-01-29 09:36:17 2018-01-29 10:36:17 1:00:00 0:39:51 0:20:09 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 2116326 2018-01-27 03:28:10 2018-01-29 09:38:09 2018-01-29 10:14:11 0:36:02 0:19:48 0:16:14 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/bluestore.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 2116327 2018-01-27 03:28:11 2018-01-29 09:38:09 2018-01-29 10:42:11 1:04:02 0:07:24 0:56:38 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Command failed on smithi015 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data allow_ec_overwrites true'

pass 2116328 2018-01-27 03:28:12 2018-01-29 09:40:16 2018-01-29 10:42:17 1:02:01 0:40:09 0:21:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
fail 2116329 2018-01-27 03:28:12 2018-01-29 09:42:08 2018-01-29 10:24:08 0:42:00 0:18:07 0:23:53 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2018-01-29 10:11:44.404250 mon.a mon.0 172.21.15.80:6789/0 113 : cluster [WRN] Health check failed: 52 slow ops, oldest one blocked for 32 sec (SLOW_OPS)" in cluster log

pass 2116330 2018-01-27 03:28:13 2018-01-29 09:42:08 2018-01-29 11:24:09 1:42:01 0:09:56 1:32:05 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 2116331 2018-01-27 03:28:14 2018-01-29 09:42:08 2018-01-29 10:06:08 0:24:00 0:09:42 0:14:18 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 2116332 2018-01-27 03:28:14 2018-01-29 09:44:00 2018-01-29 10:28:00 0:44:00 0:24:08 0:19:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 2116333 2018-01-27 03:28:15 2018-01-29 09:46:15 2018-01-29 10:34:15 0:48:00 0:12:19 0:35:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
fail 2116334 2018-01-27 03:28:16 2018-01-29 09:46:15 2018-01-29 10:18:14 0:31:59 0:22:03 0:09:56 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore-ec/filestore-xfs.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2018-01-29 10:02:40.495032 mon.a mon.0 172.21.15.2:6789/0 25 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

pass 2116335 2018-01-27 03:28:16 2018-01-29 09:46:15 2018-01-29 10:18:14 0:31:59 0:09:12 0:22:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore-comp.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
pass 2116336 2018-01-27 03:28:17 2018-01-29 09:47:44 2018-01-29 10:51:45 1:04:01 0:26:05 0:37:56 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/bluestore.yaml tasks/strays.yaml whitelist_health.yaml} 4
pass 2116337 2018-01-27 03:28:18 2018-01-29 09:47:49 2018-01-29 10:13:48 0:25:59 0:11:29 0:14:30 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 2116338 2018-01-27 03:28:18 2018-01-29 09:48:04 2018-01-29 10:22:04 0:34:00 0:10:01 0:23:59 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore-ec/bluestore.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 2116339 2018-01-27 03:28:19 2018-01-29 09:48:10 2018-01-29 16:00:18 6:12:08 0:12:30 5:59:38 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient)

pass 2116340 2018-01-27 03:28:20 2018-01-29 09:50:05 2018-01-29 10:18:05 0:28:00 0:08:26 0:19:34 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore-ec/filestore-xfs.yaml tasks/kclient_workunit_trivial_sync.yaml} 3