Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 1640861 2017-09-16 05:20:16 2017-09-19 19:16:49 2017-09-19 19:36:49 0:20:00 0:10:20 0:09:40 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 1640862 2017-09-16 05:20:17 2017-09-19 19:16:50 2017-09-19 21:34:53 2:18:03 0:50:30 1:27:33 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore/bluestore.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
fail 1640863 2017-09-16 05:20:18 2017-09-19 19:18:50 2017-09-19 20:12:50 0:54:00 0:12:42 0:41:18 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 20:07:00.462205 mon.a mon.0 172.21.15.135:6789/0 526 : cluster [WRN] Health check failed: 1 MDSs are read only (MDS_READ_ONLY)" in cluster log

pass 1640864 2017-09-16 05:20:18 2017-09-19 19:18:50 2017-09-19 19:48:50 0:30:00 0:22:15 0:07:45 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 1640865 2017-09-16 05:20:19 2017-09-19 19:18:50 2017-09-19 19:52:50 0:34:00 0:23:43 0:10:17 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 1640866 2017-09-16 05:20:20 2017-09-19 19:18:50 2017-09-19 20:48:51 1:30:01 0:10:30 1:19:31 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 1640867 2017-09-16 05:20:20 2017-09-19 19:18:57 2017-09-19 19:52:57 0:34:00 0:29:27 0:04:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_misc.yaml} 3
pass 1640868 2017-09-16 05:20:21 2017-09-19 19:19:06 2017-09-19 19:41:05 0:21:59 0:19:56 0:02:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_o_trunc.yaml} 3
fail 1640869 2017-09-16 05:20:21 2017-09-19 19:19:15 2017-09-19 19:45:15 0:26:00 0:15:51 0:10:09 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 19:34:54.462525 mon.a mon.0 172.21.15.82:6789/0 664 : cluster [WRN] MDS health message (mds.0): MDS cache is too large (521kB/1GB); 202 inodes in use by clients, 0 stray files" in cluster log

pass 1640870 2017-09-16 05:20:22 2017-09-19 19:20:54 2017-09-19 20:20:54 1:00:00 0:35:34 0:24:26 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 1640871 2017-09-16 05:20:23 2017-09-19 19:20:54 2017-09-19 20:02:53 0:41:59 0:25:01 0:16:58 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
dead 1640872 2017-09-16 05:20:24 2017-09-19 19:21:08 2017-09-20 07:27:49 12:06:41 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
pass 1640873 2017-09-16 05:20:24 2017-09-19 19:22:29 2017-09-19 20:06:30 0:44:01 0:38:57 0:05:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
pass 1640874 2017-09-16 05:20:25 2017-09-19 19:22:48 2017-09-19 19:42:47 0:19:59 0:19:22 0:00:37 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
pass 1640875 2017-09-16 05:20:26 2017-09-19 19:22:50 2017-09-19 19:48:50 0:26:00 0:11:47 0:14:13 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 1640876 2017-09-16 05:20:26 2017-09-19 19:22:51 2017-09-19 19:36:51 0:14:00 0:08:48 0:05:12 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 1640877 2017-09-16 05:20:27 2017-09-19 19:22:59 2017-09-19 19:46:59 0:24:00 0:17:35 0:06:25 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 1640878 2017-09-16 05:20:28 2017-09-19 19:23:50 2017-09-19 20:21:51 0:58:01 0:22:50 0:35:11 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/damage.yaml whitelist_health.yaml} 4
pass 1640879 2017-09-16 05:20:28 2017-09-19 19:23:50 2017-09-19 19:49:50 0:26:00 0:19:35 0:06:25 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 1640880 2017-09-16 05:20:29 2017-09-19 19:24:28 2017-09-19 19:42:28 0:18:00 0:10:17 0:07:43 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
fail 1640881 2017-09-16 05:20:29 2017-09-19 19:24:47 2017-09-19 20:10:47 0:46:00 0:23:21 0:22:39 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_rebuild_simple_altpool (tasks.cephfs.test_data_scan.TestDataScan)

pass 1640882 2017-09-16 05:20:30 2017-09-19 19:25:06 2017-09-19 19:37:06 0:12:00 0:10:10 0:01:50 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 1640883 2017-09-16 05:20:31 2017-09-19 19:26:33 2017-09-19 20:04:33 0:38:00 0:11:27 0:26:33 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 1640884 2017-09-16 05:20:31 2017-09-19 19:26:53 2017-09-19 20:00:53 0:34:00 0:19:46 0:14:14 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/failover.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 19:44:00.093122 mon.a mon.0 172.21.15.73:6789/0 457 : cluster [WRN] daemon mds.d is not responding, replacing it as rank 0 with standby daemon mds.b" in cluster log

pass 1640885 2017-09-16 05:20:32 2017-09-19 19:26:53 2017-09-19 19:48:52 0:21:59 0:07:54 0:14:05 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 1640886 2017-09-16 05:20:33 2017-09-19 19:26:53 2017-09-19 19:42:52 0:15:59 0:10:17 0:05:42 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 1640887 2017-09-16 05:20:33 2017-09-19 19:26:53 2017-09-19 20:26:53 1:00:00 0:31:33 0:28:27 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore/filestore-xfs.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
pass 1640888 2017-09-16 05:20:34 2017-09-19 19:27:07 2017-09-19 19:43:07 0:16:00 0:14:06 0:01:54 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 1640889 2017-09-16 05:20:35 2017-09-19 19:27:08 2017-09-19 19:53:07 0:25:59 0:24:30 0:01:29 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 1640890 2017-09-16 05:20:35 2017-09-19 19:28:42 2017-09-19 19:52:41 0:23:59 0:22:31 0:01:28 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 1640891 2017-09-16 05:20:36 2017-09-19 19:28:50 2017-09-19 20:28:50 1:00:00 0:21:41 0:38:19 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
pass 1640892 2017-09-16 05:20:37 2017-09-19 19:28:55 2017-09-19 20:02:55 0:34:00 0:25:57 0:08:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_misc.yaml} 3
pass 1640893 2017-09-16 05:20:37 2017-09-19 19:28:55 2017-09-19 19:58:55 0:30:00 0:18:52 0:11:08 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 1640894 2017-09-16 05:20:38 2017-09-19 19:29:04 2017-09-19 19:57:04 0:28:00 0:11:21 0:16:39 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 1640895 2017-09-16 05:20:39 2017-09-19 19:29:33 2017-09-19 20:07:32 0:37:59 0:35:27 0:02:32 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 1640896 2017-09-16 05:20:39 2017-09-19 19:30:50 2017-09-19 19:54:48 0:23:58 0:20:55 0:03:03 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
fail 1640897 2017-09-16 05:20:40 2017-09-19 19:30:50 2017-09-19 20:58:51 1:28:01 0:08:33 1:19:28 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Command failed on smithi031 with status 1: 'sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmpTCTf6s'

fail 1640898 2017-09-16 05:20:40 2017-09-19 19:30:58 2017-09-19 20:18:58 0:48:00 0:38:59 0:09:01 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
Failure Reason:

"2017-09-19 19:56:44.624079 mon.b mon.0 172.21.15.81:6789/0 272 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 1640899 2017-09-16 05:20:41 2017-09-19 19:30:59 2017-09-19 20:04:58 0:33:59 0:25:31 0:08:28 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
pass 1640900 2017-09-16 05:20:42 2017-09-19 19:32:35 2017-09-19 19:56:35 0:24:00 0:11:05 0:12:55 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 1640901 2017-09-16 05:20:42 2017-09-19 19:32:35 2017-09-19 20:00:35 0:28:00 0:10:56 0:17:04 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 1640902 2017-09-16 05:20:43 2017-09-19 19:32:48 2017-09-19 19:50:47 0:17:59 0:14:20 0:03:39 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 1640903 2017-09-16 05:20:44 2017-09-19 19:32:48 2017-09-19 20:28:49 0:56:01 0:13:02 0:42:59 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
pass 1640904 2017-09-16 05:20:44 2017-09-19 19:34:39 2017-09-19 20:02:39 0:28:00 0:24:19 0:03:41 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 1640905 2017-09-16 05:20:45 2017-09-19 19:34:42 2017-09-19 19:46:42 0:12:00 0:09:21 0:02:39 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
fail 1640906 2017-09-16 05:20:46 2017-09-19 19:34:43 2017-09-19 21:24:45 1:50:02 0:30:10 1:19:52 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/strays.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)

pass 1640907 2017-09-16 05:20:46 2017-09-19 19:34:53 2017-09-19 20:10:53 0:36:00 0:24:05 0:11:55 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 1640908 2017-09-16 05:20:47 2017-09-19 19:36:43 2017-09-19 19:48:42 0:11:59 0:09:32 0:02:27 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 1640909 2017-09-16 05:20:47 2017-09-19 19:36:48 2017-09-19 20:44:48 1:08:00 0:16:29 0:51:31 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
pass 1640910 2017-09-16 05:20:48 2017-09-19 19:36:50 2017-09-19 19:52:49 0:15:59 0:08:44 0:07:15 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 1640911 2017-09-16 05:20:49 2017-09-19 19:36:52 2017-09-19 19:58:51 0:21:59 0:09:58 0:12:01 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_direct_io.yaml} 3
pass 1640912 2017-09-16 05:20:49 2017-09-19 19:36:56 2017-09-19 20:34:57 0:58:01 0:41:20 0:16:41 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore/filestore-xfs.yaml tasks/kernel_cfuse_workunits_dbench_iozone.yaml} 4
fail 1640913 2017-09-16 05:20:50 2017-09-19 19:37:07 2017-09-19 20:09:07 0:32:00 0:12:03 0:19:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/auto-repair.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 20:02:19.458289 mon.a mon.0 172.21.15.11:6789/0 511 : cluster [WRN] Health check failed: 1 MDSs are read only (MDS_READ_ONLY)" in cluster log

pass 1640914 2017-09-16 05:20:51 2017-09-19 19:38:41 2017-09-19 20:18:41 0:40:00 0:26:44 0:13:16 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 1640915 2017-09-16 05:20:51 2017-09-19 19:38:42 2017-09-19 20:06:42 0:28:00 0:25:08 0:02:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
pass 1640916 2017-09-16 05:20:52 2017-09-19 19:38:47 2017-09-19 20:14:47 0:36:00 0:11:36 0:24:24 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/backtrace.yaml whitelist_health.yaml} 4
pass 1640917 2017-09-16 05:20:52 2017-09-19 19:38:47 2017-09-19 20:10:47 0:32:00 0:28:36 0:03:24 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_misc.yaml} 3
pass 1640918 2017-09-16 05:20:53 2017-09-19 19:38:50 2017-09-19 20:00:50 0:22:00 0:17:45 0:04:15 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_o_trunc.yaml} 3
fail 1640919 2017-09-16 05:20:54 2017-09-19 19:38:57 2017-09-19 19:58:56 0:19:59 0:12:30 0:07:29 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/client-limits.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 19:49:43.505163 mon.a mon.0 172.21.15.179:6789/0 644 : cluster [WRN] MDS health message (mds.0): MDS cache is too large (496kB/1GB); 202 inodes in use by clients, 0 stray files" in cluster log

pass 1640920 2017-09-16 05:20:54 2017-09-19 19:40:51 2017-09-19 20:30:51 0:50:00 0:36:55 0:13:05 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_snaps.yaml} 3
pass 1640921 2017-09-16 05:20:55 2017-09-19 19:40:50 2017-09-19 20:02:50 0:22:00 0:11:04 0:10:56 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
fail 1640922 2017-09-16 05:20:56 2017-09-19 19:40:50 2017-09-19 21:18:52 1:38:02 1:09:27 0:28:35 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/client-recovery.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 20:19:38.787698 mds.a mds.0 172.21.15.39:6805/2385220844 1 : cluster [WRN] evicting unresponsive client smithi074: (4315), after waiting 45 seconds during MDS startup" in cluster log

pass 1640923 2017-09-16 05:20:56 2017-09-19 19:41:08 2017-09-19 20:25:08 0:44:00 0:37:45 0:06:15 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
pass 1640924 2017-09-16 05:20:57 2017-09-19 19:42:40 2017-09-19 20:04:40 0:22:00 0:19:08 0:02:52 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
pass 1640925 2017-09-16 05:20:58 2017-09-19 19:42:48 2017-09-19 20:20:48 0:38:00 0:11:35 0:26:25 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/config-commands.yaml whitelist_health.yaml} 4
pass 1640926 2017-09-16 05:20:58 2017-09-19 19:42:49 2017-09-19 19:54:49 0:12:00 0:09:09 0:02:51 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 1640927 2017-09-16 05:20:59 2017-09-19 19:42:53 2017-09-19 20:06:53 0:24:00 0:16:09 0:07:51 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 1640928 2017-09-16 05:20:59 2017-09-19 19:42:58 2017-09-19 20:20:58 0:38:00 0:19:09 0:18:51 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/damage.yaml whitelist_health.yaml} 4
pass 1640929 2017-09-16 05:21:00 2017-09-19 19:43:08 2017-09-19 20:11:07 0:27:59 0:25:05 0:02:54 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
pass 1640930 2017-09-16 05:21:01 2017-09-19 19:44:49 2017-09-19 20:24:49 0:40:00 0:10:57 0:29:03 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
fail 1640931 2017-09-16 05:21:01 2017-09-19 19:44:50 2017-09-19 20:54:50 1:10:00 0:24:43 0:45:17 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/data-scan.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_rebuild_simple_altpool (tasks.cephfs.test_data_scan.TestDataScan)

pass 1640932 2017-09-16 05:21:02 2017-09-19 19:44:50 2017-09-19 20:00:49 0:15:59 0:10:10 0:05:49 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
pass 1640933 2017-09-16 05:21:02 2017-09-19 19:45:16 2017-09-19 19:57:15 0:11:59 0:09:45 0:02:14 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
fail 1640934 2017-09-16 05:21:03 2017-09-19 19:46:51 2017-09-19 20:16:50 0:29:59 0:20:01 0:09:58 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/failover.yaml whitelist_health.yaml} 4
Failure Reason:

"2017-09-19 19:59:57.822448 mon.a mon.0 172.21.15.75:6789/0 461 : cluster [WRN] daemon mds.b is not responding, replacing it as rank 0 with standby daemon mds.d" in cluster log

pass 1640935 2017-09-16 05:21:04 2017-09-19 19:46:51 2017-09-19 20:00:50 0:13:59 0:09:12 0:04:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_trivial_sync.yaml} 3
pass 1640936 2017-09-16 05:21:04 2017-09-19 19:46:50 2017-09-19 20:08:50 0:22:00 0:09:47 0:12:13 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_direct_io.yaml} 3
fail 1640937 2017-09-16 05:21:05 2017-09-19 19:47:00 2017-09-19 20:35:00 0:48:00 0:33:41 0:14:19 smithi master kcephfs/mixed-clients/{clusters/2-clients.yaml conf.yaml objectstore/bluestore.yaml tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} 4
Failure Reason:

"2017-09-19 20:06:52.388837 mon.b mon.0 172.21.15.7:6789/0 124 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 1640938 2017-09-16 05:21:06 2017-09-19 19:48:50 2017-09-19 20:16:50 0:28:00 0:13:49 0:14:11 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/forward-scrub.yaml whitelist_health.yaml} 4
pass 1640939 2017-09-16 05:21:06 2017-09-19 19:48:50 2017-09-19 20:02:50 0:14:00 0:10:15 0:03:45 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
fail 1640940 2017-09-16 05:21:07 2017-09-19 19:48:51 2017-09-19 20:46:51 0:58:00 0:24:12 0:33:48 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_kernel_untar_build.yaml} 3
Failure Reason:

"2017-09-19 20:26:27.273134 mon.b mon.0 172.21.15.68:6789/0 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 1640941 2017-09-16 05:21:07 2017-09-19 19:48:51 2017-09-19 20:40:51 0:52:00 0:19:03 0:32:57 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/journal-repair.yaml whitelist_health.yaml} 4
fail 1640942 2017-09-16 05:21:08 2017-09-19 19:48:53 2017-09-19 20:32:53 0:44:00 0:24:40 0:19:20 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_misc.yaml} 3
Failure Reason:

"2017-09-19 20:12:36.751773 mon.b mon.0 172.21.15.64:6789/0 133 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 1640943 2017-09-16 05:21:09 2017-09-19 19:48:54 2017-09-19 20:18:54 0:30:00 0:19:53 0:10:07 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_o_trunc.yaml} 3
pass 1640944 2017-09-16 05:21:09 2017-09-19 19:49:02 2017-09-19 20:33:02 0:44:00 0:09:08 0:34:52 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/mds-flush.yaml whitelist_health.yaml} 4
pass 1640945 2017-09-16 05:21:10 2017-09-19 19:50:02 2017-09-19 20:26:02 0:36:00 0:34:44 0:01:16 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_snaps.yaml} 3
fail 1640946 2017-09-16 05:21:10 2017-09-19 19:50:50 2017-09-19 20:30:50 0:40:00 0:30:15 0:09:45 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/filestore-xfs.yaml thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2017-09-19 20:02:48.910455 osd.0 osd.0 172.21.15.132:6804/10219 124 : cluster [WRN] Monitor daemon marked osd.0 down, but it is still running" in cluster log

fail 1640947 2017-09-16 05:21:11 2017-09-19 19:52:54 2017-09-19 21:12:54 1:20:00 0:06:26 1:13:34 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/mds-full.yaml whitelist_health.yaml} 4
Failure Reason:

Command failed on smithi092 with status 1: 'sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmpHWECqU'

pass 1640948 2017-09-16 05:21:12 2017-09-19 19:52:54 2017-09-19 20:36:54 0:44:00 0:36:21 0:07:39 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_dbench.yaml} 3
pass 1640949 2017-09-16 05:21:12 2017-09-19 19:52:54 2017-09-19 20:20:53 0:27:59 0:25:02 0:02:57 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_ffsb.yaml} 3
pass 1640950 2017-09-16 05:21:13 2017-09-19 19:52:54 2017-09-19 20:36:54 0:44:00 0:12:02 0:31:58 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/pool-perm.yaml whitelist_health.yaml} 4
pass 1640951 2017-09-16 05:21:13 2017-09-19 19:52:54 2017-09-19 20:10:53 0:17:59 0:09:12 0:08:47 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsstress.yaml} 3
pass 1640952 2017-09-16 05:21:14 2017-09-19 19:52:54 2017-09-19 20:18:53 0:25:59 0:16:01 0:09:58 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_fsx.yaml} 3
pass 1640953 2017-09-16 05:21:15 2017-09-19 19:52:54 2017-09-19 20:50:54 0:58:00 0:13:35 0:44:25 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/sessionmap.yaml whitelist_health.yaml} 4
pass 1640954 2017-09-16 05:21:15 2017-09-19 19:52:54 2017-09-19 20:08:53 0:15:59 0:12:30 0:03:29 smithi master kcephfs/thrash/{clusters/fixed-3-cephfs.yaml conf.yaml objectstore/bluestore.yaml thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} 3
pass 1640955 2017-09-16 05:21:16 2017-09-19 19:52:58 2017-09-19 20:04:57 0:11:59 0:08:52 0:03:07 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_fsync.yaml} 3
fail 1640956 2017-09-16 05:21:16 2017-09-19 19:53:08 2017-09-19 20:41:09 0:48:01 0:38:19 0:09:42 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/filestore-xfs.yaml tasks/strays.yaml whitelist_health.yaml} 4
Failure Reason:

Test failure: test_replicated_delete_speed (tasks.cephfs.test_strays.TestStrays)

fail 1640957 2017-09-16 05:21:17 2017-09-20 06:42:03 2017-09-20 07:22:02 0:39:59 0:28:17 0:11:42 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_suites_iozone.yaml} 3
Failure Reason:

"2017-09-20 07:07:26.256841 mon.a mon.0 172.21.15.24:6789/0 171 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 1640958 2017-09-16 05:21:18 2017-09-20 06:42:03 2017-09-20 06:56:02 0:13:59 0:09:53 0:04:06 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/yes.yaml objectstore/bluestore.yaml tasks/kclient_workunit_suites_pjd.yaml} 3
pass 1640959 2017-09-16 05:21:18 2017-09-20 06:42:03 2017-09-20 07:10:03 0:28:00 0:22:36 0:05:24 smithi master kcephfs/recovery/{clusters/4-remote-clients.yaml debug/mds_client.yaml dirfrag/frag_enable.yaml mounts/kmounts.yaml objectstore/bluestore.yaml tasks/volume-client.yaml whitelist_health.yaml} 4
pass 1640960 2017-09-16 05:21:19 2017-09-20 06:42:13 2017-09-20 07:02:12 0:19:59 0:11:09 0:08:50 smithi master kcephfs/cephfs/{clusters/fixed-3-cephfs.yaml conf.yaml inline/no.yaml objectstore/filestore-xfs.yaml tasks/kclient_workunit_trivial_sync.yaml} 3