User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
---|---|---|---|---|---|---|---|---|---|---|---|
teuthology | 2018-11-10 05:20:02 | 2018-11-17 14:17:59 | 2018-11-18 08:22:18 | 18:04:19 | kcephfs | luminous | ovh | f6a1ee4 | 189 | 46 | 15 |
Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pass | 3245390 | 2018-11-10 05:20:26 | 2018-11-15 18:56:28 | 2018-11-15 19:36:28 | 0:40:00 | 0:26:02 | 0:13:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245391 | 2018-11-10 05:20:27 | 2018-11-15 19:02:39 | 2018-11-15 20:50:41 | 1:48:02 | 1:38:05 | 0:09:57 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-11-15 19:42:03.221781 mon.b mon.0 158.69.65.26:6789/0 147 : cluster [WRN] Health check failed: 8 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245392 | 2018-11-10 05:20:28 | 2018-11-15 19:06:08 | 2018-11-15 20:02:08 | 0:56:00 | 0:24:01 | 0:31:59 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 3245393 | 2018-11-10 05:20:29 | 2018-11-15 19:08:07 | 2018-11-15 20:36:08 | 1:28:01 | 1:11:41 | 0:16:20 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3245394 | 2018-11-10 05:20:29 | 2018-11-15 19:20:24 | 2018-11-15 20:50:25 | 1:30:01 | 1:07:31 | 0:22:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 3245395 | 2018-11-10 05:20:30 | 2018-11-15 19:26:23 | 2018-11-15 20:10:22 | 0:43:59 | 0:22:46 | 0:21:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3245396 | 2018-11-10 05:20:31 | 2018-11-15 19:30:30 | 2018-11-15 21:26:31 | 1:56:01 | 1:29:03 | 0:26:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245397 | 2018-11-10 05:20:32 | 2018-11-15 19:34:12 | 2018-11-15 20:26:12 | 0:52:00 | 0:29:42 | 0:22:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245398 | 2018-11-10 05:20:33 | 2018-11-15 19:34:16 | 2018-11-15 20:54:17 | 1:20:01 | 0:30:24 | 0:49:37 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 3245399 | 2018-11-10 05:20:33 | 2018-11-15 19:36:31 | 2018-11-15 21:26:32 | 1:50:01 | 1:36:28 | 0:13:33 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3245400 | 2018-11-10 05:20:34 | 2018-11-15 20:02:23 | 2018-11-15 21:48:24 | 1:46:01 | 1:27:12 | 0:18:49 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3245401 | 2018-11-10 05:20:35 | 2018-11-15 20:10:31 | 2018-11-16 08:17:48 | 12:07:17 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245402 | 2018-11-10 05:20:36 | 2018-11-15 20:26:23 | 2018-11-15 21:34:23 | 1:08:00 | 0:51:29 | 0:16:31 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245403 | 2018-11-10 05:20:36 | 2018-11-15 20:36:22 | 2018-11-15 21:48:22 | 1:12:00 | 0:57:39 | 0:14:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-15 21:09:08.680122 mon.b mon.0 158.69.65.174:6789/0 137 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245404 | 2018-11-10 05:20:37 | 2018-11-15 20:40:26 | 2018-11-15 22:06:27 | 1:26:01 | 0:22:37 | 1:03:24 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
fail | 3245405 | 2018-11-10 05:20:38 | 2018-11-15 20:50:36 | 2018-11-15 21:50:36 | 1:00:00 | 0:40:31 | 0:19:29 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
Failure Reason:
"2018-11-15 21:16:32.328461 mon.b mon.0 158.69.69.198:6789/0 231 : cluster [WRN] Health check failed: 47 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245406 | 2018-11-10 05:20:39 | 2018-11-15 20:50:42 | 2018-11-15 22:00:42 | 1:10:00 | 0:52:26 | 0:17:34 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245407 | 2018-11-10 05:20:39 | 2018-11-15 20:52:09 | 2018-11-15 21:56:10 | 1:04:01 | 0:34:38 | 0:29:23 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 3245408 | 2018-11-10 05:20:40 | 2018-11-15 20:54:20 | 2018-11-15 22:30:21 | 1:36:01 | 1:11:01 | 0:25:00 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3245409 | 2018-11-10 05:20:41 | 2018-11-15 21:26:46 | 2018-11-15 22:04:45 | 0:37:59 | 0:21:02 | 0:16:57 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245410 | 2018-11-10 05:20:42 | 2018-11-15 21:26:46 | 2018-11-15 22:38:46 | 1:12:00 | 0:38:34 | 0:33:26 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 3245411 | 2018-11-10 05:20:42 | 2018-11-15 21:34:26 | 2018-11-15 23:18:28 | 1:44:02 | 1:28:18 | 0:15:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-11-15 22:14:25.896197 mon.b mon.0 158.69.72.215:6789/0 236 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245412 | 2018-11-10 05:20:43 | 2018-11-15 21:48:35 | 2018-11-15 22:24:35 | 0:36:00 | 0:21:56 | 0:14:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245413 | 2018-11-10 05:20:44 | 2018-11-15 21:48:36 | 2018-11-15 22:50:36 | 1:02:00 | 0:27:18 | 0:34:42 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3245414 | 2018-11-10 05:20:45 | 2018-11-15 21:50:39 | 2018-11-15 22:32:39 | 0:42:00 | 0:18:16 | 0:23:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245415 | 2018-11-10 05:20:45 | 2018-11-15 21:56:23 | 2018-11-15 22:34:23 | 0:38:00 | 0:21:11 | 0:16:49 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3245416 | 2018-11-10 05:20:46 | 2018-11-15 22:00:45 | 2018-11-15 23:52:47 | 1:52:02 | 1:12:01 | 0:40:01 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 3245417 | 2018-11-10 05:20:47 | 2018-11-15 22:04:48 | 2018-11-15 23:22:49 | 1:18:01 | 0:25:35 | 0:52:26 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
dead | 3245418 | 2018-11-10 05:20:48 | 2018-11-15 22:06:32 | 2018-11-16 10:18:28 | 12:11:56 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||||
pass | 3245419 | 2018-11-10 05:20:48 | 2018-11-15 22:24:49 | 2018-11-15 23:40:50 | 1:16:01 | 1:03:52 | 0:12:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3245420 | 2018-11-10 05:20:49 | 2018-11-15 22:30:34 | 2018-11-16 10:47:23 | 12:16:49 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245421 | 2018-11-10 05:20:50 | 2018-11-15 22:32:42 | 2018-11-16 00:16:43 | 1:44:01 | 1:31:04 | 0:12:57 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245422 | 2018-11-10 05:20:51 | 2018-11-15 22:34:36 | 2018-11-15 23:26:36 | 0:52:00 | 0:29:24 | 0:22:36 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245423 | 2018-11-10 05:20:51 | 2018-11-15 22:38:49 | 2018-11-15 23:46:50 | 1:08:01 | 0:23:33 | 0:44:28 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
dead | 3245424 | 2018-11-10 05:20:52 | 2018-11-15 22:50:52 | 2018-11-16 10:58:07 | 12:07:15 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
pass | 3245425 | 2018-11-10 05:20:53 | 2018-11-15 22:56:09 | 2018-11-16 00:06:10 | 1:10:01 | 0:46:43 | 0:23:18 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 3245426 | 2018-11-10 05:20:54 | 2018-11-15 23:18:42 | 2018-11-16 00:02:42 | 0:44:00 | 0:24:54 | 0:19:06 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) |
||||||||||||||
pass | 3245427 | 2018-11-10 05:20:54 | 2018-11-15 23:23:02 | 2018-11-16 00:31:03 | 1:08:01 | 0:50:25 | 0:17:36 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245428 | 2018-11-10 05:20:55 | 2018-11-15 23:26:48 | 2018-11-16 00:42:49 | 1:16:01 | 0:47:38 | 0:28:23 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-16 00:10:56.431443 mon.a mon.0 158.69.71.162:6789/0 155 : cluster [WRN] Health check failed: 17 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245429 | 2018-11-10 05:20:56 | 2018-11-15 23:41:02 | 2018-11-16 00:27:02 | 0:46:00 | 0:24:37 | 0:21:23 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3245430 | 2018-11-10 05:20:56 | 2018-11-15 23:47:02 | 2018-11-16 00:55:03 | 1:08:01 | 0:40:21 | 0:27:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245431 | 2018-11-10 05:20:57 | 2018-11-15 23:52:59 | 2018-11-16 01:23:00 | 1:30:01 | 0:52:21 | 0:37:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245432 | 2018-11-10 05:20:58 | 2018-11-16 00:02:55 | 2018-11-16 01:02:56 | 1:00:01 | 0:24:57 | 0:35:04 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 3245433 | 2018-11-10 05:20:59 | 2018-11-16 00:06:22 | 2018-11-16 01:58:23 | 1:52:01 | 1:27:06 | 0:24:55 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245434 | 2018-11-10 05:20:59 | 2018-11-16 00:16:56 | 2018-11-16 00:52:56 | 0:36:00 | 0:20:38 | 0:15:22 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245435 | 2018-11-10 05:21:00 | 2018-11-16 00:27:18 | 2018-11-16 01:45:18 | 1:18:00 | 0:57:34 | 0:20:26 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 3245436 | 2018-11-10 05:21:01 | 2018-11-16 00:31:15 | 2018-11-16 02:09:16 | 1:38:01 | 1:25:00 | 0:13:01 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-11-16 01:09:23.521861 mon.a mon.0 158.69.74.29:6789/0 258 : cluster [WRN] Health check failed: 2 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245437 | 2018-11-10 05:21:02 | 2018-11-16 00:43:01 | 2018-11-16 01:39:01 | 0:56:00 | 0:24:13 | 0:31:47 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245438 | 2018-11-10 05:21:02 | 2018-11-16 00:53:09 | 2018-11-16 02:17:05 | 1:23:56 | 0:38:44 | 0:45:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3245439 | 2018-11-10 05:21:03 | 2018-11-16 00:55:31 | 2018-11-16 01:41:25 | 0:45:54 | 0:21:25 | 0:24:29 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245440 | 2018-11-10 05:21:04 | 2018-11-16 01:02:59 | 2018-11-16 01:44:59 | 0:42:00 | 0:23:49 | 0:18:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245441 | 2018-11-10 05:21:05 | 2018-11-16 01:23:03 | 2018-11-16 03:33:05 | 2:10:02 | 1:45:56 | 0:24:06 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-11-16 02:20:09.910842 mon.a mon.0 158.69.70.14:6789/0 287 : cluster [WRN] Health check failed: Degraded data redundancy: 5687/22844 objects degraded (24.895%), 9 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
pass | 3245442 | 2018-11-10 05:21:06 | 2018-11-16 01:28:48 | 2018-11-16 02:26:48 | 0:58:00 | 0:23:45 | 0:34:15 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 3245443 | 2018-11-10 05:21:06 | 2018-11-16 01:39:05 | 2018-11-16 02:47:05 | 1:08:00 | 0:59:18 | 0:08:42 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 3245444 | 2018-11-10 05:21:07 | 2018-11-16 01:41:28 | 2018-11-16 03:23:29 | 1:42:01 | 1:20:56 | 0:21:05 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-11-16 02:15:57.438581 mon.b mon.0 158.69.69.58:6789/0 243 : cluster [WRN] Health check failed: 534 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245445 | 2018-11-10 05:21:08 | 2018-11-16 01:45:02 | 2018-11-16 02:53:02 | 1:08:00 | 0:23:40 | 0:44:20 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3245446 | 2018-11-10 05:21:09 | 2018-11-16 01:45:19 | 2018-11-16 03:15:20 | 1:30:01 | 1:17:58 | 0:12:03 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245447 | 2018-11-10 05:21:10 | 2018-11-16 01:58:37 | 2018-11-16 02:50:37 | 0:52:00 | 0:29:25 | 0:22:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245448 | 2018-11-10 05:21:10 | 2018-11-16 02:09:19 | 2018-11-16 03:19:19 | 1:10:00 | 0:29:15 | 0:40:45 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
dead | 3245449 | 2018-11-10 05:21:11 | 2018-11-16 02:17:08 | 2018-11-16 14:24:19 | 12:07:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
fail | 3245450 | 2018-11-10 05:21:12 | 2018-11-16 04:27:40 | 2018-11-16 06:15:42 | 1:48:02 | 1:33:01 | 0:15:01 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Found coredumps on ubuntu@ovh089.front.sepia.ceph.com |
||||||||||||||
dead | 3245451 | 2018-11-10 05:21:13 | 2018-11-16 04:37:16 | 2018-11-16 16:49:10 | 12:11:54 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245452 | 2018-11-10 05:21:14 | 2018-11-16 04:43:28 | 2018-11-16 05:41:28 | 0:58:00 | 0:49:46 | 0:08:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245453 | 2018-11-10 05:21:14 | 2018-11-16 15:06:56 | 2018-11-16 16:32:57 | 1:26:01 | 1:17:37 | 0:08:24 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-16 15:33:26.880392 mon.a mon.0 158.69.70.144:6789/0 150 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245454 | 2018-11-10 05:21:15 | 2018-11-16 15:09:08 | 2018-11-16 16:03:08 | 0:54:00 | 0:22:35 | 0:31:25 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 3245455 | 2018-11-10 05:21:16 | 2018-11-16 15:17:24 | 2018-11-16 16:03:24 | 0:46:00 | 0:36:32 | 0:09:28 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245456 | 2018-11-10 05:21:17 | 2018-11-16 15:28:43 | 2018-11-16 16:30:44 | 1:02:01 | 0:42:24 | 0:19:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245457 | 2018-11-10 05:21:17 | 2018-11-16 15:28:46 | 2018-11-16 16:24:46 | 0:56:00 | 0:35:05 | 0:20:55 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 3245458 | 2018-11-10 05:21:18 | 2018-11-16 15:31:27 | 2018-11-16 16:41:28 | 1:10:01 | 0:49:48 | 0:20:13 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3245459 | 2018-11-10 05:21:19 | 2018-11-16 15:33:16 | 2018-11-16 16:03:16 | 0:30:00 | 0:20:35 | 0:09:25 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245460 | 2018-11-10 05:21:20 | 2018-11-16 15:41:21 | 2018-11-16 17:05:22 | 1:24:01 | 0:38:09 | 0:45:52 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 3245461 | 2018-11-10 05:21:20 | 2018-11-16 15:45:14 | 2018-11-16 17:29:15 | 1:44:01 | 1:25:41 | 0:18:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245462 | 2018-11-10 05:21:21 | 2018-11-16 16:03:22 | 2018-11-16 16:35:22 | 0:32:00 | 0:22:56 | 0:09:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245463 | 2018-11-10 05:21:22 | 2018-11-16 16:03:22 | 2018-11-16 16:57:22 | 0:54:00 | 0:27:36 | 0:26:24 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3245464 | 2018-11-10 05:21:23 | 2018-11-16 16:03:26 | 2018-11-16 16:31:25 | 0:27:59 | 0:18:57 | 0:09:02 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245465 | 2018-11-10 05:21:23 | 2018-11-16 16:05:12 | 2018-11-16 16:35:12 | 0:30:00 | 0:21:15 | 0:08:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3245466 | 2018-11-10 05:21:24 | 2018-11-16 16:11:16 | 2018-11-16 17:57:18 | 1:46:02 | 1:07:02 | 0:39:00 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 3245467 | 2018-11-10 05:21:25 | 2018-11-16 16:25:03 | 2018-11-16 17:19:03 | 0:54:00 | 0:26:13 | 0:27:47 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 3245468 | 2018-11-10 05:21:26 | 2018-11-16 16:30:57 | 2018-11-16 18:38:58 | 2:08:01 | 1:52:24 | 0:15:37 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 3245469 | 2018-11-10 05:21:26 | 2018-11-16 16:31:11 | 2018-11-16 18:11:12 | 1:40:01 | 1:26:26 | 0:13:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-11-16 17:41:38.757053 mon.b mon.0 158.69.70.196:6789/0 145 : cluster [ERR] Health check failed: mon c is very low on available space (MON_DISK_CRIT)" in cluster log |
||||||||||||||
dead | 3245470 | 2018-11-10 05:21:27 | 2018-11-16 16:31:26 | 2018-11-17 04:43:22 | 12:11:56 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245471 | 2018-11-10 05:21:28 | 2018-11-16 16:33:11 | 2018-11-16 17:59:12 | 1:26:01 | 1:17:03 | 0:08:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245472 | 2018-11-10 05:21:29 | 2018-11-16 16:35:25 | 2018-11-16 17:17:25 | 0:42:00 | 0:30:30 | 0:11:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245473 | 2018-11-10 05:21:30 | 2018-11-16 16:35:25 | 2018-11-16 17:23:25 | 0:48:00 | 0:24:47 | 0:23:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 3245474 | 2018-11-10 05:21:30 | 2018-11-16 16:41:41 | 2018-11-16 18:25:43 | 1:44:02 | 1:27:14 | 0:16:48 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 3245475 | 2018-11-10 05:21:31 | 2018-11-16 16:49:24 | 2018-11-16 18:05:25 | 1:16:01 | 1:06:58 | 0:09:03 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh080 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
fail | 3245476 | 2018-11-10 05:21:32 | 2018-11-16 16:57:36 | 2018-11-16 17:55:36 | 0:58:00 | 0:26:15 | 0:31:45 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) |
||||||||||||||
pass | 3245477 | 2018-11-10 05:21:33 | 2018-11-16 17:05:35 | 2018-11-16 18:13:36 | 1:08:01 | 0:49:33 | 0:18:28 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245478 | 2018-11-10 05:21:34 | 2018-11-16 17:09:07 | 2018-11-16 21:21:12 | 4:12:05 | 3:45:54 | 0:26:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh027 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
pass | 3245479 | 2018-11-10 05:21:34 | 2018-11-16 17:17:37 | 2018-11-16 18:05:37 | 0:48:00 | 0:25:33 | 0:22:27 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3245480 | 2018-11-10 05:21:35 | 2018-11-16 17:19:16 | 2018-11-16 18:11:16 | 0:52:00 | 0:37:44 | 0:14:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
fail | 3245481 | 2018-11-10 05:21:36 | 2018-11-16 17:23:37 | 2018-11-16 21:17:41 | 3:54:04 | 3:44:50 | 0:09:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/fsx.sh) on ovh088 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
pass | 3245482 | 2018-11-10 05:21:37 | 2018-11-16 17:29:28 | 2018-11-16 18:23:28 | 0:54:00 | 0:24:29 | 0:29:31 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 3245483 | 2018-11-10 05:21:37 | 2018-11-16 17:55:49 | 2018-11-16 19:43:50 | 1:48:01 | 1:25:56 | 0:22:05 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245484 | 2018-11-10 05:21:38 | 2018-11-16 17:57:30 | 2018-11-16 18:45:30 | 0:48:00 | 0:20:23 | 0:27:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245485 | 2018-11-10 05:21:39 | 2018-11-16 17:59:24 | 2018-11-16 19:29:25 | 1:30:01 | 0:58:50 | 0:31:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
pass | 3245486 | 2018-11-10 05:21:39 | 2018-11-16 18:05:37 | 2018-11-16 19:43:38 | 1:38:01 | 1:26:43 | 0:11:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245487 | 2018-11-10 05:21:40 | 2018-11-16 18:05:38 | 2018-11-16 18:37:38 | 0:32:00 | 0:22:55 | 0:09:05 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245488 | 2018-11-10 05:21:41 | 2018-11-16 18:11:24 | 2018-11-16 19:27:25 | 1:16:01 | 0:36:33 | 0:39:28 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3245489 | 2018-11-10 05:21:42 | 2018-11-16 18:11:24 | 2018-11-16 18:41:24 | 0:30:00 | 0:19:34 | 0:10:26 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245490 | 2018-11-10 05:21:42 | 2018-11-16 18:13:48 | 2018-11-16 18:51:48 | 0:38:00 | 0:25:04 | 0:12:56 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245491 | 2018-11-10 05:21:43 | 2018-11-16 18:23:40 | 2018-11-16 20:11:42 | 1:48:02 | 1:32:52 | 0:15:10 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-11-16 19:01:31.967551 mon.b mon.0 158.69.70.164:6789/0 224 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245492 | 2018-11-10 05:21:44 | 2018-11-16 18:25:55 | 2018-11-16 19:45:56 | 1:20:01 | 0:22:12 | 0:57:49 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
fail | 3245493 | 2018-11-10 05:21:45 | 2018-11-16 18:37:50 | 2018-11-16 20:09:51 | 1:32:01 | 1:16:50 | 0:15:11 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-16 19:27:07.600751 mon.b mon.0 158.69.69.99:6789/0 2853 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 3245494 | 2018-11-10 05:21:45 | 2018-11-16 18:39:11 | 2018-11-16 19:51:12 | 1:12:01 | 1:02:29 | 0:09:32 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 3245495 | 2018-11-10 05:21:46 | 2018-11-16 18:41:36 | 2018-11-16 19:37:37 | 0:56:01 | 0:22:24 | 0:33:37 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3245496 | 2018-11-10 05:21:47 | 2018-11-16 18:45:42 | 2018-11-16 20:09:43 | 1:24:01 | 1:05:03 | 0:18:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245497 | 2018-11-10 05:21:48 | 2018-11-16 18:52:01 | 2018-11-16 19:38:01 | 0:46:00 | 0:30:15 | 0:15:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245498 | 2018-11-10 05:21:48 | 2018-11-16 19:27:37 | 2018-11-16 20:25:37 | 0:58:00 | 0:29:17 | 0:28:43 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 3245499 | 2018-11-10 05:21:49 | 2018-11-16 19:29:37 | 2018-11-16 21:15:38 | 1:46:01 | 1:30:46 | 0:15:15 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 3245500 | 2018-11-10 05:21:50 | 2018-11-16 19:37:49 | 2018-11-16 23:35:53 | 3:58:04 | 3:35:44 | 0:22:20 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/iozone.sh) on ovh069 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' |
||||||||||||||
dead | 3245501 | 2018-11-10 05:21:50 | 2018-11-16 19:38:02 | 2018-11-17 07:45:20 | 12:07:18 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245502 | 2018-11-10 05:21:51 | 2018-11-16 19:43:50 | 2018-11-16 20:45:51 | 1:02:01 | 0:52:22 | 0:09:39 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245503 | 2018-11-10 05:21:52 | 2018-11-16 19:43:51 | 2018-11-16 20:51:51 | 1:08:00 | 0:50:12 | 0:17:48 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-16 20:09:14.920457 mon.b mon.0 158.69.64.114:6789/0 133 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245504 | 2018-11-10 05:21:53 | 2018-11-16 19:46:09 | 2018-11-16 20:48:09 | 1:02:00 | 0:22:28 | 0:39:32 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 3245505 | 2018-11-10 05:21:53 | 2018-11-16 19:51:24 | 2018-11-16 20:33:24 | 0:42:00 | 0:33:54 | 0:08:06 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245506 | 2018-11-10 05:21:54 | 2018-11-16 20:09:55 | 2018-11-16 22:05:57 | 1:56:02 | 1:42:57 | 0:13:05 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245507 | 2018-11-10 05:21:55 | 2018-11-16 20:09:55 | 2018-11-16 21:45:56 | 1:36:01 | 0:34:33 | 1:01:28 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 3245508 | 2018-11-10 05:21:56 | 2018-11-16 20:11:53 | 2018-11-16 21:25:54 | 1:14:01 | 0:49:56 | 0:24:05 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3245509 | 2018-11-10 05:21:56 | 2018-11-16 20:25:57 | 2018-11-16 21:01:56 | 0:35:59 | 0:20:45 | 0:15:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245510 | 2018-11-10 05:21:57 | 2018-11-16 20:33:59 | 2018-11-16 21:53:59 | 1:20:00 | 0:39:25 | 0:40:35 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 3245511 | 2018-11-10 05:21:58 | 2018-11-16 20:45:54 | 2018-11-16 22:25:55 | 1:40:01 | 1:30:42 | 0:09:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-11-16 21:41:32.490566 mon.b mon.0 158.69.64.186:6789/0 238 : cluster [WRN] Health check failed: Degraded data redundancy: 2565/10284 objects degraded (24.942%), 10 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
pass | 3245512 | 2018-11-10 05:21:59 | 2018-11-16 20:48:12 | 2018-11-16 21:24:12 | 0:36:00 | 0:21:46 | 0:14:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245513 | 2018-11-10 05:21:59 | 2018-11-16 20:51:55 | 2018-11-16 22:13:55 | 1:22:00 | 0:26:54 | 0:55:06 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3245514 | 2018-11-10 05:22:00 | 2018-11-16 21:01:59 | 2018-11-16 21:35:59 | 0:34:00 | 0:19:16 | 0:14:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245515 | 2018-11-10 05:22:01 | 2018-11-16 21:15:43 | 2018-11-16 22:05:43 | 0:50:00 | 0:25:52 | 0:24:08 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3245516 | 2018-11-10 05:22:02 | 2018-11-16 21:17:44 | 2018-11-16 23:01:46 | 1:44:02 | 1:27:36 | 0:16:26 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 3245517 | 2018-11-10 05:22:02 | 2018-11-16 21:21:28 | 2018-11-16 22:29:29 | 1:08:01 | 0:24:18 | 0:43:43 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 3245518 | 2018-11-10 05:22:03 | 2018-11-16 21:24:15 | 2018-11-16 23:54:18 | 2:30:03 | 1:51:13 | 0:38:50 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245519 | 2018-11-10 05:22:04 | 2018-11-16 21:25:57 | 2018-11-16 22:35:58 | 1:10:01 | 0:57:57 | 0:12:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3245520 | 2018-11-10 05:22:05 | 2018-11-16 21:36:03 | 2018-11-17 09:43:32 | 12:07:29 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245521 | 2018-11-10 05:22:05 | 2018-11-16 21:46:01 | 2018-11-16 23:34:02 | 1:48:01 | 1:32:59 | 0:15:02 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245522 | 2018-11-10 05:22:06 | 2018-11-16 21:54:04 | 2018-11-16 22:44:04 | 0:50:00 | 0:30:44 | 0:19:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245523 | 2018-11-10 05:22:07 | 2018-11-16 22:05:48 | 2018-11-16 23:05:49 | 1:00:01 | 0:22:58 | 0:37:03 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 3245524 | 2018-11-10 05:22:08 | 2018-11-16 22:05:58 | 2018-11-16 23:41:59 | 1:36:01 | 1:26:23 | 0:09:38 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 3245525 | 2018-11-10 05:22:08 | 2018-11-16 22:14:01 | 2018-11-16 23:18:01 | 1:04:00 | 0:49:42 | 0:14:18 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh063 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
fail | 3245526 | 2018-11-10 05:22:09 | 2018-11-16 22:26:08 | 2018-11-16 23:22:08 | 0:56:00 | 0:26:39 | 0:29:21 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) |
||||||||||||||
pass | 3245527 | 2018-11-10 05:22:10 | 2018-11-16 22:29:43 | 2018-11-16 23:37:44 | 1:08:01 | 0:51:40 | 0:16:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245528 | 2018-11-10 05:22:11 | 2018-11-16 22:36:03 | 2018-11-16 23:48:03 | 1:12:00 | 0:55:10 | 0:16:50 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-16 23:18:28.394148 mon.a mon.0 158.69.68.9:6789/0 239 : cluster [WRN] Health check failed: 278 slow requests are blocked > 32 sec. Implicated osds 0,1,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245529 | 2018-11-10 05:22:12 | 2018-11-16 22:44:09 | 2018-11-16 23:46:09 | 1:02:00 | 0:22:55 | 0:39:05 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3245530 | 2018-11-10 05:22:12 | 2018-11-16 23:02:04 | 2018-11-17 00:04:04 | 1:02:00 | 0:42:58 | 0:19:02 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245531 | 2018-11-10 05:22:13 | 2018-11-16 23:06:03 | 2018-11-17 00:18:03 | 1:12:00 | 0:51:23 | 0:20:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245532 | 2018-11-10 05:22:14 | 2018-11-16 23:18:17 | 2018-11-17 00:04:17 | 0:46:00 | 0:26:16 | 0:19:44 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
fail | 3245533 | 2018-11-10 05:22:15 | 2018-11-16 23:22:22 | 2018-11-17 01:28:23 | 2:06:01 | 1:43:10 | 0:22:51 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Scrubbing terminated -- not all pgs were active and clean. |
||||||||||||||
pass | 3245534 | 2018-11-10 05:22:15 | 2018-11-16 23:34:18 | 2018-11-17 00:10:18 | 0:36:00 | 0:20:30 | 0:15:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245535 | 2018-11-10 05:22:16 | 2018-11-16 23:36:07 | 2018-11-17 01:10:08 | 1:34:01 | 0:55:42 | 0:38:19 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 3245536 | 2018-11-10 05:22:17 | 2018-11-16 23:37:58 | 2018-11-17 01:21:59 | 1:44:01 | 1:28:29 | 0:15:32 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-11-17 00:11:33.609392 mon.b mon.0 158.69.68.211:6789/0 148 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245537 | 2018-11-10 05:22:18 | 2018-11-16 23:42:13 | 2018-11-17 00:10:12 | 0:27:59 | 0:20:27 | 0:07:32 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245538 | 2018-11-10 05:22:18 | 2018-11-16 23:46:22 | 2018-11-17 00:58:23 | 1:12:01 | 0:36:18 | 0:35:43 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3245539 | 2018-11-10 05:22:19 | 2018-11-16 23:48:16 | 2018-11-17 00:58:16 | 1:10:00 | 0:18:12 | 0:51:48 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245540 | 2018-11-10 05:22:20 | 2018-11-16 23:54:30 | 2018-11-17 00:34:30 | 0:40:00 | 0:20:15 | 0:19:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245541 | 2018-11-10 05:22:21 | 2018-11-17 00:04:17 | 2018-11-17 01:56:18 | 1:52:01 | 1:36:49 | 0:15:12 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-11-17 00:43:36.201622 mon.a mon.0 158.69.71.108:6789/0 152 : cluster [WRN] Health check failed: 2 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245542 | 2018-11-10 05:22:21 | 2018-11-17 00:04:18 | 2018-11-17 00:48:18 | 0:44:00 | 0:23:15 | 0:20:45 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 3245543 | 2018-11-10 05:22:22 | 2018-11-17 00:10:24 | 2018-11-17 01:24:25 | 1:14:01 | 0:54:20 | 0:19:41 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3245544 | 2018-11-10 05:22:23 | 2018-11-17 00:10:24 | 2018-11-17 01:20:25 | 1:10:01 | 1:00:25 | 0:09:36 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 3245545 | 2018-11-10 05:22:24 | 2018-11-17 00:18:15 | 2018-11-17 01:24:16 | 1:06:01 | 0:21:20 | 0:44:41 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3245546 | 2018-11-10 05:22:24 | 2018-11-17 00:34:43 | 2018-11-17 02:40:44 | 2:06:01 | 1:36:41 | 0:29:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245547 | 2018-11-10 05:22:25 | 2018-11-17 00:48:31 | 2018-11-17 01:32:31 | 0:44:00 | 0:30:33 | 0:13:27 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245548 | 2018-11-10 05:22:26 | 2018-11-17 00:58:28 | 2018-11-17 01:52:29 | 0:54:01 | 0:28:01 | 0:26:00 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
dead | 3245549 | 2018-11-10 05:22:27 | 2018-11-17 00:58:28 | 2018-11-17 13:05:49 | 12:07:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
pass | 3245550 | 2018-11-10 05:22:27 | 2018-11-17 01:10:20 | 2018-11-17 03:00:21 | 1:50:01 | 1:26:58 | 0:23:03 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3245551 | 2018-11-10 05:22:28 | 2018-11-17 01:20:37 | 2018-11-17 13:32:36 | 12:11:59 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245552 | 2018-11-10 05:22:29 | 2018-11-17 01:22:11 | 2018-11-17 02:28:12 | 1:06:01 | 0:51:25 | 0:14:36 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245553 | 2018-11-10 05:22:30 | 2018-11-17 01:24:29 | 2018-11-17 02:28:29 | 1:04:00 | 0:50:07 | 0:13:53 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 02:09:18.322065 mon.a mon.0 158.69.66.8:6789/0 146 : cluster [WRN] Health check failed: 27 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245554 | 2018-11-10 05:22:30 | 2018-11-17 01:24:29 | 2018-11-17 02:48:30 | 1:24:01 | 0:23:39 | 1:00:22 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 3245555 | 2018-11-10 05:22:31 | 2018-11-17 01:28:36 | 2018-11-17 02:36:37 | 1:08:01 | 0:42:51 | 0:25:10 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245556 | 2018-11-10 05:22:32 | 2018-11-17 01:32:34 | 2018-11-17 02:40:34 | 1:08:00 | 0:55:43 | 0:12:17 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245557 | 2018-11-10 05:22:32 | 2018-11-17 01:52:30 | 2018-11-17 02:50:30 | 0:58:00 | 0:35:12 | 0:22:48 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
fail | 3245558 | 2018-11-10 05:22:33 | 2018-11-17 01:56:19 | 2018-11-17 05:54:23 | 3:58:04 | 3:42:01 | 0:16:03 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh094 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
pass | 3245559 | 2018-11-10 05:22:34 | 2018-11-17 04:43:47 | 2018-11-17 05:21:47 | 0:38:00 | 0:19:55 | 0:18:05 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245560 | 2018-11-10 05:22:35 | 2018-11-17 04:46:35 | 2018-11-17 06:20:36 | 1:34:01 | 0:36:57 | 0:57:04 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 3245561 | 2018-11-10 05:22:36 | 2018-11-17 04:52:48 | 2018-11-17 07:06:50 | 2:14:02 | 1:26:15 | 0:47:47 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245562 | 2018-11-10 05:22:36 | 2018-11-17 14:17:59 | 2018-11-17 14:49:59 | 0:32:00 | 0:21:47 | 0:10:13 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245563 | 2018-11-10 05:22:37 | 2018-11-17 14:17:59 | 2018-11-17 15:11:59 | 0:54:00 | 0:27:58 | 0:26:02 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3245564 | 2018-11-10 05:22:38 | 2018-11-17 14:17:59 | 2018-11-17 14:47:59 | 0:30:00 | 0:20:25 | 0:09:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245565 | 2018-11-10 05:22:39 | 2018-11-17 14:17:59 | 2018-11-17 14:53:59 | 0:36:00 | 0:21:49 | 0:14:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245566 | 2018-11-10 05:22:39 | 2018-11-17 14:24:01 | 2018-11-17 16:04:02 | 1:40:01 | 1:25:13 | 0:14:48 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
"2018-11-17 14:59:30.725223 mon.b mon.0 158.69.71.95:6789/0 151 : cluster [WRN] Health check failed: 6 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245567 | 2018-11-10 05:22:40 | 2018-11-17 14:37:57 | 2018-11-17 15:37:57 | 1:00:00 | 0:24:49 | 0:35:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 3245568 | 2018-11-10 05:22:41 | 2018-11-17 14:46:05 | 2018-11-17 17:00:07 | 2:14:02 | 1:45:50 | 0:28:12 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3245569 | 2018-11-10 05:22:41 | 2018-11-17 14:48:02 | 2018-11-18 02:55:27 | 12:07:25 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||||
fail | 3245570 | 2018-11-10 05:22:42 | 2018-11-17 14:50:06 | 2018-11-17 15:54:06 | 1:04:00 | 0:36:18 | 0:27:42 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_reset (tasks.cephfs.test_journal_repair.TestJournalRepair), test_reset (tasks.cephfs.test_journal_repair.TestJournalRepair) |
||||||||||||||
fail | 3245571 | 2018-11-10 05:22:43 | 2018-11-17 14:50:06 | 2018-11-17 16:26:07 | 1:36:01 | 1:11:54 | 0:24:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
Failure Reason:
"2018-11-17 15:47:35.378079 mon.a mon.0 158.69.68.8:6789/0 254 : cluster [WRN] Health check failed: 11 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245572 | 2018-11-10 05:22:44 | 2018-11-17 14:51:53 | 2018-11-17 15:41:53 | 0:50:00 | 0:29:47 | 0:20:13 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245573 | 2018-11-10 05:22:44 | 2018-11-17 14:52:18 | 2018-11-17 16:24:19 | 1:32:01 | 0:22:05 | 1:09:56 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
dead | 3245574 | 2018-11-10 05:22:45 | 2018-11-17 14:54:06 | 2018-11-18 02:56:49 | 12:02:43 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
fail | 3245575 | 2018-11-10 05:22:46 | 2018-11-17 14:54:06 | 2018-11-17 16:32:07 | 1:38:01 | 1:23:35 | 0:14:26 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 15:42:05.183092 mon.a mon.0 158.69.69.217:6789/0 577 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 3245576 | 2018-11-10 05:22:47 | 2018-11-17 15:12:08 | 2018-11-17 16:16:09 | 1:04:01 | 0:39:13 | 0:24:48 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 3245577 | 2018-11-10 05:22:47 | 2018-11-17 15:27:55 | 2018-11-17 16:37:56 | 1:10:01 | 0:51:00 | 0:19:01 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245578 | 2018-11-10 05:22:48 | 2018-11-17 15:31:10 | 2018-11-17 17:07:11 | 1:36:01 | 1:18:23 | 0:17:38 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 16:15:49.232760 mon.b mon.0 158.69.71.215:6789/0 158 : cluster [WRN] Health check failed: 123 slow requests are blocked > 32 sec. Implicated osds 2,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245579 | 2018-11-10 05:22:49 | 2018-11-17 15:38:02 | 2018-11-17 16:30:02 | 0:52:00 | 0:22:20 | 0:29:40 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3245580 | 2018-11-10 05:22:50 | 2018-11-17 15:42:07 | 2018-11-17 16:32:07 | 0:50:00 | 0:36:20 | 0:13:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245581 | 2018-11-10 05:22:50 | 2018-11-17 15:48:06 | 2018-11-17 16:46:06 | 0:58:00 | 0:43:18 | 0:14:42 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245582 | 2018-11-10 05:22:51 | 2018-11-17 15:54:10 | 2018-11-17 17:00:10 | 1:06:00 | 0:24:53 | 0:41:07 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 3245583 | 2018-11-10 05:22:52 | 2018-11-17 16:04:18 | 2018-11-17 17:48:19 | 1:44:01 | 1:26:28 | 0:17:33 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245584 | 2018-11-10 05:22:52 | 2018-11-17 16:16:26 | 2018-11-17 16:52:25 | 0:35:59 | 0:20:17 | 0:15:42 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
fail | 3245585 | 2018-11-10 05:22:53 | 2018-11-17 16:24:25 | 2018-11-17 17:44:25 | 1:20:00 | 0:58:44 | 0:21:16 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2018-11-17 17:27:24.812237 mon.a mon.0 158.69.65.142:6789/0 1082 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245586 | 2018-11-10 05:22:54 | 2018-11-17 16:26:13 | 2018-11-17 18:02:14 | 1:36:01 | 1:26:48 | 0:09:13 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245587 | 2018-11-10 05:22:54 | 2018-11-17 16:30:05 | 2018-11-17 17:10:05 | 0:40:00 | 0:21:29 | 0:18:31 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
fail | 3245588 | 2018-11-10 05:22:55 | 2018-11-17 16:32:21 | 2018-11-17 17:38:21 | 1:06:00 | 0:36:14 | 0:29:46 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2018-11-17 17:09:31.710508 mon.a mon.0 158.69.69.77:6789/0 7 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log |
||||||||||||||
pass | 3245589 | 2018-11-10 05:22:56 | 2018-11-17 16:32:21 | 2018-11-17 17:10:21 | 0:38:00 | 0:19:40 | 0:18:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245590 | 2018-11-10 05:22:57 | 2018-11-17 16:37:59 | 2018-11-17 17:59:59 | 1:22:00 | 0:21:22 | 1:00:38 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3245591 | 2018-11-10 05:22:57 | 2018-11-17 16:46:14 | 2018-11-17 18:38:16 | 1:52:02 | 1:37:04 | 0:14:58 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-11-17 17:25:41.044177 mon.a mon.0 158.69.66.115:6789/0 240 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245592 | 2018-11-10 05:22:58 | 2018-11-17 16:52:40 | 2018-11-17 17:38:40 | 0:46:00 | 0:22:59 | 0:23:01 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
fail | 3245593 | 2018-11-10 05:22:59 | 2018-11-17 17:00:20 | 2018-11-17 18:12:21 | 1:12:01 | 0:52:30 | 0:19:31 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 17:59:50.296486 mon.b mon.0 158.69.65.190:6789/0 3137 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 3245594 | 2018-11-10 05:23:00 | 2018-11-17 17:00:21 | 2018-11-17 18:42:22 | 1:42:01 | 1:19:34 | 0:22:27 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-11-17 17:33:47.402485 mon.b mon.0 158.69.69.249:6789/0 148 : cluster [WRN] Health check failed: 631 slow requests are blocked > 32 sec. Implicated osds 0,3,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245595 | 2018-11-10 05:23:00 | 2018-11-17 17:07:24 | 2018-11-17 18:19:25 | 1:12:01 | 0:22:08 | 0:49:53 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3245596 | 2018-11-10 05:23:01 | 2018-11-17 17:10:18 | 2018-11-17 18:42:19 | 1:32:01 | 1:09:10 | 0:22:51 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245597 | 2018-11-10 05:23:02 | 2018-11-17 17:10:22 | 2018-11-17 18:06:22 | 0:56:00 | 0:30:22 | 0:25:38 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245598 | 2018-11-10 05:23:03 | 2018-11-17 17:38:26 | 2018-11-17 18:34:26 | 0:56:00 | 0:29:47 | 0:26:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 3245599 | 2018-11-10 05:23:03 | 2018-11-17 17:38:42 | 2018-11-17 19:24:43 | 1:46:01 | 1:36:37 | 0:09:24 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3245600 | 2018-11-10 05:23:04 | 2018-11-17 17:44:36 | 2018-11-17 19:46:37 | 2:02:01 | 1:29:18 | 0:32:43 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3245601 | 2018-11-10 05:23:05 | 2018-11-17 17:48:32 | 2018-11-18 05:51:30 | 12:02:58 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245602 | 2018-11-10 05:23:05 | 2018-11-17 18:00:12 | 2018-11-17 19:10:13 | 1:10:01 | 0:50:55 | 0:19:06 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245603 | 2018-11-10 05:23:06 | 2018-11-17 18:02:26 | 2018-11-17 22:00:30 | 3:58:04 | 3:44:34 | 0:13:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh063 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f6a1ee461c2cf2899316dd84572536f2cb3df14b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
pass | 3245604 | 2018-11-10 05:23:07 | 2018-11-17 18:06:35 | 2018-11-17 19:38:36 | 1:32:01 | 0:22:34 | 1:09:27 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 3245605 | 2018-11-10 05:23:08 | 2018-11-17 18:12:24 | 2018-11-17 19:06:24 | 0:54:00 | 0:37:23 | 0:16:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
fail | 3245606 | 2018-11-10 05:23:08 | 2018-11-17 18:19:39 | 2018-11-17 20:21:40 | 2:02:01 | 1:43:25 | 0:18:36 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
Failure Reason:
"2018-11-17 19:45:25.122522 mon.a mon.0 158.69.69.108:6789/0 255 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
pass | 3245607 | 2018-11-10 05:23:09 | 2018-11-17 18:34:31 | 2018-11-17 19:38:31 | 1:04:00 | 0:33:42 | 0:30:18 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
fail | 3245608 | 2018-11-10 05:23:10 | 2018-11-17 18:38:28 | 2018-11-17 19:46:29 | 1:08:01 | 0:55:23 | 0:12:38 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 19:25:11.374979 mon.b mon.0 158.69.70.55:6789/0 18 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 3245609 | 2018-11-10 05:23:10 | 2018-11-17 18:42:22 | 2018-11-17 19:32:22 | 0:50:00 | 0:20:52 | 0:29:08 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245610 | 2018-11-10 05:23:11 | 2018-11-17 18:42:23 | 2018-11-17 20:00:23 | 1:18:00 | 0:37:51 | 0:40:09 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 3245611 | 2018-11-10 05:23:12 | 2018-11-17 19:06:36 | 2018-11-17 20:52:37 | 1:46:01 | 1:27:54 | 0:18:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245612 | 2018-11-10 05:23:13 | 2018-11-17 19:10:26 | 2018-11-17 19:50:26 | 0:40:00 | 0:22:20 | 0:17:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245613 | 2018-11-10 05:23:13 | 2018-11-17 19:24:46 | 2018-11-17 20:20:46 | 0:56:00 | 0:29:14 | 0:26:46 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3245614 | 2018-11-10 05:23:14 | 2018-11-17 19:32:36 | 2018-11-17 20:10:35 | 0:37:59 | 0:19:01 | 0:18:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3245615 | 2018-11-10 05:23:15 | 2018-11-17 19:38:44 | 2018-11-17 20:24:44 | 0:46:00 | 0:26:34 | 0:19:26 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3245616 | 2018-11-10 05:23:16 | 2018-11-17 19:38:44 | 2018-11-17 21:32:45 | 1:54:01 | 1:14:13 | 0:39:48 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 3245617 | 2018-11-10 05:23:16 | 2018-11-17 19:46:41 | 2018-11-17 21:02:42 | 1:16:01 | 0:26:58 | 0:49:03 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
fail | 3245618 | 2018-11-10 05:23:17 | 2018-11-17 19:46:41 | 2018-11-17 21:44:42 | 1:58:01 | 1:43:39 | 0:14:22 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
timed out waiting for admin_socket to appear after osd.4 restart |
||||||||||||||
pass | 3245619 | 2018-11-10 05:23:18 | 2018-11-17 19:50:40 | 2018-11-17 21:16:40 | 1:26:00 | 1:07:18 | 0:18:42 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3245620 | 2018-11-10 05:23:19 | 2018-11-17 20:00:37 | 2018-11-18 08:22:18 | 12:21:41 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
pass | 3245621 | 2018-11-10 05:23:19 | 2018-11-17 20:10:49 | 2018-11-17 21:40:50 | 1:30:01 | 1:09:52 | 0:20:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3245622 | 2018-11-10 05:23:20 | 2018-11-17 20:20:50 | 2018-11-17 21:04:49 | 0:43:59 | 0:30:55 | 0:13:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3245623 | 2018-11-10 05:23:21 | 2018-11-17 20:21:42 | 2018-11-17 21:19:42 | 0:58:00 | 0:21:39 | 0:36:21 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 3245624 | 2018-11-10 05:23:21 | 2018-11-17 20:24:47 | 2018-11-17 22:20:48 | 1:56:01 | 1:42:09 | 0:13:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3245625 | 2018-11-10 05:23:22 | 2018-11-17 20:52:51 | 2018-11-17 22:00:51 | 1:08:00 | 0:49:56 | 0:18:04 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 3245626 | 2018-11-10 05:23:23 | 2018-11-17 21:02:49 | 2018-11-17 21:58:49 | 0:56:00 | 0:24:21 | 0:31:39 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) |
||||||||||||||
pass | 3245627 | 2018-11-10 05:23:24 | 2018-11-17 21:04:54 | 2018-11-17 22:10:54 | 1:06:00 | 0:52:43 | 0:13:17 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3245628 | 2018-11-10 05:23:25 | 2018-11-17 21:16:54 | 2018-11-17 22:16:54 | 1:00:00 | 0:50:03 | 0:09:57 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-11-17 22:02:32.692707 mon.b mon.0 158.69.68.97:6789/0 146 : cluster [WRN] Health check failed: 25 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245629 | 2018-11-10 05:23:25 | 2018-11-17 21:19:45 | 2018-11-17 22:17:45 | 0:58:00 | 0:22:46 | 0:35:14 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3245630 | 2018-11-10 05:23:26 | 2018-11-17 21:32:49 | 2018-11-17 22:42:49 | 1:10:00 | 0:34:08 | 0:35:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3245631 | 2018-11-10 05:23:27 | 2018-11-17 21:40:54 | 2018-11-17 23:36:55 | 1:56:01 | 1:42:52 | 0:13:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3245632 | 2018-11-10 05:23:28 | 2018-11-17 21:44:55 | 2018-11-17 22:58:56 | 1:14:01 | 0:26:29 | 0:47:32 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 3245633 | 2018-11-10 05:23:29 | 2018-11-17 21:59:06 | 2018-11-17 23:59:07 | 2:00:01 | 1:26:29 | 0:33:32 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3245634 | 2018-11-10 05:23:29 | 2018-11-17 22:00:44 | 2018-11-17 22:40:43 | 0:39:59 | 0:21:30 | 0:18:29 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3245635 | 2018-11-10 05:23:30 | 2018-11-17 22:00:52 | 2018-11-17 23:20:53 | 1:20:01 | 0:58:22 | 0:21:39 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 3245636 | 2018-11-10 05:23:31 | 2018-11-17 22:11:00 | 2018-11-17 23:57:02 | 1:46:02 | 1:31:54 | 0:14:08 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-11-17 22:44:16.306042 mon.b mon.0 158.69.68.168:6789/0 244 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3245637 | 2018-11-10 05:23:32 | 2018-11-17 22:17:07 | 2018-11-17 22:55:07 | 0:38:00 | 0:23:36 | 0:14:24 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3245638 | 2018-11-10 05:23:33 | 2018-11-17 22:17:47 | 2018-11-17 23:25:47 | 1:08:00 | 0:38:52 | 0:29:08 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3245639 | 2018-11-10 05:23:33 | 2018-11-17 22:21:02 | 2018-11-17 22:59:02 | 0:38:00 | 0:18:51 | 0:19:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 |