User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
---|---|---|---|---|---|---|---|---|---|---|---|
teuthology | 2019-12-14 05:20:02 | 2019-12-18 13:10:31 | 2019-12-19 00:39:02 | 11:28:31 | kcephfs | luminous | smithi | af44984 | 229 | 18 | 3 |
Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pass | 4602533 | 2019-12-14 05:20:34 | 2019-12-18 09:13:59 | 2019-12-18 09:49:59 | 0:36:00 | 0:19:03 | 0:16:57 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602534 | 2019-12-14 05:20:35 | 2019-12-18 09:16:41 | 2019-12-18 11:14:42 | 1:58:01 | 1:10:20 | 0:47:41 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4602535 | 2019-12-14 05:20:35 | 2019-12-18 09:18:14 | 2019-12-18 10:18:15 | 1:00:01 | 0:14:53 | 0:45:08 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602536 | 2019-12-14 05:20:36 | 2019-12-18 09:19:18 | 2019-12-18 10:17:18 | 0:58:00 | 0:35:39 | 0:22:21 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602537 | 2019-12-14 05:20:37 | 2019-12-18 09:19:18 | 2019-12-18 11:35:20 | 2:16:02 | 1:21:18 | 0:54:44 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602538 | 2019-12-14 05:20:37 | 2019-12-18 09:19:28 | 2019-12-18 11:37:29 | 2:18:01 | 0:11:01 | 2:07:00 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4602539 | 2019-12-14 05:20:38 | 2019-12-18 09:19:27 | 2019-12-18 10:27:28 | 1:08:01 | 0:44:53 | 0:23:08 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602540 | 2019-12-14 05:20:39 | 2019-12-18 09:19:46 | 2019-12-18 10:01:45 | 0:41:59 | 0:28:59 | 0:13:00 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602541 | 2019-12-14 05:20:39 | 2019-12-18 09:20:52 | 2019-12-18 09:56:51 | 0:35:59 | 0:18:06 | 0:17:53 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4602542 | 2019-12-14 05:20:40 | 2019-12-18 09:20:52 | 2019-12-18 10:26:52 | 1:06:00 | 0:50:24 | 0:15:36 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602543 | 2019-12-14 05:20:41 | 2019-12-18 09:21:15 | 2019-12-18 10:21:15 | 1:00:00 | 0:22:32 | 0:37:28 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602544 | 2019-12-14 05:20:42 | 2019-12-18 09:21:28 | 2019-12-18 14:09:33 | 4:48:05 | 0:47:19 | 4:00:46 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4602545 | 2019-12-14 05:20:42 | 2019-12-18 09:21:29 | 2019-12-18 10:47:29 | 1:26:00 | 1:05:21 | 0:20:39 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602546 | 2019-12-14 05:20:43 | 2019-12-18 09:22:50 | 2019-12-18 11:14:53 | 1:52:03 | 1:15:23 | 0:36:40 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 10:13:54.442126 mon.a mon.0 172.21.15.12:6789/0 154 : cluster [WRN] Health check failed: 1384 slow requests are blocked > 32 sec. Implicated osds 1,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602547 | 2019-12-14 05:20:44 | 2019-12-18 09:24:11 | 2019-12-18 16:10:19 | 6:46:08 | 0:16:10 | 6:29:58 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4602548 | 2019-12-14 05:20:44 | 2019-12-18 09:24:11 | 2019-12-18 10:00:10 | 0:35:59 | 0:24:29 | 0:11:30 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602549 | 2019-12-14 05:20:45 | 2019-12-18 09:24:11 | 2019-12-18 10:30:49 | 1:06:38 | 0:55:18 | 0:11:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602550 | 2019-12-14 05:20:46 | 2019-12-18 09:24:11 | 2019-12-18 12:20:14 | 2:56:03 | 0:20:03 | 2:36:00 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4602551 | 2019-12-14 05:20:46 | 2019-12-18 09:24:20 | 2019-12-18 10:54:20 | 1:30:00 | 0:43:53 | 0:46:07 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602552 | 2019-12-14 05:20:47 | 2019-12-18 09:25:02 | 2019-12-18 09:59:01 | 0:33:59 | 0:20:15 | 0:13:44 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602553 | 2019-12-14 05:20:48 | 2019-12-18 09:25:09 | 2019-12-18 13:55:13 | 4:30:04 | 0:24:00 | 4:06:04 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4602554 | 2019-12-14 05:20:49 | 2019-12-18 09:26:47 | 2019-12-18 10:56:48 | 1:30:01 | 0:19:49 | 1:10:12 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602555 | 2019-12-14 05:20:49 | 2019-12-18 09:28:04 | 2019-12-18 10:04:03 | 0:35:59 | 0:22:26 | 0:13:33 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602556 | 2019-12-14 05:20:50 | 2019-12-18 09:30:05 | 2019-12-18 12:40:07 | 3:10:02 | 0:16:37 | 2:53:25 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4602557 | 2019-12-14 05:20:51 | 2019-12-18 09:30:31 | 2019-12-18 10:54:31 | 1:24:00 | 0:16:48 | 1:07:12 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602558 | 2019-12-14 05:20:51 | 2019-12-18 09:30:49 | 2019-12-18 10:06:48 | 0:35:59 | 0:20:14 | 0:15:45 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602559 | 2019-12-14 05:20:52 | 2019-12-18 09:31:38 | 2019-12-18 11:57:39 | 2:26:01 | 1:43:58 | 0:42:03 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 4602560 | 2019-12-14 05:20:53 | 2019-12-18 09:31:37 | 2019-12-18 14:09:42 | 4:38:05 | 0:14:48 | 4:23:17 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4602561 | 2019-12-14 05:20:54 | 2019-12-18 09:31:49 | 2019-12-18 10:15:48 | 0:43:59 | 0:24:57 | 0:19:02 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602562 | 2019-12-14 05:20:54 | 2019-12-18 09:31:54 | 2019-12-18 10:25:54 | 0:54:00 | 0:35:59 | 0:18:01 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602563 | 2019-12-14 05:20:55 | 2019-12-18 09:31:55 | 2019-12-18 14:14:00 | 4:42:05 | 0:45:10 | 3:56:55 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602564 | 2019-12-14 05:20:56 | 2019-12-18 09:34:49 | 2019-12-18 10:52:50 | 1:18:01 | 0:47:16 | 0:30:45 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602565 | 2019-12-14 05:20:56 | 2019-12-18 09:36:06 | 2019-12-18 10:18:06 | 0:42:00 | 0:27:37 | 0:14:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602566 | 2019-12-14 05:20:57 | 2019-12-18 09:37:27 | 2019-12-18 10:43:28 | 1:06:01 | 0:11:08 | 0:54:53 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 4602567 | 2019-12-14 05:20:58 | 2019-12-18 09:37:29 | 2019-12-18 11:19:30 | 1:42:01 | 1:07:22 | 0:34:39 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602568 | 2019-12-14 05:20:58 | 2019-12-18 09:37:40 | 2019-12-18 11:01:40 | 1:24:00 | 0:31:57 | 0:52:03 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602569 | 2019-12-14 05:20:59 | 2019-12-18 09:38:05 | 2019-12-18 10:40:05 | 1:02:00 | 0:27:30 | 0:34:30 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4602570 | 2019-12-14 05:21:00 | 2019-12-18 09:40:00 | 2019-12-18 11:48:01 | 2:08:01 | 1:42:06 | 0:25:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602571 | 2019-12-14 05:21:01 | 2019-12-18 09:40:33 | 2019-12-18 10:26:32 | 0:45:59 | 0:30:10 | 0:15:49 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 10:10:11.974024 mon.a mon.0 172.21.15.85:6789/0 158 : cluster [WRN] Health check failed: 65 slow requests are blocked > 32 sec. Implicated osds 3,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602572 | 2019-12-14 05:21:01 | 2019-12-18 09:44:42 | 2019-12-18 10:32:42 | 0:48:00 | 0:11:56 | 0:36:04 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4602573 | 2019-12-14 05:21:02 | 2019-12-18 09:45:22 | 2019-12-18 10:37:22 | 0:52:00 | 0:23:01 | 0:28:59 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602574 | 2019-12-14 05:21:03 | 2019-12-18 09:47:35 | 2019-12-18 11:59:36 | 2:12:01 | 0:56:31 | 1:15:30 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602575 | 2019-12-14 05:21:03 | 2019-12-18 09:47:35 | 2019-12-18 13:55:42 | 4:08:07 | 0:14:42 | 3:53:25 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4602576 | 2019-12-14 05:21:04 | 2019-12-18 09:49:00 | 2019-12-18 10:29:00 | 0:40:00 | 0:24:00 | 0:16:00 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602577 | 2019-12-14 05:21:05 | 2019-12-18 09:49:17 | 2019-12-18 10:43:17 | 0:54:00 | 0:18:03 | 0:35:57 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
fail | 4602578 | 2019-12-14 05:21:06 | 2019-12-18 09:50:22 | 2019-12-18 10:42:21 | 0:51:59 | 0:27:27 | 0:24:32 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2019-12-18 10:31:16.589012 mon.a mon.0 172.21.15.154:6789/0 1109 : cluster [WRN] Health check failed: 208 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602579 | 2019-12-14 05:21:06 | 2019-12-18 09:52:14 | 2019-12-18 10:38:14 | 0:46:00 | 0:20:54 | 0:25:06 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602580 | 2019-12-14 05:21:07 | 2019-12-18 09:54:29 | 2019-12-18 10:58:29 | 1:04:00 | 0:18:26 | 0:45:34 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602581 | 2019-12-14 05:21:08 | 2019-12-18 09:54:42 | 2019-12-18 14:56:47 | 5:02:05 | 0:29:10 | 4:32:55 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 4602582 | 2019-12-14 05:21:09 | 2019-12-18 09:55:12 | 2019-12-18 10:27:24 | 0:32:12 | 0:20:49 | 0:11:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602583 | 2019-12-14 05:21:09 | 2019-12-18 09:57:24 | 2019-12-18 10:29:24 | 0:32:00 | 0:20:06 | 0:11:54 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602584 | 2019-12-14 05:21:10 | 2019-12-18 09:57:24 | 2019-12-18 11:21:25 | 1:24:01 | 1:07:52 | 0:16:09 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4602585 | 2019-12-14 05:21:11 | 2019-12-18 09:59:26 | 2019-12-18 13:37:31 | 3:38:05 | 0:12:09 | 3:25:56 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
fail | 4602586 | 2019-12-14 05:21:11 | 2019-12-18 09:59:26 | 2019-12-18 11:53:27 | 1:54:01 | 1:05:43 | 0:48:18 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:06:00.326199 mon.b mon.0 172.21.15.171:6789/0 1443 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 4602587 | 2019-12-14 05:21:12 | 2019-12-18 09:59:26 | 2019-12-18 11:15:26 | 1:16:00 | 0:46:07 | 0:29:53 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 4602588 | 2019-12-14 05:21:13 | 2019-12-17 13:01:00 | 2019-12-17 15:19:01 | 2:18:01 | 0:04:05 | 2:13:56 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 4 | |||
Failure Reason:
{'smithi148.front.sepia.ceph.com': {'msg': 'Data could not be sent to remote host "smithi148.front.sepia.ceph.com". Make sure this host can be reached over ssh: Warning: Permanently added \'smithi148.front.sepia.ceph.com,172.21.15.148\' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey,password,keyboard-interactive).\r\n', 'unreachable': True, 'changed': False}, 'smithi138.front.sepia.ceph.com': {'msg': 'Data could not be sent to remote host "smithi138.front.sepia.ceph.com". Make sure this host can be reached over ssh: ssh: connect to host smithi138.front.sepia.ceph.com port 22: No route to host\r\n', 'unreachable': True, 'changed': False}} |
||||||||||||||
pass | 4602589 | 2019-12-14 05:21:13 | 2019-12-18 09:59:32 | 2019-12-18 11:27:33 | 1:28:01 | 1:03:22 | 0:24:39 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602590 | 2019-12-14 05:21:14 | 2019-12-18 09:59:56 | 2019-12-18 11:03:56 | 1:04:00 | 0:27:03 | 0:36:57 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602591 | 2019-12-14 05:21:15 | 2019-12-18 10:00:10 | 2019-12-18 11:12:10 | 1:12:00 | 0:17:10 | 0:54:50 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4602592 | 2019-12-14 05:21:16 | 2019-12-18 10:00:11 | 2019-12-18 11:32:15 | 1:32:04 | 1:03:44 | 0:28:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602593 | 2019-12-14 05:21:16 | 2019-12-18 10:00:17 | 2019-12-18 11:10:17 | 1:10:00 | 0:20:36 | 0:49:24 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602594 | 2019-12-14 05:21:17 | 2019-12-18 10:02:17 | 2019-12-18 12:56:20 | 2:54:03 | 0:46:50 | 2:07:13 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4602595 | 2019-12-14 05:21:18 | 2019-12-18 10:02:17 | 2019-12-18 11:50:19 | 1:48:02 | 1:31:38 | 0:16:24 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602596 | 2019-12-14 05:21:18 | 2019-12-18 10:02:17 | 2019-12-18 11:06:18 | 1:04:01 | 0:37:09 | 0:26:52 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 10:43:44.988684 mon.a mon.0 172.21.15.37:6789/0 227 : cluster [WRN] Health check failed: 9092 slow requests are blocked > 32 sec. Implicated osds 0,1,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602597 | 2019-12-14 05:21:19 | 2019-12-18 10:02:17 | 2019-12-18 13:08:20 | 3:06:03 | 0:12:55 | 2:53:08 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4602598 | 2019-12-14 05:21:20 | 2019-12-18 10:04:19 | 2019-12-18 11:04:19 | 1:00:00 | 0:22:37 | 0:37:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602599 | 2019-12-14 05:21:21 | 2019-12-18 10:05:36 | 2019-12-18 10:47:35 | 0:41:59 | 0:26:52 | 0:15:07 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602600 | 2019-12-14 05:21:21 | 2019-12-18 10:07:21 | 2019-12-18 14:47:26 | 4:40:05 | 0:20:28 | 4:19:37 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4602601 | 2019-12-14 05:21:22 | 2019-12-18 10:07:27 | 2019-12-18 11:45:28 | 1:38:01 | 1:18:49 | 0:19:12 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602602 | 2019-12-14 05:21:23 | 2019-12-18 10:11:08 | 2019-12-18 11:13:09 | 1:02:01 | 0:18:32 | 0:43:29 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602603 | 2019-12-14 05:21:24 | 2019-12-18 10:13:53 | 2019-12-18 15:01:57 | 4:48:04 | 0:23:26 | 4:24:38 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4602604 | 2019-12-14 05:21:24 | 2019-12-18 10:13:53 | 2019-12-18 11:27:53 | 1:14:00 | 0:21:15 | 0:52:45 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602605 | 2019-12-14 05:21:25 | 2019-12-18 10:16:13 | 2019-12-18 11:24:13 | 1:08:00 | 0:20:03 | 0:47:57 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602606 | 2019-12-14 05:21:26 | 2019-12-18 10:17:40 | 2019-12-18 14:39:44 | 4:22:04 | 0:15:29 | 4:06:35 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4602607 | 2019-12-14 05:21:26 | 2019-12-18 10:17:40 | 2019-12-18 11:03:40 | 0:46:00 | 0:18:02 | 0:27:58 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602608 | 2019-12-14 05:21:27 | 2019-12-18 10:18:07 | 2019-12-18 10:50:07 | 0:32:00 | 0:18:05 | 0:13:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602609 | 2019-12-14 05:21:28 | 2019-12-18 10:18:16 | 2019-12-18 12:06:18 | 1:48:02 | 1:23:42 | 0:24:20 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 4602610 | 2019-12-14 05:21:29 | 2019-12-18 10:19:04 | 2019-12-18 12:49:05 | 2:30:01 | 0:13:52 | 2:16:09 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4602611 | 2019-12-14 05:21:29 | 2019-12-18 10:19:44 | 2019-12-18 11:09:44 | 0:50:00 | 0:29:44 | 0:20:16 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602612 | 2019-12-14 05:21:30 | 2019-12-18 10:20:25 | 2019-12-18 11:32:29 | 1:12:04 | 0:46:17 | 0:25:47 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602613 | 2019-12-14 05:21:31 | 2019-12-18 10:21:17 | 2019-12-18 20:29:28 | 10:08:11 | 0:46:35 | 9:21:36 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602614 | 2019-12-14 05:21:31 | 2019-12-18 10:25:15 | 2019-12-18 12:25:16 | 2:00:01 | 1:12:33 | 0:47:28 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602615 | 2019-12-14 05:21:32 | 2019-12-18 10:25:56 | 2019-12-18 11:47:56 | 1:22:00 | 0:28:26 | 0:53:34 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602616 | 2019-12-14 05:21:33 | 2019-12-16 18:57:30 | 2019-12-16 23:27:34 | 4:30:04 | 0:12:38 | 4:17:26 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
fail | 4602617 | 2019-12-14 05:21:33 | 2019-12-18 10:26:56 | 2019-12-18 12:30:57 | 2:04:01 | 1:38:06 | 0:25:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:27:39.216981 mon.b mon.0 172.21.15.72:6789/0 410 : cluster [WRN] Health check failed: 860 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602618 | 2019-12-14 05:21:34 | 2019-12-18 10:26:56 | 2019-12-18 11:48:56 | 1:22:00 | 0:33:44 | 0:48:16 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602619 | 2019-12-14 05:21:35 | 2019-12-18 10:27:25 | 2019-12-18 15:41:30 | 5:14:05 | 0:24:13 | 4:49:52 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4602620 | 2019-12-14 05:21:36 | 2019-12-18 10:27:29 | 2019-12-18 11:27:29 | 1:00:00 | 0:46:07 | 0:13:53 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602621 | 2019-12-14 05:21:36 | 2019-12-18 10:29:25 | 2019-12-18 11:53:25 | 1:24:00 | 0:37:45 | 0:46:15 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:30:42.862780 mon.a mon.0 172.21.15.23:6789/0 244 : cluster [WRN] Health check failed: 2307 slow requests are blocked > 32 sec. Implicated osds 3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602622 | 2019-12-14 05:21:37 | 2019-12-18 10:29:25 | 2019-12-18 13:15:28 | 2:46:03 | 0:12:36 | 2:33:27 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4602623 | 2019-12-14 05:21:38 | 2019-12-18 10:31:12 | 2019-12-18 11:07:11 | 0:35:59 | 0:23:43 | 0:12:16 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602624 | 2019-12-14 05:21:38 | 2019-12-18 10:31:12 | 2019-12-18 11:17:11 | 0:45:59 | 0:34:08 | 0:11:51 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602625 | 2019-12-14 05:21:39 | 2019-12-18 10:31:44 | 2019-12-18 14:11:48 | 3:40:04 | 0:14:53 | 3:25:11 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4602626 | 2019-12-14 05:21:40 | 2019-12-18 10:32:46 | 2019-12-18 11:12:45 | 0:39:59 | 0:22:01 | 0:17:58 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602627 | 2019-12-14 05:21:41 | 2019-12-18 10:35:53 | 2019-12-18 11:17:53 | 0:42:00 | 0:17:57 | 0:24:03 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602628 | 2019-12-14 05:21:41 | 2019-12-18 10:37:49 | 2019-12-18 14:19:52 | 3:42:03 | 0:27:09 | 3:14:54 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 4602629 | 2019-12-14 05:21:42 | 2019-12-18 10:37:49 | 2019-12-18 11:15:48 | 0:37:59 | 0:24:32 | 0:13:27 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:05:50.918417 mon.a mon.0 172.21.15.168:6789/0 253 : cluster [WRN] Health check failed: 493 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602630 | 2019-12-14 05:21:43 | 2019-12-18 10:38:15 | 2019-12-18 11:38:15 | 1:00:00 | 0:20:49 | 0:39:11 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602631 | 2019-12-14 05:21:43 | 2019-12-18 10:40:29 | 2019-12-18 19:54:39 | 9:14:10 | 0:29:32 | 8:44:38 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 4602632 | 2019-12-14 05:21:44 | 2019-12-18 10:40:29 | 2019-12-18 11:10:28 | 0:29:59 | 0:17:03 | 0:12:56 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602633 | 2019-12-14 05:21:45 | 2019-12-18 10:42:02 | 2019-12-18 11:22:02 | 0:40:00 | 0:18:27 | 0:21:33 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602634 | 2019-12-14 05:21:45 | 2019-12-18 10:42:02 | 2019-12-18 11:44:03 | 1:02:01 | 0:46:00 | 0:16:01 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4602635 | 2019-12-14 05:21:46 | 2019-12-18 10:42:23 | 2019-12-18 12:38:24 | 1:56:01 | 0:13:11 | 1:42:50 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602636 | 2019-12-14 05:21:47 | 2019-12-18 10:43:40 | 2019-12-18 11:41:40 | 0:58:00 | 0:39:56 | 0:18:04 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 4602637 | 2019-12-14 05:21:47 | 2019-12-18 10:43:40 | 2019-12-18 12:25:41 | 1:42:01 | 1:19:43 | 0:22:18 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:23:05.190708 mon.a mon.0 172.21.15.17:6789/0 158 : cluster [WRN] Health check failed: 2341 slow requests are blocked > 32 sec. Implicated osds 2,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602638 | 2019-12-14 05:21:48 | 2019-12-18 10:47:58 | 2019-12-18 15:42:02 | 4:54:04 | 0:11:17 | 4:42:47 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4602639 | 2019-12-14 05:21:49 | 2019-12-18 10:50:29 | 2019-12-18 12:00:29 | 1:10:00 | 0:37:58 | 0:32:02 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602640 | 2019-12-14 05:21:49 | 2019-12-18 10:51:48 | 2019-12-18 11:43:48 | 0:52:00 | 0:27:59 | 0:24:01 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602641 | 2019-12-14 05:21:50 | 2019-12-18 10:51:57 | 2019-12-18 16:52:05 | 6:00:08 | 0:16:01 | 5:44:07 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4602642 | 2019-12-14 05:21:51 | 2019-12-18 10:53:12 | 2019-12-18 13:35:14 | 2:42:02 | 1:56:59 | 0:45:03 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602643 | 2019-12-14 05:21:52 | 2019-12-18 10:53:15 | 2019-12-18 11:25:15 | 0:32:00 | 0:21:22 | 0:10:38 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602644 | 2019-12-14 05:21:52 | 2019-12-18 10:53:30 | 2019-12-18 13:01:32 | 2:08:02 | 0:45:01 | 1:23:01 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4602645 | 2019-12-14 05:21:53 | 2019-12-18 10:54:40 | 2019-12-18 12:24:42 | 1:30:02 | 1:03:21 | 0:26:41 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602646 | 2019-12-14 05:21:54 | 2019-12-18 10:54:41 | 2019-12-18 12:56:42 | 2:02:01 | 1:31:41 | 0:30:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:40:53.278567 mon.a mon.0 172.21.15.168:6789/0 252 : cluster [WRN] Health check failed: 10820 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602647 | 2019-12-14 05:21:54 | 2019-12-18 11:02:09 | 2019-12-18 13:42:12 | 2:40:03 | 0:12:03 | 2:28:00 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4602648 | 2019-12-14 05:21:55 | 2019-12-18 11:04:03 | 2019-12-18 11:58:02 | 0:53:59 | 0:20:30 | 0:33:29 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602649 | 2019-12-14 05:21:56 | 2019-12-18 11:04:03 | 2019-12-18 12:12:03 | 1:08:00 | 0:33:40 | 0:34:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602650 | 2019-12-14 05:21:56 | 2019-12-18 11:04:03 | 2019-12-18 15:46:07 | 4:42:04 | 0:20:52 | 4:21:12 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4602651 | 2019-12-14 05:21:57 | 2019-12-18 11:04:05 | 2019-12-18 11:48:05 | 0:44:00 | 0:30:50 | 0:13:10 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602652 | 2019-12-14 05:21:58 | 2019-12-18 11:04:20 | 2019-12-18 11:36:20 | 0:32:00 | 0:19:05 | 0:12:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602653 | 2019-12-14 05:21:59 | 2019-12-18 11:06:43 | 2019-12-18 16:22:47 | 5:16:04 | 0:25:44 | 4:50:20 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4602654 | 2019-12-14 05:21:59 | 2019-12-18 11:07:12 | 2019-12-18 11:47:12 | 0:40:00 | 0:20:44 | 0:19:16 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602655 | 2019-12-14 05:22:00 | 2019-12-18 11:10:03 | 2019-12-18 11:50:02 | 0:39:59 | 0:20:04 | 0:19:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602656 | 2019-12-14 05:22:01 | 2019-12-18 11:10:03 | 2019-12-18 16:20:07 | 5:10:04 | 0:19:15 | 4:50:49 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4602657 | 2019-12-14 05:22:01 | 2019-12-18 11:10:19 | 2019-12-18 11:42:18 | 0:31:59 | 0:17:36 | 0:14:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602658 | 2019-12-14 05:22:02 | 2019-12-18 11:10:29 | 2019-12-18 11:42:29 | 0:32:00 | 0:18:58 | 0:13:02 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602659 | 2019-12-14 05:22:03 | 2019-12-18 11:12:29 | 2019-12-18 12:20:29 | 1:08:00 | 0:54:13 | 0:13:47 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 4602660 | 2019-12-14 05:22:03 | 2019-12-18 11:12:47 | 2019-12-18 12:08:47 | 0:56:00 | 0:17:10 | 0:38:50 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4602661 | 2019-12-14 05:22:04 | 2019-12-18 11:13:11 | 2019-12-18 11:45:10 | 0:31:59 | 0:21:17 | 0:10:42 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602662 | 2019-12-14 05:22:05 | 2019-12-18 11:14:49 | 2019-12-18 12:56:49 | 1:42:00 | 1:24:21 | 0:17:39 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602663 | 2019-12-14 05:22:06 | 2019-12-18 11:14:54 | 2019-12-18 14:06:56 | 2:52:02 | 0:45:56 | 2:06:06 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602664 | 2019-12-14 05:22:06 | 2019-12-18 11:15:37 | 2019-12-18 12:07:36 | 0:51:59 | 0:40:23 | 0:11:36 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602665 | 2019-12-14 05:22:07 | 2019-12-18 11:15:37 | 2019-12-18 11:59:36 | 0:43:59 | 0:28:45 | 0:15:14 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602666 | 2019-12-14 05:22:08 | 2019-12-18 11:15:55 | 2019-12-18 16:18:00 | 5:02:05 | 0:14:21 | 4:47:44 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 4602667 | 2019-12-14 05:22:08 | 2019-12-18 11:17:40 | 2019-12-18 13:13:40 | 1:56:00 | 1:09:17 | 0:46:43 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 4602668 | 2019-12-14 05:22:09 | 2019-12-18 11:17:54 | 2019-12-18 12:53:55 | 1:36:01 | 1:21:58 | 0:14:03 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:53:38.598843 mon.b mon.0 172.21.15.45:6789/0 320 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 4602669 | 2019-12-14 05:22:10 | 2019-12-18 11:19:53 | 2019-12-18 17:05:59 | 5:46:06 | 0:22:52 | 5:23:14 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4602670 | 2019-12-14 05:22:10 | 2019-12-18 11:21:50 | 2019-12-18 12:53:50 | 1:32:00 | 1:12:33 | 0:19:27 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602671 | 2019-12-14 05:22:11 | 2019-12-18 11:22:03 | 2019-12-18 13:08:04 | 1:46:01 | 1:33:48 | 0:12:13 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 11:48:32.658477 mon.b mon.0 172.21.15.44:6789/0 229 : cluster [WRN] Health check failed: 8551 slow requests are blocked > 32 sec. Implicated osds 0,2,4,5,6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602672 | 2019-12-14 05:22:12 | 2019-12-18 11:24:38 | 2019-12-18 12:16:37 | 0:51:59 | 0:14:26 | 0:37:33 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4602673 | 2019-12-14 05:22:12 | 2019-12-18 11:27:17 | 2019-12-18 12:01:16 | 0:33:59 | 0:21:52 | 0:12:07 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602674 | 2019-12-14 05:22:13 | 2019-12-18 11:27:31 | 2019-12-18 12:43:31 | 1:16:00 | 1:01:41 | 0:14:19 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602675 | 2019-12-14 05:22:14 | 2019-12-18 11:32:19 | 2019-12-18 12:44:19 | 1:12:00 | 0:13:41 | 0:58:19 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4602676 | 2019-12-14 05:22:14 | 2019-12-18 11:32:19 | 2019-12-18 12:08:19 | 0:36:00 | 0:20:14 | 0:15:46 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602677 | 2019-12-14 05:22:15 | 2019-12-18 11:32:31 | 2019-12-18 12:02:30 | 0:29:59 | 0:17:35 | 0:12:24 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602678 | 2019-12-14 05:22:16 | 2019-12-18 11:35:44 | 2019-12-18 19:11:51 | 7:36:07 | 0:27:22 | 7:08:45 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
pass | 4602679 | 2019-12-14 05:22:16 | 2019-12-18 11:36:06 | 2019-12-18 12:08:06 | 0:32:00 | 0:19:34 | 0:12:26 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602680 | 2019-12-14 05:22:17 | 2019-12-18 11:36:21 | 2019-12-18 12:10:21 | 0:34:00 | 0:18:52 | 0:15:08 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602681 | 2019-12-14 05:22:18 | 2019-12-18 11:37:57 | 2019-12-18 13:23:57 | 1:46:00 | 0:30:07 | 1:15:53 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 4602682 | 2019-12-14 05:22:18 | 2019-12-18 11:38:17 | 2019-12-18 12:10:16 | 0:31:59 | 0:17:24 | 0:14:35 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602683 | 2019-12-14 05:22:19 | 2019-12-18 11:42:04 | 2019-12-18 12:10:03 | 0:27:59 | 0:17:56 | 0:10:03 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602684 | 2019-12-14 05:22:20 | 2019-12-18 11:42:20 | 2019-12-18 13:24:21 | 1:42:01 | 1:28:32 | 0:13:29 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4602685 | 2019-12-14 05:22:20 | 2019-12-18 11:42:30 | 2019-12-18 12:36:30 | 0:54:00 | 0:13:25 | 0:40:35 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602686 | 2019-12-14 05:22:21 | 2019-12-18 11:44:13 | 2019-12-18 13:10:13 | 1:26:00 | 1:13:48 | 0:12:12 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602687 | 2019-12-14 05:22:22 | 2019-12-18 11:44:13 | 2019-12-18 12:34:13 | 0:50:00 | 0:37:50 | 0:12:10 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 4602688 | 2019-12-14 05:22:23 | 2019-12-18 11:45:35 | 2019-12-18 23:47:50 | 12:02:15 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | — | |||||
pass | 4602689 | 2019-12-14 05:22:23 | 2019-12-18 11:45:35 | 2019-12-18 12:37:34 | 0:51:59 | 0:41:31 | 0:10:28 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602690 | 2019-12-14 05:22:24 | 2019-12-18 11:47:36 | 2019-12-18 12:27:35 | 0:39:59 | 0:28:37 | 0:11:22 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602691 | 2019-12-14 05:22:25 | 2019-12-18 11:48:03 | 2019-12-18 15:18:05 | 3:30:02 | 0:17:16 | 3:12:46 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4602692 | 2019-12-14 05:22:25 | 2019-12-18 11:48:06 | 2019-12-18 13:10:07 | 1:22:01 | 1:09:39 | 0:12:22 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602693 | 2019-12-14 05:22:26 | 2019-12-18 11:49:20 | 2019-12-18 12:27:19 | 0:37:59 | 0:25:40 | 0:12:19 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602694 | 2019-12-14 05:22:27 | 2019-12-18 11:49:20 | 2019-12-18 22:41:31 | 10:52:11 | 0:47:33 | 10:04:38 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4602695 | 2019-12-14 05:22:28 | 2019-12-18 11:50:04 | 2019-12-18 13:30:05 | 1:40:01 | 1:24:59 | 0:15:02 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602696 | 2019-12-14 05:22:28 | 2019-12-18 11:50:43 | 2019-12-18 12:36:42 | 0:45:59 | 0:32:50 | 0:13:09 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 12:18:22.434611 mon.a mon.0 172.21.15.23:6789/0 238 : cluster [WRN] Health check failed: 259 slow requests are blocked > 32 sec. Implicated osds 0,1,3,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602697 | 2019-12-14 05:22:29 | 2019-12-18 11:53:51 | 2019-12-18 13:15:51 | 1:22:00 | 0:12:12 | 1:09:48 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4602698 | 2019-12-14 05:22:30 | 2019-12-18 11:53:51 | 2019-12-18 12:35:50 | 0:41:59 | 0:27:12 | 0:14:47 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602699 | 2019-12-14 05:22:30 | 2019-12-18 11:58:04 | 2019-12-18 13:06:03 | 1:07:59 | 0:56:00 | 0:11:59 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602700 | 2019-12-14 05:22:31 | 2019-12-18 11:58:04 | 2019-12-18 13:16:04 | 1:18:00 | 0:21:49 | 0:56:11 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4602701 | 2019-12-14 05:22:32 | 2019-12-18 11:58:04 | 2019-12-18 12:58:04 | 1:00:00 | 0:41:35 | 0:18:25 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602702 | 2019-12-14 05:22:32 | 2019-12-18 12:00:00 | 2019-12-18 12:36:01 | 0:36:01 | 0:19:57 | 0:16:04 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602703 | 2019-12-14 05:22:33 | 2019-12-18 12:00:00 | 2019-12-18 15:08:03 | 3:08:03 | 0:25:35 | 2:42:28 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4602704 | 2019-12-14 05:22:34 | 2019-12-18 12:00:31 | 2019-12-18 12:38:30 | 0:37:59 | 0:21:17 | 0:16:42 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602705 | 2019-12-14 05:22:34 | 2019-12-18 12:03:57 | 2019-12-18 12:39:57 | 0:36:00 | 0:18:54 | 0:17:06 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602706 | 2019-12-14 05:22:35 | 2019-12-18 12:06:41 | 2019-12-18 14:44:43 | 2:38:02 | 0:16:09 | 2:21:53 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4602707 | 2019-12-14 05:22:36 | 2019-12-18 12:07:58 | 2019-12-18 12:37:57 | 0:29:59 | 0:17:17 | 0:12:42 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602708 | 2019-12-14 05:22:37 | 2019-12-18 12:08:06 | 2019-12-18 13:52:07 | 1:44:01 | 0:52:43 | 0:51:18 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602709 | 2019-12-14 05:22:37 | 2019-12-18 12:08:07 | 2019-12-18 13:26:08 | 1:18:01 | 0:52:07 | 0:25:54 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 4602710 | 2019-12-14 05:22:38 | 2019-12-18 12:08:20 | 2019-12-18 13:34:20 | 1:26:00 | 0:19:16 | 1:06:44 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4602711 | 2019-12-14 05:22:39 | 2019-12-18 12:08:49 | 2019-12-18 12:43:59 | 0:35:10 | 0:22:49 | 0:12:21 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602712 | 2019-12-14 05:22:39 | 2019-12-18 12:10:31 | 2019-12-18 13:24:30 | 1:13:59 | 0:45:33 | 0:28:26 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602713 | 2019-12-14 05:22:40 | 2019-12-18 12:13:22 | 2019-12-18 17:11:27 | 4:58:05 | 0:45:45 | 4:12:20 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602714 | 2019-12-14 05:22:41 | 2019-12-18 12:10:31 | 2019-12-18 13:38:31 | 1:28:00 | 1:02:46 | 0:25:14 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602715 | 2019-12-14 05:22:41 | 2019-12-18 12:13:22 | 2019-12-18 13:07:22 | 0:54:00 | 0:26:51 | 0:27:09 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602716 | 2019-12-14 05:22:42 | 2019-12-18 12:17:05 | 2019-12-18 13:07:04 | 0:49:59 | 0:11:54 | 0:38:05 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 4602717 | 2019-12-14 05:22:43 | 2019-12-18 12:20:40 | 2019-12-18 13:44:40 | 1:24:00 | 1:04:19 | 0:19:41 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602718 | 2019-12-14 05:22:43 | 2019-12-18 12:20:40 | 2019-12-18 13:14:39 | 0:53:59 | 0:38:16 | 0:15:43 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602719 | 2019-12-14 05:22:44 | 2019-12-18 12:25:05 | 2019-12-18 15:47:08 | 3:22:03 | 0:24:15 | 2:57:48 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4602720 | 2019-12-14 05:22:45 | 2019-12-18 12:25:17 | 2019-12-18 14:07:18 | 1:42:01 | 1:28:10 | 0:13:51 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602721 | 2019-12-14 05:22:45 | 2019-12-18 12:25:42 | 2019-12-18 13:31:42 | 1:06:00 | 0:45:40 | 0:20:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 13:02:55.959841 mon.a mon.0 172.21.15.60:6789/0 149 : cluster [WRN] Health check failed: 1574 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602722 | 2019-12-14 05:22:46 | 2019-12-18 12:26:52 | 2019-12-18 13:42:53 | 1:16:01 | 0:11:56 | 1:04:05 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4602723 | 2019-12-14 05:22:47 | 2019-12-18 12:27:20 | 2019-12-18 13:05:20 | 0:38:00 | 0:22:55 | 0:15:05 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602724 | 2019-12-14 05:22:47 | 2019-12-18 12:27:37 | 2019-12-18 13:15:36 | 0:47:59 | 0:29:42 | 0:18:17 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602725 | 2019-12-14 05:22:48 | 2019-12-18 12:31:42 | 2019-12-18 14:25:43 | 1:54:01 | 0:15:57 | 1:38:04 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4602726 | 2019-12-14 05:22:49 | 2019-12-18 12:34:44 | 2019-12-18 13:12:43 | 0:37:59 | 0:24:08 | 0:13:51 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602727 | 2019-12-14 05:22:50 | 2019-12-18 12:36:13 | 2019-12-18 13:08:12 | 0:31:59 | 0:20:19 | 0:11:40 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602728 | 2019-12-14 05:22:50 | 2019-12-18 12:36:13 | 2019-12-18 19:54:21 | 7:18:08 | 0:27:02 | 6:51:06 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
pass | 4602729 | 2019-12-14 05:22:51 | 2019-12-18 12:36:13 | 2019-12-18 13:12:12 | 0:35:59 | 0:24:32 | 0:11:27 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602730 | 2019-12-14 05:22:52 | 2019-12-18 12:36:32 | 2019-12-18 13:10:31 | 0:33:59 | 0:21:02 | 0:12:57 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4602731 | 2019-12-14 05:22:52 | 2019-12-18 12:36:44 | 2019-12-19 00:39:02 | 12:02:18 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 4602732 | 2019-12-14 05:22:53 | 2019-12-18 12:38:05 | 2019-12-18 13:12:04 | 0:33:59 | 0:17:26 | 0:16:33 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602733 | 2019-12-14 05:22:54 | 2019-12-18 12:38:05 | 2019-12-18 13:12:04 | 0:33:59 | 0:17:14 | 0:16:45 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602734 | 2019-12-14 05:22:54 | 2019-12-18 12:38:26 | 2019-12-18 15:20:28 | 2:42:02 | 1:43:37 | 0:58:25 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4602735 | 2019-12-14 05:22:55 | 2019-12-18 12:38:31 | 2019-12-18 13:34:31 | 0:56:00 | 0:12:50 | 0:43:10 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602736 | 2019-12-14 05:22:56 | 2019-12-18 12:40:31 | 2019-12-18 13:40:29 | 0:59:58 | 0:32:35 | 0:27:23 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602737 | 2019-12-14 05:22:57 | 2019-12-18 12:40:31 | 2019-12-18 13:40:29 | 0:59:58 | 0:45:19 | 0:14:39 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602738 | 2019-12-14 05:22:57 | 2019-12-18 12:43:56 | 2019-12-18 14:43:56 | 2:00:00 | 0:11:05 | 1:48:55 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4602739 | 2019-12-14 05:22:58 | 2019-12-18 12:44:00 | 2019-12-18 14:14:01 | 1:30:01 | 1:09:33 | 0:20:28 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602740 | 2019-12-14 05:22:59 | 2019-12-18 12:44:21 | 2019-12-18 13:38:21 | 0:54:00 | 0:27:18 | 0:26:42 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602741 | 2019-12-14 05:22:59 | 2019-12-18 12:53:29 | 2019-12-18 15:49:31 | 2:56:02 | 0:19:26 | 2:36:36 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4602742 | 2019-12-14 05:23:00 | 2019-12-18 12:53:52 | 2019-12-18 14:51:53 | 1:58:01 | 1:45:02 | 0:12:59 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602743 | 2019-12-14 05:23:01 | 2019-12-18 12:54:06 | 2019-12-18 13:28:05 | 0:33:59 | 0:21:12 | 0:12:47 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602744 | 2019-12-14 05:23:01 | 2019-12-18 12:56:45 | 2019-12-18 15:10:46 | 2:14:01 | 0:49:27 | 1:24:34 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4602745 | 2019-12-14 05:23:02 | 2019-12-18 12:56:45 | 2019-12-18 13:54:44 | 0:57:59 | 0:45:50 | 0:12:09 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602746 | 2019-12-14 05:23:03 | 2019-12-18 12:56:50 | 2019-12-18 13:46:50 | 0:50:00 | 0:36:16 | 0:13:44 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 13:24:36.094002 mon.a mon.0 172.21.15.83:6789/0 148 : cluster [WRN] Health check failed: 1693 slow requests are blocked > 32 sec. Implicated osds 0,1,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602747 | 2019-12-14 05:23:03 | 2019-12-18 12:58:27 | 2019-12-18 15:46:29 | 2:48:02 | 0:11:49 | 2:36:13 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4602748 | 2019-12-14 05:23:04 | 2019-12-18 13:01:56 | 2019-12-18 13:39:55 | 0:37:59 | 0:23:39 | 0:14:20 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602749 | 2019-12-14 05:23:05 | 2019-12-18 13:05:44 | 2019-12-18 13:59:44 | 0:54:00 | 0:38:20 | 0:15:40 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602750 | 2019-12-14 05:23:05 | 2019-12-18 13:06:05 | 2019-12-18 15:22:06 | 2:16:01 | 0:21:17 | 1:54:44 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4602751 | 2019-12-14 05:23:06 | 2019-12-18 13:07:30 | 2019-12-18 14:41:30 | 1:34:00 | 1:18:54 | 0:15:06 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602752 | 2019-12-14 05:23:07 | 2019-12-18 13:07:30 | 2019-12-18 13:39:29 | 0:31:59 | 0:18:25 | 0:13:34 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602753 | 2019-12-14 05:23:08 | 2019-12-18 13:08:06 | 2019-12-18 21:30:15 | 8:22:09 | 0:25:07 | 7:57:02 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 4602754 | 2019-12-14 05:23:08 | 2019-12-18 13:08:14 | 2019-12-18 13:46:13 | 0:37:59 | 0:23:54 | 0:14:05 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2019-12-18 13:36:58.571367 mon.a mon.0 172.21.15.68:6789/0 151 : cluster [WRN] Health check failed: 560 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602755 | 2019-12-14 05:23:09 | 2019-12-18 13:08:21 | 2019-12-18 13:40:21 | 0:32:00 | 0:19:27 | 0:12:33 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4602756 | 2019-12-14 05:23:10 | 2019-12-18 13:10:31 | 2019-12-18 23:06:42 | 9:56:11 | 0:16:07 | 9:40:04 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4602757 | 2019-12-14 05:23:10 | 2019-12-18 13:10:31 | 2019-12-18 13:40:30 | 0:29:59 | 0:17:19 | 0:12:40 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4602758 | 2019-12-14 05:23:11 | 2019-12-18 13:10:33 | 2019-12-18 13:42:32 | 0:31:59 | 0:18:58 | 0:13:01 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4602759 | 2019-12-14 05:23:12 | 2019-12-18 13:12:27 | 2019-12-18 14:30:27 | 1:18:00 | 0:39:12 | 0:38:48 | smithi | master | kcephfs/mixed-clients/{begin.yaml clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
pass | 4602760 | 2019-12-14 05:23:12 | 2019-12-18 13:12:27 | 2019-12-18 14:02:26 | 0:49:59 | 0:14:08 | 0:35:51 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4602761 | 2019-12-14 05:23:13 | 2019-12-18 13:12:27 | 2019-12-18 13:44:26 | 0:31:59 | 0:20:57 | 0:11:02 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602762 | 2019-12-14 05:23:14 | 2019-12-18 13:12:45 | 2019-12-18 14:48:45 | 1:36:00 | 1:13:47 | 0:22:13 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
pass | 4602763 | 2019-12-14 05:23:14 | 2019-12-18 13:14:00 | 2019-12-18 16:48:05 | 3:34:05 | 0:47:55 | 2:46:10 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4602764 | 2019-12-14 05:23:15 | 2019-12-18 13:14:02 | 2019-12-18 14:12:02 | 0:58:00 | 0:36:30 | 0:21:30 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4602765 | 2019-12-14 05:23:16 | 2019-12-18 13:14:41 | 2019-12-18 13:54:40 | 0:39:59 | 0:26:53 | 0:13:06 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4602766 | 2019-12-14 05:23:16 | 2019-12-18 13:15:55 | 2019-12-18 15:07:56 | 1:52:01 | 0:13:16 | 1:38:45 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 4602767 | 2019-12-14 05:23:17 | 2019-12-18 13:15:55 | 2019-12-18 15:15:56 | 2:00:01 | 1:46:38 | 0:13:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4602768 | 2019-12-14 05:23:18 | 2019-12-16 18:57:30 | 2019-12-16 20:29:30 | 1:32:00 | 1:19:31 | 0:12:29 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4602769 | 2019-12-14 05:23:19 | 2019-12-18 13:15:55 | 2019-12-18 19:12:37 | 5:56:42 | 0:23:40 | 5:33:02 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4602770 | 2019-12-14 05:23:19 | 2019-12-18 13:16:05 | 2019-12-18 14:44:06 | 1:28:01 | 1:09:34 | 0:18:27 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4602771 | 2019-12-14 05:23:20 | 2019-12-18 13:20:34 | 2019-12-18 15:10:35 | 1:50:01 | 1:25:23 | 0:24:38 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-12-18 13:59:35.385026 mon.a mon.0 172.21.15.2:6789/0 160 : cluster [WRN] Health check failed: 6085 slow requests are blocked > 32 sec. Implicated osds 0,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4602772 | 2019-12-14 05:23:21 | 2019-12-18 13:22:23 | 2019-12-18 14:22:22 | 0:59:59 | 0:12:23 | 0:47:36 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4602773 | 2019-12-14 05:23:21 | 2019-12-18 13:24:24 | 2019-12-18 14:02:23 | 0:37:59 | 0:21:36 | 0:16:23 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4602774 | 2019-12-14 05:23:22 | 2019-12-18 13:24:24 | 2019-12-18 14:14:23 | 0:49:59 | 0:34:55 | 0:15:04 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4602775 | 2019-12-14 05:23:23 | 2019-12-18 13:24:32 | 2019-12-18 14:28:32 | 1:04:00 | 0:14:08 | 0:49:52 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4602776 | 2019-12-14 05:23:23 | 2019-12-18 13:26:35 | 2019-12-18 14:12:34 | 0:45:59 | 0:26:19 | 0:19:40 | smithi | master | kcephfs/thrash/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602777 | 2019-12-14 05:23:24 | 2019-12-18 13:28:30 | 2019-12-18 14:00:30 | 0:32:00 | 0:19:19 | 0:12:41 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4602778 | 2019-12-14 05:23:25 | 2019-12-18 13:30:27 | 2019-12-18 16:30:29 | 3:00:02 | 0:30:31 | 2:29:31 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
pass | 4602779 | 2019-12-14 05:23:25 | 2019-12-18 13:32:04 | 2019-12-18 14:06:04 | 0:34:00 | 0:21:02 | 0:12:58 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4602780 | 2019-12-14 05:23:26 | 2019-12-18 13:34:41 | 2019-12-18 14:10:40 | 0:35:59 | 0:21:04 | 0:14:55 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
fail | 4602781 | 2019-12-14 05:23:27 | 2019-12-18 13:34:41 | 2019-12-18 14:52:41 | 1:18:00 | 0:30:28 | 0:47:32 | smithi | master | kcephfs/recovery/{begin.yaml clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2019-12-18 14:31:35.551459 mon.b mon.0 172.21.15.47:6789/0 7 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log |
||||||||||||||
pass | 4602782 | 2019-12-14 05:23:28 | 2019-12-18 13:35:15 | 2019-12-18 14:07:15 | 0:32:00 | 0:17:07 | 0:14:53 | smithi | master | kcephfs/cephfs/{begin.yaml clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 |