User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
---|---|---|---|---|---|---|---|---|---|---|---|
teuthology | 2018-12-08 05:20:02 | 2018-12-19 20:51:28 | 2018-12-21 23:24:17 | 2 days, 2:32:49 | kcephfs | luminous | ovh | 5e2a4bf | 129 | 60 | 61 |
Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pass | 3318073 | 2018-12-08 05:20:21 | 2018-12-19 19:51:32 | 2018-12-19 21:57:33 | 2:06:01 | 0:23:01 | 1:43:00 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318074 | 2018-12-08 05:20:22 | 2018-12-19 19:59:19 | 2018-12-20 03:31:26 | 7:32:07 | 2:35:39 | 4:56:28 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-12-20 01:27:59.335446 mon.b mon.0 158.69.66.91:6789/0 146 : cluster [WRN] Health check failed: 5 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318075 | 2018-12-08 05:20:22 | 2018-12-19 20:01:30 | 2018-12-20 08:03:41 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | — | |||||
fail | 3318076 | 2018-12-08 05:20:23 | 2018-12-19 20:23:21 | 2018-12-20 01:41:26 | 5:18:05 | 3:39:21 | 1:38:44 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh096 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2a4bf731c2a02f4afa4da597c4954437d14a87 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
fail | 3318077 | 2018-12-08 05:20:24 | 2018-12-19 20:39:17 | 2018-12-20 00:35:20 | 3:56:03 | 1:47:30 | 2:08:33 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-19 23:08:14.099149 mon.b mon.0 158.69.69.238:6789/0 139 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 7 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318078 | 2018-12-08 05:20:24 | 2018-12-19 20:47:12 | 2018-12-20 08:49:25 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | — | |||||
dead | 3318079 | 2018-12-08 05:20:25 | 2018-12-19 20:49:16 | 2018-12-20 08:51:50 | 12:02:34 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||||
pass | 3318080 | 2018-12-08 05:20:26 | 2018-12-19 20:51:28 | 2018-12-19 21:51:28 | 1:00:00 | 0:27:54 | 0:32:06 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 3318081 | 2018-12-08 05:20:26 | 2018-12-19 20:51:28 | 2018-12-20 08:53:39 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 3318082 | 2018-12-08 05:20:27 | 2018-12-19 20:57:39 | 2018-12-19 23:27:40 | 2:30:01 | 1:35:37 | 0:54:24 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3318083 | 2018-12-08 05:20:28 | 2018-12-19 20:57:39 | 2018-12-20 00:27:46 | 3:30:07 | 1:39:26 | 1:50:41 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3318084 | 2018-12-08 05:20:28 | 2018-12-19 21:03:26 | 2018-12-20 09:05:37 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | — | |||||
pass | 3318085 | 2018-12-08 05:20:29 | 2018-12-19 21:05:21 | 2018-12-19 22:47:21 | 1:42:00 | 0:47:31 | 0:54:29 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318086 | 2018-12-08 05:20:30 | 2018-12-19 21:21:24 | 2018-12-20 00:11:26 | 2:50:02 | 0:53:58 | 1:56:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-19 23:45:43.880671 mon.b mon.0 158.69.68.158:6789/0 137 : cluster [WRN] Health check failed: 219 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318087 | 2018-12-08 05:20:30 | 2018-12-19 21:25:19 | 2018-12-20 09:27:31 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 3318088 | 2018-12-08 05:20:31 | 2018-12-19 21:33:23 | 2018-12-19 23:03:24 | 1:30:01 | 0:38:47 | 0:51:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3318089 | 2018-12-08 05:20:32 | 2018-12-19 21:35:10 | 2018-12-20 01:13:13 | 3:38:03 | 0:57:16 | 2:40:47 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 3318090 | 2018-12-08 05:20:32 | 2018-12-19 21:39:03 | 2018-12-20 09:41:14 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | |||||
fail | 3318091 | 2018-12-08 05:20:33 | 2018-12-19 21:51:32 | 2018-12-20 00:29:34 | 2:38:02 | 1:48:15 | 0:49:47 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-19 23:32:53.558299 mon.a mon.1 158.69.65.7:6789/0 76 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 3318092 | 2018-12-08 05:20:34 | 2018-12-19 21:52:49 | 2018-12-20 00:10:50 | 2:18:01 | 0:28:33 | 1:49:28 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
Failure Reason:
"2018-12-20 00:01:36.472632 mon.a mon.0 158.69.65.170:6789/0 152 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318093 | 2018-12-08 05:20:35 | 2018-12-19 21:55:28 | 2018-12-20 09:57:39 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | — | |||||
fail | 3318094 | 2018-12-08 05:20:35 | 2018-12-19 21:57:38 | 2018-12-20 00:55:40 | 2:58:02 | 2:15:00 | 0:43:02 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-19 23:16:17.480649 mon.a mon.0 158.69.72.177:6789/0 150 : cluster [WRN] Health check failed: 2 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318095 | 2018-12-08 05:20:36 | 2018-12-19 22:06:34 | 2018-12-19 23:58:34 | 1:52:00 | 0:18:44 | 1:33:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 3318096 | 2018-12-08 05:20:37 | 2018-12-19 22:17:30 | 2018-12-20 10:19:41 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | — | |||||
pass | 3318097 | 2018-12-08 05:20:38 | 2018-12-19 22:19:23 | 2018-12-19 23:21:23 | 1:02:00 | 0:38:15 | 0:23:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318098 | 2018-12-08 05:20:38 | 2018-12-19 22:41:27 | 2018-12-19 23:55:27 | 1:14:00 | 0:18:30 | 0:55:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3318099 | 2018-12-08 05:20:39 | 2018-12-19 22:47:35 | 2018-12-20 02:23:37 | 3:36:02 | 1:33:00 | 2:03:02 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
dead | 3318100 | 2018-12-08 05:20:40 | 2018-12-19 22:55:23 | 2018-12-20 10:57:35 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
fail | 3318101 | 2018-12-08 05:20:40 | 2018-12-19 22:58:34 | 2018-12-20 01:26:35 | 2:28:01 | 1:39:27 | 0:48:34 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
timed out waiting for admin_socket to appear after osd.0 restart |
||||||||||||||
fail | 3318102 | 2018-12-08 05:20:41 | 2018-12-19 23:03:35 | 2018-12-20 02:07:37 | 3:04:02 | 1:24:42 | 1:39:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-20 01:17:49.523995 mon.b mon.0 158.69.68.129:6789/0 255 : cluster [WRN] Health check failed: 85 slow requests are blocked > 32 sec. Implicated osds 5,6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318103 | 2018-12-08 05:20:42 | 2018-12-19 23:03:35 | 2018-12-20 11:05:47 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | — | |||||
fail | 3318104 | 2018-12-08 05:20:42 | 2018-12-19 23:09:01 | 2018-12-20 03:07:04 | 3:58:03 | 1:30:59 | 2:27:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
Failure Reason:
"2018-12-20 02:27:20.689043 mon.b mon.0 158.69.65.95:6789/0 251 : cluster [WRN] Health check failed: 9 slow requests are blocked > 32 sec. Implicated osds 3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318105 | 2018-12-08 05:20:43 | 2018-12-19 23:21:38 | 2018-12-20 00:47:38 | 1:26:00 | 0:26:44 | 0:59:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 3318106 | 2018-12-08 05:20:44 | 2018-12-19 23:27:19 | 2018-12-20 11:29:31 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
dead | 3318107 | 2018-12-08 05:20:44 | 2018-12-19 23:27:41 | 2018-12-20 11:34:52 | 12:07:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
pass | 3318108 | 2018-12-08 05:20:45 | 2018-12-19 23:29:00 | 2018-12-20 01:57:02 | 2:28:02 | 1:04:07 | 1:23:55 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
dead | 3318109 | 2018-12-08 05:20:46 | 2018-12-19 23:33:40 | 2018-12-20 11:35:52 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | — | |||||
pass | 3318110 | 2018-12-08 05:20:47 | 2018-12-19 23:39:35 | 2018-12-20 02:11:36 | 2:32:01 | 1:12:39 | 1:19:22 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318111 | 2018-12-08 05:20:47 | 2018-12-19 23:55:40 | 2018-12-20 01:45:41 | 1:50:01 | 0:48:40 | 1:01:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 01:20:28.572428 mon.a mon.0 158.69.72.9:6789/0 151 : cluster [WRN] Health check failed: 274 slow requests are blocked > 32 sec. Implicated osds 0,1,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318112 | 2018-12-08 05:20:48 | 2018-12-19 23:58:37 | 2018-12-20 12:00:48 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | — | |||||
fail | 3318113 | 2018-12-08 05:20:49 | 2018-12-20 00:11:03 | 2018-12-20 02:15:06 | 2:04:03 | 1:20:49 | 0:43:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
Failure Reason:
"2018-12-20 01:16:46.385938 mon.a mon.0 158.69.65.129:6789/0 148 : cluster [WRN] Health check failed: 9 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318114 | 2018-12-08 05:20:49 | 2018-12-20 00:11:27 | 2018-12-20 02:35:28 | 2:24:01 | 1:27:09 | 0:56:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 3318115 | 2018-12-08 05:20:50 | 2018-12-20 00:22:42 | 2018-12-20 12:24:54 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | — | |||||
pass | 3318116 | 2018-12-08 05:20:51 | 2018-12-20 00:27:47 | 2018-12-20 03:31:49 | 3:04:02 | 1:44:47 | 1:19:15 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318117 | 2018-12-08 05:20:52 | 2018-12-20 00:27:47 | 2018-12-20 02:09:48 | 1:42:01 | 0:18:11 | 1:23:50 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
dead | 3318118 | 2018-12-08 05:20:52 | 2018-12-20 00:29:47 | 2018-12-20 12:31:58 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | — | |||||
pass | 3318119 | 2018-12-08 05:20:53 | 2018-12-20 00:35:34 | 2018-12-20 03:13:36 | 2:38:02 | 1:22:44 | 1:15:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318120 | 2018-12-08 05:20:54 | 2018-12-20 00:43:33 | 2018-12-20 01:19:32 | 0:35:59 | 0:19:40 | 0:16:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3318121 | 2018-12-08 05:20:54 | 2018-12-20 00:47:51 | 2018-12-20 04:59:54 | 4:12:03 | 0:42:52 | 3:29:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3318122 | 2018-12-08 05:20:55 | 2018-12-20 04:31:33 | 2018-12-20 05:07:33 | 0:36:00 | 0:16:08 | 0:19:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318123 | 2018-12-08 05:20:56 | 2018-12-20 04:37:41 | 2018-12-20 05:17:41 | 0:40:00 | 0:19:03 | 0:20:57 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318124 | 2018-12-08 05:20:56 | 2018-12-20 04:46:16 | 2018-12-20 07:18:18 | 2:32:02 | 0:42:48 | 1:49:14 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
while scanning a plain scalar in "/tmp/teuth_ansible_failures_usS0e_", line 1, column 251 found unexpected ':' in "/tmp/teuth_ansible_failures_usS0e_", line 1, column 271 Please check http://pyyaml.org/wiki/YAMLColonInFlowContext for details. |
||||||||||||||
dead | 3318125 | 2018-12-08 05:20:57 | 2018-12-20 04:52:03 | 2018-12-20 16:54:14 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | — | |||||
dead | 3318126 | 2018-12-08 05:20:58 | 2018-12-20 05:00:04 | 2018-12-20 17:16:41 | 12:16:37 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||||
fail | 3318127 | 2018-12-08 05:20:58 | 2018-12-20 05:00:05 | 2018-12-20 06:38:05 | 1:38:00 | 1:15:07 | 0:22:53 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-20 05:40:23.578040 mon.a mon.0 158.69.73.7:6789/0 155 : cluster [WRN] Health check failed: 31 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318128 | 2018-12-08 05:20:59 | 2018-12-20 12:55:17 | 2018-12-20 22:09:25 | 9:14:08 | 0:25:33 | 8:48:35 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 3318129 | 2018-12-08 05:21:00 | 2018-12-20 13:00:37 | 2018-12-20 14:46:38 | 1:46:01 | 1:11:43 | 0:34:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3318130 | 2018-12-08 05:21:01 | 2018-12-20 13:02:43 | 2018-12-20 13:56:43 | 0:54:00 | 0:31:21 | 0:22:39 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318131 | 2018-12-08 05:21:01 | 2018-12-20 13:04:31 | 2018-12-20 23:48:41 | 10:44:10 | 0:33:06 | 10:11:04 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
dead | 3318132 | 2018-12-08 05:21:02 | 2018-12-20 13:08:22 | 2018-12-21 01:15:31 | 12:07:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
fail | 3318133 | 2018-12-08 05:21:03 | 2018-12-20 13:16:01 | 2018-12-20 15:44:03 | 2:28:02 | 1:31:58 | 0:56:04 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Found coredumps on ubuntu@ovh069.front.sepia.ceph.com |
||||||||||||||
dead | 3318134 | 2018-12-08 05:21:03 | 2018-12-20 13:16:11 | 2018-12-21 01:28:36 | 12:12:25 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
pass | 3318135 | 2018-12-08 05:21:04 | 2018-12-20 13:16:55 | 2018-12-20 15:26:56 | 2:10:01 | 0:53:22 | 1:16:39 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318136 | 2018-12-08 05:21:05 | 2018-12-20 13:26:18 | 2018-12-20 16:54:20 | 3:28:02 | 2:39:34 | 0:48:28 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 14:45:20.972257 mon.a mon.0 158.69.69.94:6789/0 152 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318137 | 2018-12-08 05:21:05 | 2018-12-20 13:26:55 | 2018-12-21 01:29:07 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 3318138 | 2018-12-08 05:21:06 | 2018-12-20 13:27:32 | 2018-12-20 14:31:32 | 1:04:00 | 0:36:39 | 0:27:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3318139 | 2018-12-08 05:21:07 | 2018-12-20 13:32:47 | 2018-12-20 15:08:47 | 1:36:00 | 1:02:25 | 0:33:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 3318140 | 2018-12-08 05:21:08 | 2018-12-20 13:35:18 | 2018-12-21 01:37:30 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | |||||
pass | 3318141 | 2018-12-08 05:21:08 | 2018-12-20 13:36:39 | 2018-12-20 15:24:40 | 1:48:01 | 1:12:00 | 0:36:01 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3318142 | 2018-12-08 05:21:09 | 2018-12-20 13:45:06 | 2018-12-20 14:37:06 | 0:52:00 | 0:30:51 | 0:21:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3318143 | 2018-12-08 05:21:10 | 2018-12-20 13:46:05 | 2018-12-20 19:38:09 | 5:52:04 | 0:37:52 | 5:14:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 3318144 | 2018-12-08 05:21:11 | 2018-12-20 13:48:27 | 2018-12-20 16:12:28 | 2:24:01 | 1:30:09 | 0:53:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318145 | 2018-12-08 05:21:11 | 2018-12-20 13:50:59 | 2018-12-20 15:23:00 | 1:32:01 | 0:21:10 | 1:10:51 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3318146 | 2018-12-08 05:21:12 | 2018-12-20 13:54:30 | 2018-12-20 20:42:36 | 6:48:06 | 0:46:28 | 6:01:38 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3318147 | 2018-12-08 05:21:13 | 2018-12-20 13:56:23 | 2018-12-20 16:24:24 | 2:28:01 | 0:36:06 | 1:51:55 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318148 | 2018-12-08 05:21:13 | 2018-12-20 13:56:44 | 2018-12-20 15:44:45 | 1:48:01 | 0:19:35 | 1:28:26 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3318149 | 2018-12-08 05:21:14 | 2018-12-20 14:00:17 | 2018-12-20 19:04:26 | 5:04:09 | 1:31:38 | 3:32:31 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
dead | 3318150 | 2018-12-08 05:21:15 | 2018-12-20 14:06:54 | 2018-12-21 02:09:07 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
pass | 3318151 | 2018-12-08 05:21:15 | 2018-12-20 14:19:07 | 2018-12-20 16:17:08 | 1:58:01 | 1:44:04 | 0:13:57 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 3318152 | 2018-12-08 05:21:16 | 2018-12-20 14:24:28 | 2018-12-20 16:16:29 | 1:52:01 | 1:21:02 | 0:30:59 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-20 15:10:40.848842 mon.b mon.0 158.69.69.85:6789/0 153 : cluster [WRN] Health check failed: 614 slow requests are blocked > 32 sec. Implicated osds 0,1,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318153 | 2018-12-08 05:21:17 | 2018-12-20 14:26:18 | 2018-12-21 02:28:30 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | — | |||||
fail | 3318154 | 2018-12-08 05:21:18 | 2018-12-20 14:30:18 | 2018-12-20 17:48:20 | 3:18:02 | 2:02:35 | 1:15:27 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
Failure Reason:
"2018-12-20 16:18:53.738465 mon.a mon.0 158.69.65.161:6789/0 184 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log |
||||||||||||||
pass | 3318155 | 2018-12-08 05:21:18 | 2018-12-20 14:31:45 | 2018-12-20 16:39:46 | 2:08:01 | 0:54:57 | 1:13:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 3318156 | 2018-12-08 05:21:19 | 2018-12-20 14:37:10 | 2018-12-21 02:39:22 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
pass | 3318157 | 2018-12-08 05:21:20 | 2018-12-20 14:45:06 | 2018-12-20 17:33:07 | 2:48:01 | 1:55:50 | 0:52:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 3318158 | 2018-12-08 05:21:21 | 2018-12-20 14:46:41 | 2018-12-20 16:44:42 | 1:58:01 | 0:50:38 | 1:07:23 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2a4bf731c2a02f4afa4da597c4954437d14a87 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
pass | 3318159 | 2018-12-08 05:21:21 | 2018-12-20 14:51:07 | 2018-12-20 19:23:10 | 4:32:03 | 0:44:13 | 3:47:50 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 3318160 | 2018-12-08 05:21:22 | 2018-12-20 14:56:31 | 2018-12-20 17:26:33 | 2:30:02 | 0:46:33 | 1:43:29 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318161 | 2018-12-08 05:21:23 | 2018-12-20 14:59:11 | 2018-12-20 17:33:12 | 2:34:01 | 1:21:20 | 1:12:41 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 16:28:29.623738 mon.a mon.0 158.69.68.92:6789/0 145 : cluster [WRN] Health check failed: 3 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318162 | 2018-12-08 05:21:23 | 2018-12-20 15:04:00 | 2018-12-20 21:52:06 | 6:48:06 | 0:34:15 | 6:13:51 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3318163 | 2018-12-08 05:21:24 | 2018-12-20 15:04:20 | 2018-12-20 16:44:21 | 1:40:01 | 1:00:29 | 0:39:32 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
fail | 3318164 | 2018-12-08 05:21:25 | 2018-12-20 15:06:09 | 2018-12-20 19:34:12 | 4:28:03 | 3:33:14 | 0:54:49 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
Failure Reason:
"2018-12-20 16:49:01.706782 mon.b mon.0 158.69.68.14:6789/0 253 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318165 | 2018-12-08 05:21:26 | 2018-12-20 15:08:49 | 2018-12-21 02:15:02 | 11:06:13 | 0:25:25 | 10:40:48 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 3318166 | 2018-12-08 05:21:26 | 2018-12-20 15:23:13 | 2018-12-20 17:41:14 | 2:18:01 | 2:02:31 | 0:15:30 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318167 | 2018-12-08 05:21:27 | 2018-12-20 15:24:47 | 2018-12-20 16:24:46 | 0:59:59 | 0:43:01 | 0:16:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3318168 | 2018-12-08 05:21:28 | 2018-12-20 15:27:05 | 2018-12-20 23:35:12 | 8:08:07 | 1:33:09 | 6:34:58 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 3318169 | 2018-12-08 05:21:28 | 2018-12-20 15:32:04 | 2018-12-20 18:56:07 | 3:24:03 | 1:40:19 | 1:43:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-20 18:09:04.871560 mon.b mon.0 158.69.68.90:6789/0 147 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318170 | 2018-12-08 05:21:29 | 2018-12-20 15:44:11 | 2018-12-20 17:02:11 | 1:18:00 | 0:19:39 | 0:58:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3318171 | 2018-12-08 05:21:30 | 2018-12-20 15:44:47 | 2018-12-20 20:44:51 | 5:00:04 | 0:53:54 | 4:06:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | |||
pass | 3318172 | 2018-12-08 05:21:31 | 2018-12-20 15:51:50 | 2018-12-20 17:45:51 | 1:54:01 | 0:31:50 | 1:22:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318173 | 2018-12-08 05:21:31 | 2018-12-20 15:54:56 | 2018-12-20 16:58:56 | 1:04:00 | 0:24:35 | 0:39:25 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3318174 | 2018-12-08 05:21:32 | 2018-12-20 16:12:40 | 2018-12-20 20:02:43 | 3:50:03 | 1:45:46 | 2:04:17 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 3318175 | 2018-12-08 05:21:33 | 2018-12-20 16:16:31 | 2018-12-20 21:26:35 | 5:10:04 | 0:23:12 | 4:46:52 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
fail | 3318176 | 2018-12-08 05:21:33 | 2018-12-20 16:17:09 | 2018-12-20 17:43:10 | 1:26:01 | 1:07:13 | 0:18:48 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 17:07:25.257299 mon.a mon.0 158.69.67.89:6789/0 2708 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 3318177 | 2018-12-08 05:21:34 | 2018-12-20 16:24:37 | 2018-12-20 19:28:39 | 3:04:02 | 0:58:07 | 2:05:55 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3318178 | 2018-12-08 05:21:35 | 2018-12-20 16:24:48 | 2018-12-21 04:27:05 | 12:02:17 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | — | |||||
pass | 3318179 | 2018-12-08 05:21:36 | 2018-12-20 16:30:05 | 2018-12-20 18:54:06 | 2:24:01 | 1:13:38 | 1:10:23 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3318180 | 2018-12-08 05:21:37 | 2018-12-20 16:39:58 | 2018-12-20 20:40:01 | 4:00:03 | 0:46:44 | 3:13:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318181 | 2018-12-08 05:21:37 | 2018-12-20 16:44:23 | 2018-12-20 22:14:30 | 5:30:07 | 0:37:55 | 4:52:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 3318182 | 2018-12-08 05:21:38 | 2018-12-20 16:44:43 | 2018-12-20 18:36:44 | 1:52:01 | 1:32:21 | 0:19:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 3318183 | 2018-12-08 05:21:39 | 2018-12-20 16:54:26 | 2018-12-20 21:08:29 | 4:14:03 | 3:52:36 | 0:21:27 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/iozone.sh) on ovh076 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2a4bf731c2a02f4afa4da597c4954437d14a87 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' |
||||||||||||||
dead | 3318184 | 2018-12-08 05:21:40 | 2018-12-20 16:54:26 | 2018-12-21 04:56:41 | 12:02:15 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | — | |||||
pass | 3318185 | 2018-12-08 05:21:40 | 2018-12-20 16:58:58 | 2018-12-20 20:31:00 | 3:32:02 | 0:48:48 | 2:43:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318186 | 2018-12-08 05:21:41 | 2018-12-20 17:02:23 | 2018-12-20 19:26:25 | 2:24:02 | 1:09:43 | 1:14:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 18:48:07.242641 mon.a mon.0 158.69.65.183:6789/0 232 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318187 | 2018-12-08 05:21:42 | 2018-12-20 17:16:57 | 2018-12-21 05:19:10 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 3318188 | 2018-12-08 05:21:43 | 2018-12-20 17:26:22 | 2018-12-20 18:16:22 | 0:50:00 | 0:34:30 | 0:15:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3318189 | 2018-12-08 05:21:44 | 2018-12-20 17:26:34 | 2018-12-20 19:52:35 | 2:26:01 | 1:45:54 | 0:40:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3318190 | 2018-12-08 05:21:44 | 2018-12-20 17:33:10 | 2018-12-20 19:25:10 | 1:52:00 | 0:36:16 | 1:15:44 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 3318191 | 2018-12-08 05:21:45 | 2018-12-20 17:33:13 | 2018-12-20 18:45:13 | 1:12:00 | 0:46:28 | 0:25:32 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3318192 | 2018-12-08 05:21:46 | 2018-12-20 17:36:52 | 2018-12-20 19:24:53 | 1:48:01 | 0:18:28 | 1:29:33 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3318193 | 2018-12-08 05:21:47 | 2018-12-20 17:41:27 | 2018-12-21 01:33:34 | 7:52:07 | 0:47:43 | 7:04:24 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 3318194 | 2018-12-08 05:21:47 | 2018-12-20 17:43:21 | 2018-12-20 20:37:23 | 2:54:02 | 2:13:12 | 0:40:50 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-20 18:46:14.061404 mon.b mon.0 158.69.64.51:6789/0 142 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318195 | 2018-12-08 05:21:48 | 2018-12-20 17:44:34 | 2018-12-20 18:52:34 | 1:08:00 | 0:35:20 | 0:32:40 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3318196 | 2018-12-08 05:21:49 | 2018-12-20 17:46:03 | 2018-12-21 01:44:10 | 7:58:07 | 0:38:31 | 7:19:36 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3318197 | 2018-12-08 05:21:50 | 2018-12-20 17:48:16 | 2018-12-20 19:10:16 | 1:22:00 | 0:15:52 | 1:06:08 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318198 | 2018-12-08 05:21:50 | 2018-12-20 17:48:21 | 2018-12-20 19:36:22 | 1:48:01 | 0:21:16 | 1:26:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 3318199 | 2018-12-08 05:21:51 | 2018-12-20 18:16:48 | 2018-12-20 21:24:49 | 3:08:01 | 1:23:47 | 1:44:14 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
dead | 3318200 | 2018-12-08 05:21:52 | 2018-12-20 18:36:55 | 2018-12-21 06:39:06 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
pass | 3318201 | 2018-12-08 05:21:53 | 2018-12-20 18:37:33 | 2018-12-20 21:51:34 | 3:14:01 | 2:41:36 | 0:32:25 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318202 | 2018-12-08 05:21:53 | 2018-12-20 18:42:40 | 2018-12-20 21:30:41 | 2:48:01 | 1:45:40 | 1:02:21 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3318203 | 2018-12-08 05:21:54 | 2018-12-20 18:45:18 | 2018-12-21 06:47:31 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | — | |||||
dead | 3318204 | 2018-12-08 05:21:55 | 2018-12-20 18:52:52 | 2018-12-21 07:00:01 | 12:07:09 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||||
pass | 3318205 | 2018-12-08 05:21:55 | 2018-12-20 18:54:25 | 2018-12-20 20:00:25 | 1:06:00 | 0:27:15 | 0:38:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318206 | 2018-12-08 05:21:56 | 2018-12-20 18:56:25 | 2018-12-21 02:22:31 | 7:26:06 | 0:33:10 | 6:52:56 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 3318207 | 2018-12-08 05:21:57 | 2018-12-20 19:04:31 | 2018-12-20 20:56:32 | 1:52:01 | 1:33:01 | 0:19:00 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3318208 | 2018-12-08 05:21:58 | 2018-12-20 19:10:22 | 2018-12-20 21:00:22 | 1:50:00 | 1:06:02 | 0:43:58 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
dead | 3318209 | 2018-12-08 05:21:58 | 2018-12-20 19:23:28 | 2018-12-21 07:25:41 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | — | |||||
pass | 3318210 | 2018-12-08 05:21:59 | 2018-12-20 19:24:58 | 2018-12-20 20:52:59 | 1:28:01 | 1:13:01 | 0:15:00 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318211 | 2018-12-08 05:22:00 | 2018-12-20 19:25:13 | 2018-12-20 20:43:13 | 1:18:00 | 1:03:19 | 0:14:41 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 20:21:14.491291 mon.a mon.0 158.69.70.10:6789/0 156 : cluster [WRN] Health check failed: 86 slow requests are blocked > 32 sec. Implicated osds 0,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318212 | 2018-12-08 05:22:01 | 2018-12-20 19:26:42 | 2018-12-21 01:28:47 | 6:02:05 | 0:23:53 | 5:38:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
fail | 3318213 | 2018-12-08 05:22:01 | 2018-12-20 19:28:44 | 2018-12-20 21:08:44 | 1:40:00 | 1:18:49 | 0:21:11 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
Failure Reason:
"2018-12-20 20:17:12.581419 mon.b mon.1 158.69.71.119:6789/0 113 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log |
||||||||||||||
pass | 3318214 | 2018-12-08 05:22:02 | 2018-12-20 19:34:30 | 2018-12-20 22:10:31 | 2:36:01 | 0:49:26 | 1:46:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3318215 | 2018-12-08 05:22:03 | 2018-12-20 19:36:27 | 2018-12-20 22:40:29 | 3:04:02 | 0:26:11 | 2:37:51 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
fail | 3318216 | 2018-12-08 05:22:04 | 2018-12-20 19:38:41 | 2018-12-20 22:58:42 | 3:20:01 | 2:00:51 | 1:19:10 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Found coredumps on ubuntu@ovh083.front.sepia.ceph.com |
||||||||||||||
pass | 3318217 | 2018-12-08 05:22:05 | 2018-12-20 19:52:51 | 2018-12-20 21:38:51 | 1:46:00 | 0:28:32 | 1:17:28 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3318218 | 2018-12-08 05:22:05 | 2018-12-20 20:00:37 | 2018-12-21 04:02:44 | 8:02:07 | 1:07:53 | 6:54:14 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 3318219 | 2018-12-08 05:22:06 | 2018-12-20 20:02:57 | 2018-12-20 22:20:58 | 2:18:01 | 1:27:41 | 0:50:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-20 21:29:58.346342 mon.b mon.0 158.69.65.40:6789/0 146 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318220 | 2018-12-08 05:22:07 | 2018-12-20 20:31:05 | 2018-12-20 21:19:05 | 0:48:00 | 0:18:13 | 0:29:47 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 3318221 | 2018-12-08 05:22:08 | 2018-12-20 20:37:28 | 2018-12-21 08:39:50 | 12:02:22 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 3318222 | 2018-12-08 05:22:09 | 2018-12-20 20:40:07 | 2018-12-20 21:24:06 | 0:43:59 | 0:25:15 | 0:18:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318223 | 2018-12-08 05:22:09 | 2018-12-20 20:42:53 | 2018-12-20 21:36:55 | 0:54:02 | 0:29:56 | 0:24:06 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318224 | 2018-12-08 05:22:10 | 2018-12-20 20:43:18 | 2018-12-21 00:07:19 | 3:24:01 | 2:07:03 | 1:16:58 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-12-20 22:38:35.365555 mon.a mon.0 158.69.66.72:6789/0 156 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318225 | 2018-12-08 05:22:11 | 2018-12-20 20:44:56 | 2018-12-20 23:02:57 | 2:18:01 | 0:40:32 | 1:37:29 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
dead | 3318226 | 2018-12-08 05:22:12 | 2018-12-20 20:53:21 | 2018-12-21 09:00:33 | 12:07:12 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||||
pass | 3318227 | 2018-12-08 05:22:12 | 2018-12-20 20:57:04 | 2018-12-20 22:19:02 | 1:21:58 | 0:54:57 | 0:27:01 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3318228 | 2018-12-08 05:22:13 | 2018-12-20 21:00:29 | 2018-12-21 09:02:39 | 12:02:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | — | |||||
dead | 3318229 | 2018-12-08 05:22:14 | 2018-12-20 21:08:36 | 2018-12-21 09:15:48 | 12:07:12 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||||
pass | 3318230 | 2018-12-08 05:22:14 | 2018-12-20 21:08:46 | 2018-12-20 21:54:46 | 0:46:00 | 0:27:23 | 0:18:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318231 | 2018-12-08 05:22:15 | 2018-12-20 21:19:24 | 2018-12-21 03:47:28 | 6:28:04 | 0:43:14 | 5:44:50 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
dead | 3318232 | 2018-12-08 05:22:16 | 2018-12-20 21:24:14 | 2018-12-21 09:31:21 | 12:07:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
pass | 3318233 | 2018-12-08 05:22:17 | 2018-12-20 21:25:10 | 2018-12-21 00:19:13 | 2:54:03 | 2:03:00 | 0:51:03 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3318234 | 2018-12-08 05:22:17 | 2018-12-20 21:26:56 | 2018-12-21 09:29:09 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | — | |||||
fail | 3318235 | 2018-12-08 05:22:18 | 2018-12-20 21:30:56 | 2018-12-20 23:30:57 | 2:00:01 | 1:30:09 | 0:29:52 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
Failure Reason:
"2018-12-20 22:41:49.964017 mon.b mon.0 158.69.68.206:6789/0 153 : cluster [WRN] Health check failed: 22 slow requests are blocked > 32 sec. Implicated osds 0,2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
fail | 3318236 | 2018-12-08 05:22:19 | 2018-12-20 21:36:58 | 2018-12-20 23:14:58 | 1:38:00 | 1:21:44 | 0:16:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 22:14:57.566135 mon.b mon.0 158.69.69.202:6789/0 151 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318237 | 2018-12-08 05:22:20 | 2018-12-20 21:39:05 | 2018-12-21 09:41:16 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 3318238 | 2018-12-08 05:22:21 | 2018-12-20 21:51:42 | 2018-12-20 23:05:41 | 1:13:59 | 0:39:21 | 0:34:38 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 3318239 | 2018-12-08 05:22:21 | 2018-12-20 21:52:13 | 2018-12-20 23:20:12 | 1:27:59 | 0:54:45 | 0:33:14 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 3318240 | 2018-12-08 05:22:22 | 2018-12-20 21:55:06 | 2018-12-21 01:29:08 | 3:34:02 | 0:33:08 | 3:00:54 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
fail | 3318241 | 2018-12-08 05:22:23 | 2018-12-20 22:09:32 | 2018-12-21 00:41:33 | 2:32:01 | 1:11:54 | 1:20:07 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-20 23:55:21.790579 mon.a mon.0 158.69.65.253:6789/0 135 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 3318242 | 2018-12-08 05:22:24 | 2018-12-20 22:10:45 | 2018-12-21 00:04:49 | 1:54:04 | 0:17:41 | 1:36:23 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 3318243 | 2018-12-08 05:22:24 | 2018-12-20 22:14:35 | 2018-12-21 04:48:41 | 6:34:06 | 0:38:01 | 5:56:05 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 3318244 | 2018-12-08 05:22:25 | 2018-12-20 22:19:07 | 2018-12-21 00:31:08 | 2:12:01 | 1:25:27 | 0:46:34 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-20 23:49:00.043481 mon.a mon.0 158.69.64.79:6789/0 246 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318245 | 2018-12-08 05:22:26 | 2018-12-20 22:21:06 | 2018-12-21 00:09:05 | 1:47:59 | 0:17:58 | 1:30:01 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 3318246 | 2018-12-08 05:22:27 | 2018-12-20 22:40:36 | 2018-12-21 03:14:40 | 4:34:04 | 0:27:32 | 4:06:32 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 3318247 | 2018-12-08 05:22:28 | 2018-12-20 22:58:48 | 2018-12-20 23:36:47 | 0:37:59 | 0:16:00 | 0:21:59 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318248 | 2018-12-08 05:22:28 | 2018-12-20 23:03:11 | 2018-12-21 01:51:12 | 2:48:01 | 0:18:57 | 2:29:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318249 | 2018-12-08 05:22:29 | 2018-12-20 23:05:43 | 2018-12-21 01:37:44 | 2:32:01 | 1:29:38 | 1:02:23 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
"2018-12-21 00:59:58.438173 mon.a mon.0 158.69.65.174:6789/0 257 : cluster [ERR] Health check failed: mon b is very low on available space (MON_DISK_CRIT)" in cluster log |
||||||||||||||
dead | 3318250 | 2018-12-08 05:22:30 | 2018-12-20 23:15:05 | 2018-12-21 11:17:17 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
fail | 3318251 | 2018-12-08 05:22:31 | 2018-12-20 23:20:15 | 2018-12-21 01:32:16 | 2:12:01 | 1:55:21 | 0:16:40 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-21 00:23:42.922023 mon.b mon.0 158.69.69.148:6789/0 3779 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 3318252 | 2018-12-08 05:22:32 | 2018-12-20 23:31:14 | 2018-12-21 02:23:16 | 2:52:02 | 1:49:36 | 1:02:26 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-21 00:58:07.964167 mon.b mon.1 158.69.74.142:6789/0 30 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log |
||||||||||||||
dead | 3318253 | 2018-12-08 05:22:33 | 2018-12-20 23:35:16 | 2018-12-21 11:42:37 | 12:07:21 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
fail | 3318254 | 2018-12-08 05:22:34 | 2018-12-20 23:36:49 | 2018-12-21 02:00:51 | 2:24:02 | 1:29:55 | 0:54:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
Failure Reason:
"2018-12-21 01:29:10.927327 mon.b mon.0 158.69.66.123:6789/0 261 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318255 | 2018-12-08 05:22:35 | 2018-12-20 23:48:45 | 2018-12-21 01:24:45 | 1:36:00 | 0:43:26 | 0:52:34 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318256 | 2018-12-08 05:22:35 | 2018-12-21 00:05:02 | 2018-12-21 03:25:04 | 3:20:02 | 0:36:33 | 2:43:29 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
dead | 3318257 | 2018-12-08 05:22:36 | 2018-12-21 00:07:22 | 2018-12-21 12:14:56 | 12:07:34 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||||
fail | 3318258 | 2018-12-08 05:22:37 | 2018-12-21 00:09:18 | 2018-12-21 02:41:19 | 2:32:01 | 2:04:03 | 0:27:58 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-21 01:32:44.525135 mon.a mon.0 158.69.64.217:6789/0 888 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
dead | 3318259 | 2018-12-08 05:22:38 | 2018-12-21 00:19:26 | 2018-12-21 12:21:36 | 12:02:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | — | |||||
pass | 3318260 | 2018-12-08 05:22:39 | 2018-12-21 00:31:11 | 2018-12-21 03:13:12 | 2:42:01 | 0:47:48 | 1:54:13 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 3318261 | 2018-12-08 05:22:40 | 2018-12-21 00:41:47 | 2018-12-21 04:21:49 | 3:40:02 | 1:19:46 | 2:20:16 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-21 03:31:13.436858 mon.a mon.0 158.69.69.56:6789/0 259 : cluster [WRN] Health check failed: 539 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318262 | 2018-12-08 05:22:40 | 2018-12-21 01:15:40 | 2018-12-21 13:17:52 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | — | |||||
pass | 3318263 | 2018-12-08 05:22:41 | 2018-12-21 01:24:47 | 2018-12-21 02:22:47 | 0:58:00 | 0:34:15 | 0:23:45 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
dead | 3318264 | 2018-12-08 05:22:42 | 2018-12-21 01:28:39 | 2018-12-21 13:45:14 | 12:16:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||||
dead | 3318265 | 2018-12-08 05:22:43 | 2018-12-21 01:28:50 | 2018-12-21 13:31:01 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | — | |||||
fail | 3318266 | 2018-12-08 05:22:44 | 2018-12-21 01:29:19 | 2018-12-21 06:05:22 | 4:36:03 | 3:35:12 | 1:00:51 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/iozone.sh) on ovh035 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2a4bf731c2a02f4afa4da597c4954437d14a87 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' |
||||||||||||||
pass | 3318267 | 2018-12-08 05:22:44 | 2018-12-21 01:29:19 | 2018-12-21 03:49:20 | 2:20:01 | 0:19:31 | 2:00:30 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
fail | 3318268 | 2018-12-08 05:22:45 | 2018-12-21 01:32:18 | 2018-12-21 13:32:28 | 12:00:10 | 1:05:52 | 10:54:18 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2018-12-21 13:07:14.837463 mon.b mon.0 158.69.69.167:6789/0 999 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
fail | 3318269 | 2018-12-08 05:22:46 | 2018-12-21 01:33:37 | 2018-12-21 04:19:39 | 2:46:02 | 1:44:14 | 1:01:48 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-21 03:08:59.153971 mon.a mon.0 158.69.65.146:6789/0 154 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318270 | 2018-12-08 05:22:47 | 2018-12-21 01:37:42 | 2018-12-21 03:43:43 | 2:06:01 | 0:29:35 | 1:36:26 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 3318271 | 2018-12-08 05:22:47 | 2018-12-21 01:37:46 | 2018-12-21 13:40:02 | 12:02:16 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 3318272 | 2018-12-08 05:22:48 | 2018-12-21 01:44:14 | 2018-12-21 04:08:15 | 2:24:01 | 0:21:15 | 2:02:46 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318273 | 2018-12-08 05:22:49 | 2018-12-21 01:51:14 | 2018-12-21 03:17:15 | 1:26:01 | 0:18:49 | 1:07:12 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318274 | 2018-12-08 05:22:50 | 2018-12-21 02:00:53 | 2018-12-21 05:28:55 | 3:28:02 | 2:06:17 | 1:21:45 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
Failure Reason:
"2018-12-21 03:40:11.637036 mon.a mon.0 158.69.65.170:6789/0 152 : cluster [WRN] Health check failed: 2 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318275 | 2018-12-08 05:22:51 | 2018-12-21 02:09:09 | 2018-12-21 14:25:46 | 12:16:37 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||||
fail | 3318276 | 2018-12-08 05:22:52 | 2018-12-21 02:15:16 | 2018-12-21 04:53:17 | 2:38:01 | 0:49:34 | 1:48:27 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-21 04:29:42.089594 mon.b mon.0 158.69.70.34:6789/0 2670 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 3318277 | 2018-12-08 05:22:53 | 2018-12-21 02:22:46 | 2018-12-21 05:32:48 | 3:10:02 | 1:16:16 | 1:53:46 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
"2018-12-21 04:38:09.229545 mon.b mon.0 158.69.68.93:6789/0 150 : cluster [WRN] Health check failed: 175 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318278 | 2018-12-08 05:22:54 | 2018-12-21 02:22:48 | 2018-12-21 14:25:00 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | — | |||||
fail | 3318279 | 2018-12-08 05:22:55 | 2018-12-21 02:23:17 | 2018-12-21 04:29:19 | 2:06:02 | 1:43:04 | 0:22:58 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
Failure Reason:
"2018-12-21 03:48:50.010166 mon.b mon.0 158.69.64.67:6789/0 315 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318280 | 2018-12-08 05:22:56 | 2018-12-21 04:29:36 | 2018-12-21 05:43:36 | 1:14:00 | 0:28:42 | 0:45:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 3318281 | 2018-12-08 05:22:57 | 2018-12-21 04:48:43 | 2018-12-21 16:50:53 | 12:02:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 3318282 | 2018-12-08 05:22:58 | 2018-12-21 04:53:29 | 2018-12-21 07:05:30 | 2:12:01 | 1:25:07 | 0:46:54 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 3318283 | 2018-12-08 05:22:59 | 2018-12-21 04:56:43 | 2018-12-21 07:06:44 | 2:10:01 | 1:39:06 | 0:30:55 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
dead | 3318284 | 2018-12-08 05:22:59 | 2018-12-21 09:00:39 | 2018-12-21 21:12:51 | 12:12:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||||
fail | 3318285 | 2018-12-08 05:23:00 | 2018-12-21 09:01:56 | 2018-12-21 10:53:57 | 1:52:01 | 1:06:43 | 0:45:18 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
Failure Reason:
"2018-12-21 10:30:43.380242 mon.b mon.0 158.69.69.250:6789/0 171 : cluster [WRN] Health check failed: Degraded data redundancy: 9/650 objects degraded (1.385%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
fail | 3318286 | 2018-12-08 05:23:01 | 2018-12-21 09:02:40 | 2018-12-21 14:18:50 | 5:16:10 | 4:21:06 | 0:55:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on ovh069 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2a4bf731c2a02f4afa4da597c4954437d14a87 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
dead | 3318287 | 2018-12-08 05:23:02 | 2018-12-21 09:09:12 | 2018-12-21 21:11:22 | 12:02:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 3318288 | 2018-12-08 05:23:03 | 2018-12-21 09:09:28 | 2018-12-21 10:55:29 | 1:46:01 | 0:54:24 | 0:51:37 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
fail | 3318289 | 2018-12-08 05:23:04 | 2018-12-21 09:09:38 | 2018-12-21 12:23:40 | 3:14:02 | 1:44:19 | 1:29:43 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
Failure Reason:
"2018-12-21 11:51:08.544967 mon.b mon.0 158.69.71.194:6789/0 234 : cluster [WRN] Health check failed: Degraded data redundancy: 747/8176 objects degraded (9.136%), 3 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
dead | 3318290 | 2018-12-08 05:23:05 | 2018-12-21 09:10:09 | 2018-12-21 21:12:21 | 12:02:12 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | |||||
pass | 3318291 | 2018-12-08 05:23:06 | 2018-12-21 09:15:52 | 2018-12-21 10:25:53 | 1:10:01 | 0:51:03 | 0:18:58 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 3318292 | 2018-12-08 05:23:06 | 2018-12-21 09:22:31 | 2018-12-21 10:24:31 | 1:02:00 | 0:35:16 | 0:26:44 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
dead | 3318293 | 2018-12-08 05:23:12 | 2018-12-21 09:26:08 | 2018-12-21 21:28:18 | 12:02:10 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | — | |||||
fail | 3318294 | 2018-12-08 05:23:13 | 2018-12-21 09:27:04 | 2018-12-21 11:51:05 | 2:24:01 | 2:00:05 | 0:23:56 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-21 10:38:00.814502 mon.b mon.0 158.69.70.21:6789/0 221 : cluster [WRN] Health check failed: 3 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318295 | 2018-12-08 05:23:14 | 2018-12-21 09:29:11 | 2018-12-21 10:27:11 | 0:58:00 | 0:35:53 | 0:22:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 3318296 | 2018-12-08 05:23:15 | 2018-12-21 09:31:23 | 2018-12-21 21:33:34 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | — | |||||
pass | 3318297 | 2018-12-08 05:23:16 | 2018-12-21 09:41:18 | 2018-12-21 10:37:18 | 0:56:00 | 0:31:53 | 0:24:07 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 3318298 | 2018-12-08 05:23:16 | 2018-12-21 09:42:16 | 2018-12-21 11:16:16 | 1:34:00 | 0:28:04 | 1:05:56 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 3318299 | 2018-12-08 05:23:17 | 2018-12-21 09:43:01 | 2018-12-21 12:13:02 | 2:30:01 | 1:05:24 | 1:24:37 | ovh | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
"2018-12-21 11:27:47.743039 mon.b mon.0 158.69.67.191:6789/0 217 : cluster [WRN] Health check failed: 78 slow requests are blocked > 32 sec. Implicated osds 2 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318300 | 2018-12-08 05:23:18 | 2018-12-21 09:45:26 | 2018-12-21 21:47:37 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
fail | 3318301 | 2018-12-08 05:23:19 | 2018-12-21 09:47:58 | 2018-12-21 12:09:59 | 2:22:01 | 1:56:18 | 0:25:43 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
Socket is closed |
||||||||||||||
pass | 3318302 | 2018-12-08 05:23:20 | 2018-12-21 09:50:45 | 2018-12-21 12:02:47 | 2:12:02 | 1:00:00 | 1:12:02 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
dead | 3318303 | 2018-12-08 05:23:21 | 2018-12-21 09:58:15 | 2018-12-21 22:05:36 | 12:07:21 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||||
pass | 3318304 | 2018-12-08 05:23:21 | 2018-12-21 10:21:16 | 2018-12-21 12:37:17 | 2:16:01 | 1:11:11 | 1:04:50 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 3318305 | 2018-12-08 05:23:22 | 2018-12-21 10:24:14 | 2018-12-21 11:08:14 | 0:44:00 | 0:28:40 | 0:15:20 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 3318306 | 2018-12-08 05:23:23 | 2018-12-21 10:24:33 | 2018-12-21 13:36:35 | 3:12:02 | 0:39:13 | 2:32:49 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
fail | 3318307 | 2018-12-08 05:23:24 | 2018-12-21 10:26:06 | 2018-12-21 12:18:07 | 1:52:01 | 1:26:57 | 0:25:04 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
Failure Reason:
"2018-12-21 11:33:27.012991 mon.b mon.0 158.69.66.215:6789/0 332 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log |
||||||||||||||
pass | 3318308 | 2018-12-08 05:23:25 | 2018-12-21 10:27:23 | 2018-12-21 11:35:23 | 1:08:00 | 0:46:21 | 0:21:39 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
dead | 3318309 | 2018-12-08 05:23:26 | 2018-12-21 10:30:16 | 2018-12-21 22:32:27 | 12:02:11 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | — | |||||
fail | 3318310 | 2018-12-08 05:23:27 | 2018-12-21 10:35:11 | 2018-12-21 12:17:12 | 1:42:01 | 0:59:58 | 0:42:03 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
Failure Reason:
"2018-12-21 11:33:27.550955 mon.b mon.0 158.69.69.251:6789/0 146 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log |
||||||||||||||
fail | 3318311 | 2018-12-08 05:23:27 | 2018-12-21 10:37:20 | 2018-12-21 13:13:22 | 2:36:02 | 1:02:39 | 1:33:23 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2018-12-21 12:58:07.676685 mon.b mon.0 158.69.70.34:6789/0 141 : cluster [WRN] Health check failed: 40 slow requests are blocked > 32 sec. Implicated osds 4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318312 | 2018-12-08 05:23:28 | 2018-12-21 10:45:48 | 2018-12-21 12:59:49 | 2:14:01 | 0:34:59 | 1:39:02 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 3318313 | 2018-12-08 05:23:29 | 2018-12-21 10:45:52 | 2018-12-21 12:03:52 | 1:18:00 | 0:44:43 | 0:33:17 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
fail | 3318314 | 2018-12-08 05:23:30 | 2018-12-21 10:53:59 | 2018-12-21 13:54:06 | 3:00:07 | 2:41:48 | 0:18:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
Failure Reason:
"2018-12-21 11:33:20.051302 mon.b mon.0 158.69.65.76:6789/0 153 : cluster [WRN] Health check failed: 24 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 3318315 | 2018-12-08 05:23:31 | 2018-12-21 10:55:41 | 2018-12-21 22:57:54 | 12:02:13 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | — | |||||
pass | 3318316 | 2018-12-08 05:23:32 | 2018-12-21 10:57:26 | 2018-12-21 12:47:28 | 1:50:02 | 1:38:24 | 0:11:38 | ovh | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 3318317 | 2018-12-08 05:23:33 | 2018-12-21 11:08:27 | 2018-12-21 12:38:27 | 1:30:00 | 0:30:04 | 0:59:56 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
fail | 3318318 | 2018-12-08 05:23:34 | 2018-12-21 11:12:15 | 2018-12-21 14:06:16 | 2:54:01 | 0:54:41 | 1:59:20 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
Test failure: test_ops_throttle (tasks.cephfs.test_strays.TestStrays) |
||||||||||||||
fail | 3318319 | 2018-12-08 05:23:34 | 2018-12-21 11:16:29 | 2018-12-21 13:52:31 | 2:36:02 | 1:43:27 | 0:52:35 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2018-12-21 12:48:46.361902 mon.b mon.0 158.69.73.250:6789/0 149 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 0 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 3318320 | 2018-12-08 05:23:35 | 2018-12-21 11:17:18 | 2018-12-21 12:45:18 | 1:28:00 | 0:32:41 | 0:55:19 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 3318321 | 2018-12-08 05:23:36 | 2018-12-21 11:22:00 | 2018-12-21 23:24:17 | 12:02:17 | ovh | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 3318322 | 2018-12-08 05:23:37 | 2018-12-21 11:35:26 | 2018-12-21 13:01:27 | 1:26:01 | 0:18:12 | 1:07:49 | ovh | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 |