| User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Fail | Dead |
|---|---|---|---|---|---|---|---|---|---|---|
| yuriw | 2019-01-17 23:11:03 | 2019-01-18 01:03:52 | 2019-01-18 13:26:32 | 12:22:40 | kcephfs | wip-yuri6-testing-2019-01-17-2056-luminous | smithi | a46b5be | 59 | 7 |
| Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| fail | 3475482 |
|
2019-01-17 23:11:09 | 2019-01-18 00:58:09 | 2019-01-18 01:18:08 | 0:19:59 | 0:09:07 | 0:10:52 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:08:00.020531+00:00 smithi028 kernel: [ 40.750305] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475484 |
|
2019-01-17 23:11:09 | 2019-01-18 00:58:09 | 2019-01-18 02:22:09 | 1:24:00 | 0:43:40 | 0:40:20 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:38:00.019583+00:00 smithi151 kernel: [ 72.560803] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475486 |
|
2019-01-17 23:11:10 | 2019-01-18 00:58:11 | 2019-01-18 01:46:11 | 0:48:00 | 0:12:30 | 0:35:30 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:33:00.020405+00:00 smithi067 kernel: [ 32.558299] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475488 |
|
2019-01-17 23:11:11 | 2019-01-18 00:59:56 | 2019-01-18 01:53:56 | 0:54:00 | 0:26:20 | 0:27:40 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:26:30.019248+00:00 smithi102 kernel: [ 62.761730] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475490 |
|
2019-01-17 23:11:12 | 2019-01-18 00:59:57 | 2019-01-18 02:33:57 | 1:34:00 | 0:26:57 | 1:07:03 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | ||
Failure Reason:
"2019-01-18 02:13:54.182664 mon.b mon.0 172.21.15.61:6789/0 148 : cluster [WRN] Health check failed: 43 slow requests are blocked > 32 sec. Implicated osds 6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
| fail | 3475492 |
|
2019-01-17 23:11:12 | 2019-01-18 01:00:19 | 2019-01-18 06:20:29 | 5:20:10 | 0:18:24 | 5:01:46 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T06:01:30.019959+00:00 smithi001 kernel: [ 53.741599] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475494 |
|
2019-01-17 23:11:13 | 2019-01-18 01:02:06 | 2019-01-18 01:52:06 | 0:50:00 | 0:24:43 | 0:25:17 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:26:00.020241+00:00 smithi137 kernel: [ 23.750188] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475496 |
|
2019-01-17 23:11:14 | 2019-01-18 01:02:06 | 2019-01-18 02:34:07 | 1:32:01 | 0:10:28 | 1:21:33 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:21:40.019952+00:00 smithi044 kernel: [ 69.692172] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475498 |
|
2019-01-17 23:11:15 | 2019-01-18 01:02:06 | 2019-01-18 02:04:06 | 1:02:00 | 0:27:02 | 0:34:58 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:35:10.020165+00:00 smithi081 kernel: [ 51.762390] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475500 |
|
2019-01-17 23:11:15 | 2019-01-18 01:02:12 | 2019-01-18 01:38:11 | 0:35:59 | 0:18:08 | 0:17:51 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:19:30.019948+00:00 smithi002 kernel: [ 60.737546] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475502 |
|
2019-01-17 23:11:16 | 2019-01-18 01:03:52 | 2019-01-18 13:08:21 | 12:04:29 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | ||||
| fail | 3475504 |
|
2019-01-17 23:11:17 | 2019-01-18 01:03:52 | 2019-01-18 02:23:53 | 1:20:01 | 0:41:39 | 0:38:22 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:41:00.019689+00:00 smithi073 kernel: [ 60.817371] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475506 |
|
2019-01-17 23:11:18 | 2019-01-18 01:04:05 | 2019-01-18 06:38:15 | 5:34:10 | 0:10:59 | 5:23:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T06:26:30.019341+00:00 smithi088 kernel: [ 61.742060] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475508 |
|
2019-01-17 23:11:18 | 2019-01-18 01:04:06 | 2019-01-18 01:38:05 | 0:33:59 | 0:11:29 | 0:22:30 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:24:40.024226+00:00 smithi172 kernel: [ 109.773445] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475510 |
|
2019-01-17 23:11:19 | 2019-01-18 01:04:11 | 2019-01-18 01:58:16 | 0:54:05 | 0:39:00 | 0:15:05 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:19:00.019823+00:00 smithi139 kernel: [ 64.780670] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475512 |
|
2019-01-17 23:11:20 | 2019-01-18 01:04:11 | 2019-01-18 02:48:17 | 1:44:06 | 0:43:03 | 1:01:03 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:04:00.019803+00:00 smithi102 kernel: [ 72.770448] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475514 |
|
2019-01-17 23:11:21 | 2019-01-18 01:04:12 | 2019-01-18 02:04:12 | 1:00:00 | 0:12:08 | 0:47:52 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:50:10.019990+00:00 smithi065 kernel: [ 28.772610] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475516 |
|
2019-01-17 23:11:21 | 2019-01-18 01:06:04 | 2019-01-18 02:20:04 | 1:14:00 | 0:55:30 | 0:18:30 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | ||
Failure Reason:
"2019-01-18 01:31:12.509229 mon.a mon.0 172.21.15.74:6789/0 152 : cluster [WRN] Health check failed: 9981 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
| dead | 3475518 |
|
2019-01-17 23:11:22 | 2019-01-18 01:06:12 | 2019-01-18 13:08:25 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | — | ||||
| fail | 3475520 |
|
2019-01-17 23:11:23 | 2019-01-18 01:08:06 | 2019-01-18 01:38:05 | 0:29:59 | 0:11:55 | 0:18:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:25:40.019545+00:00 smithi097 kernel: [ 68.746958] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475522 |
|
2019-01-17 23:11:24 | 2019-01-18 01:08:06 | 2019-01-18 06:00:10 | 4:52:04 | 0:19:32 | 4:32:32 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T05:39:10.020377+00:00 smithi029 kernel: [ 45.690007] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475524 |
|
2019-01-17 23:11:24 | 2019-01-18 01:08:07 | 2019-01-18 02:40:08 | 1:32:01 | 0:24:56 | 1:07:05 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:13:20.019811+00:00 smithi166 kernel: [ 68.757707] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475526 |
|
2019-01-17 23:11:25 | 2019-01-18 01:08:12 | 2019-01-18 01:34:12 | 0:26:00 | 0:15:06 | 0:10:54 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:17:10.019610+00:00 smithi134 kernel: [ 62.776758] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475528 |
|
2019-01-17 23:11:26 | 2019-01-18 01:08:16 | 2019-01-18 13:10:30 | 12:02:14 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | — | ||||
| fail | 3475530 |
|
2019-01-17 23:11:27 | 2019-01-18 01:08:22 | 2019-01-18 02:58:29 | 1:50:07 | 0:21:46 | 1:28:21 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:36:10.019671+00:00 smithi068 kernel: [ 58.560941] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475532 |
|
2019-01-17 23:11:27 | 2019-01-18 01:10:23 | 2019-01-18 02:50:23 | 1:40:00 | 0:09:51 | 1:30:09 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:39:00.020329+00:00 smithi142 kernel: [ 26.552872] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475534 |
|
2019-01-17 23:11:28 | 2019-01-18 01:10:23 | 2019-01-18 05:32:27 | 4:22:04 | 0:10:31 | 4:11:33 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T05:21:10.019824+00:00 smithi014 kernel: [ 71.692776] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475536 |
|
2019-01-17 23:11:29 | 2019-01-18 01:12:15 | 2019-01-18 01:34:14 | 0:21:59 | 0:10:54 | 0:11:05 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:22:10.020144+00:00 smithi131 kernel: [ 48.544799] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475538 |
|
2019-01-17 23:11:30 | 2019-01-18 01:12:15 | 2019-01-18 02:08:15 | 0:56:00 | 0:35:59 | 0:20:01 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:32:00.020085+00:00 smithi182 kernel: [ 42.553904] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475540 |
|
2019-01-17 23:11:31 | 2019-01-18 01:14:05 | 2019-01-18 07:32:11 | 6:18:06 | 0:16:19 | 6:01:47 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T07:15:00.024168+00:00 smithi197 kernel: [ 121.765420] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475542 |
|
2019-01-17 23:11:31 | 2019-01-18 01:14:05 | 2019-01-18 02:52:06 | 1:38:01 | 0:10:15 | 1:27:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:40:10.019561+00:00 smithi164 kernel: [ 64.564434] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475544 |
|
2019-01-17 23:11:32 | 2019-01-18 01:14:05 | 2019-01-18 09:14:13 | 8:00:08 | 0:37:13 | 7:22:55 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T08:36:50.024621+00:00 smithi041 kernel: [ 119.756517] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475546 |
|
2019-01-17 23:11:33 | 2019-01-18 01:14:06 | 2019-01-18 02:16:06 | 1:02:00 | 0:08:00 | 0:54:00 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:06:20.019435+00:00 smithi192 kernel: [ 68.751664] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475548 |
|
2019-01-17 23:11:34 | 2019-01-18 01:14:12 | 2019-01-18 01:38:12 | 0:24:00 | 0:09:26 | 0:14:34 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:27:00.019531+00:00 smithi079 kernel: [ 52.298447] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475550 |
|
2019-01-17 23:11:34 | 2019-01-18 01:16:08 | 2019-01-18 02:14:08 | 0:58:00 | 0:31:38 | 0:26:22 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:42:00.019578+00:00 smithi101 kernel: [ 60.546024] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475552 |
|
2019-01-17 23:11:35 | 2019-01-18 01:18:05 | 2019-01-18 05:26:10 | 4:08:05 | 0:10:45 | 3:57:20 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T05:14:50.024508+00:00 smithi089 kernel: [ 86.771255] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475554 |
|
2019-01-17 23:11:36 | 2019-01-18 01:18:05 | 2019-01-18 01:46:05 | 0:28:00 | 0:16:58 | 0:11:02 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:28:00.019709+00:00 smithi109 kernel: [ 43.538133] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475556 |
|
2019-01-17 23:11:37 | 2019-01-18 01:18:06 | 2019-01-18 03:58:08 | 2:40:02 | 0:25:59 | 2:14:03 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T03:32:00.019806+00:00 smithi102 kernel: [ 51.738355] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475558 |
|
2019-01-17 23:11:37 | 2019-01-18 01:18:09 | 2019-01-18 13:20:22 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | ||||
| fail | 3475560 |
|
2019-01-17 23:11:38 | 2019-01-18 01:18:13 | 2019-01-18 01:54:13 | 0:36:00 | 0:25:46 | 0:10:14 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:28:00.019821+00:00 smithi069 kernel: [ 60.738794] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475562 |
|
2019-01-17 23:11:39 | 2019-01-18 01:18:22 | 2019-01-18 13:20:39 | 12:02:17 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | — | ||||
| fail | 3475564 |
|
2019-01-17 23:11:40 | 2019-01-18 01:20:10 | 2019-01-18 02:28:10 | 1:08:00 | 0:40:02 | 0:27:58 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:47:00.019889+00:00 smithi055 kernel: [ 69.689282] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475566 |
|
2019-01-17 23:11:40 | 2019-01-18 01:20:10 | 2019-01-18 02:02:10 | 0:42:00 | 0:18:21 | 0:23:39 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:43:00.019584+00:00 smithi103 kernel: [ 31.760541] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475568 |
|
2019-01-17 23:11:41 | 2019-01-18 01:20:10 | 2019-01-18 13:22:23 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | — | ||||
| fail | 3475570 |
|
2019-01-17 23:11:42 | 2019-01-18 01:20:10 | 2019-01-18 02:48:10 | 1:28:00 | 0:35:14 | 0:52:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:12:00.019587+00:00 smithi103 kernel: [ 54.759782] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475572 |
|
2019-01-17 23:11:42 | 2019-01-18 01:20:20 | 2019-01-18 05:48:24 | 4:28:04 | 0:12:20 | 4:15:44 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T05:35:10.025063+00:00 smithi189 kernel: [ 106.570742] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475574 |
|
2019-01-17 23:11:43 | 2019-01-18 01:22:06 | 2019-01-18 02:50:06 | 1:28:00 | 1:02:53 | 0:25:07 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:45:10.019644+00:00 smithi001 kernel: [ 41.752939] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475576 |
|
2019-01-17 23:11:44 | 2019-01-18 01:22:06 | 2019-01-18 02:32:07 | 1:10:01 | 0:37:29 | 0:32:32 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:53:10.020313+00:00 smithi038 kernel: [ 60.749761] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475578 |
|
2019-01-17 23:11:45 | 2019-01-18 01:22:19 | 2019-01-18 02:06:19 | 0:44:00 | 0:29:22 | 0:14:38 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:36:00.020696+00:00 smithi050 kernel: [ 38.750361] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| dead | 3475580 |
|
2019-01-17 23:11:45 | 2019-01-18 01:24:19 | 2019-01-18 13:26:32 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | — | ||||
| fail | 3475582 |
|
2019-01-17 23:11:46 | 2019-01-18 01:26:15 | 2019-01-18 02:46:16 | 1:20:01 | 1:04:01 | 0:16:00 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | ||
Failure Reason:
"2019-01-18 01:48:55.655085 mon.a mon.0 172.21.15.134:6789/0 163 : cluster [WRN] Health check failed: 16079 slow requests are blocked > 32 sec. Implicated osds 0,1,3,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
| fail | 3475584 |
|
2019-01-17 23:11:47 | 2019-01-18 01:26:16 | 2019-01-18 08:58:28 | 7:32:12 | 0:10:02 | 7:22:10 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T08:47:40.019846+00:00 smithi064 kernel: [ 69.560557] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475586 |
|
2019-01-17 23:11:48 | 2019-01-18 01:26:15 | 2019-01-18 01:54:15 | 0:28:00 | 0:11:48 | 0:16:12 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:41:00.019682+00:00 smithi087 kernel: [ 32.749876] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475588 |
|
2019-01-17 23:11:48 | 2019-01-18 01:26:16 | 2019-01-18 07:56:22 | 6:30:06 | 0:28:09 | 6:01:57 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T07:27:00.024156+00:00 smithi204 kernel: [ 83.761448] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475590 |
|
2019-01-17 23:11:49 | 2019-01-18 01:28:07 | 2019-01-18 05:22:10 | 3:54:03 | 0:39:19 | 3:14:44 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T04:42:30.019966+00:00 smithi065 kernel: [ 54.564486] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475592 |
|
2019-01-17 23:11:50 | 2019-01-18 01:28:07 | 2019-01-18 03:06:08 | 1:38:01 | 0:23:36 | 1:14:25 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:42:20.019122+00:00 smithi161 kernel: [ 61.564713] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475594 |
|
2019-01-17 23:11:50 | 2019-01-18 01:28:13 | 2019-01-18 08:12:19 | 6:44:06 | 0:09:56 | 6:34:10 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T08:00:50.024576+00:00 smithi186 kernel: [ 90.753998] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475596 |
|
2019-01-17 23:11:51 | 2019-01-18 01:30:14 | 2019-01-18 01:58:14 | 0:28:00 | 0:10:49 | 0:17:11 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:46:00.019698+00:00 smithi096 kernel: [ 48.542089] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475598 |
|
2019-01-17 23:11:52 | 2019-01-18 01:30:16 | 2019-01-18 02:04:16 | 0:34:00 | 0:09:00 | 0:25:00 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:54:40.019768+00:00 smithi129 kernel: [ 73.556804] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475600 |
|
2019-01-17 23:11:53 | 2019-01-18 01:31:53 | 2019-01-18 05:57:57 | 4:26:04 | 0:12:21 | 4:13:43 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T05:45:00.019782+00:00 smithi086 kernel: [ 45.746321] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475602 |
|
2019-01-17 23:11:53 | 2019-01-18 01:31:57 | 2019-01-18 02:09:56 | 0:37:59 | 0:15:20 | 0:22:39 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | ||
Failure Reason:
"2019-01-18 02:00:44.946465 mon.b mon.0 172.21.15.85:6789/0 224 : cluster [WRN] Health check failed: 47 slow requests are blocked > 32 sec. Implicated osds 3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
| fail | 3475604 |
|
2019-01-17 23:11:54 | 2019-01-18 01:34:16 | 2019-01-18 02:26:16 | 0:52:00 | 0:24:47 | 0:27:13 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T02:00:50.024607+00:00 smithi174 kernel: [ 88.769410] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475606 |
|
2019-01-17 23:11:55 | 2019-01-18 01:34:16 | 2019-01-18 04:54:18 | 3:20:02 | 0:24:31 | 2:55:31 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T04:29:40.019757+00:00 smithi052 kernel: [ 63.739736] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475608 |
|
2019-01-17 23:11:56 | 2019-01-18 01:34:16 | 2019-01-18 01:56:15 | 0:21:59 | 0:09:40 | 0:12:19 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:46:10.020278+00:00 smithi061 kernel: [ 27.564851] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475610 |
|
2019-01-17 23:11:56 | 2019-01-18 01:34:16 | 2019-01-18 07:12:21 | 5:38:05 | 0:21:16 | 5:16:49 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | 6 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T06:49:30.024151+00:00 smithi121 kernel: [ 103.324813] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
| fail | 3475612 |
|
2019-01-17 23:11:57 | 2019-01-18 01:34:16 | 2019-01-18 02:04:16 | 0:30:00 | 0:07:47 | 0:22:13 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | ||
Failure Reason:
'/home/ubuntu/cephtest/archive/syslog/kern.log:2019-01-18T01:56:00.019767+00:00 smithi067 kernel: [ 64.556006] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908 ' in syslog |
||||||||||||||
Pulpito