User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
---|---|---|---|---|---|---|---|---|---|---|---|
teuthology | 2019-07-06 05:20:02 | 2019-07-08 13:01:54 | 2019-07-09 08:39:47 | 19:37:53 | kcephfs | luminous | smithi | 09c5881 | 181 | 28 | 28 |
Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
pass | 4098789 | 2019-07-06 05:20:30 | 2019-07-08 12:14:28 | 2019-07-08 12:42:27 | 0:27:59 | 0:15:46 | 0:12:13 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4098790 | 2019-07-06 05:20:31 | 2019-07-08 12:14:28 | 2019-07-08 13:18:28 | 1:04:00 | 0:45:34 | 0:18:26 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4098791 | 2019-07-06 05:20:32 | 2019-07-08 12:15:13 | 2019-07-08 17:43:18 | 5:28:05 | 0:11:51 | 5:16:14 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098792 | 2019-07-06 05:20:32 | 2019-07-08 12:15:49 | 2019-07-08 12:57:48 | 0:41:59 | 0:31:02 | 0:10:57 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 4098793 | 2019-07-06 05:20:33 | 2019-07-08 12:16:45 | 2019-07-08 12:52:44 | 0:35:59 | 0:22:31 | 0:13:28 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi081 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098794 | 2019-07-06 05:20:34 | 2019-07-08 12:17:03 | 2019-07-08 16:11:06 | 3:54:03 | 0:11:10 | 3:42:53 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4098795 | 2019-07-06 05:20:34 | 2019-07-08 12:17:03 | 2019-07-08 13:53:04 | 1:36:01 | 0:34:32 | 1:01:29 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098796 | 2019-07-06 05:20:35 | 2019-07-08 12:17:03 | 2019-07-08 13:55:04 | 1:38:01 | 0:25:23 | 1:12:38 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098797 | 2019-07-06 05:20:36 | 2019-07-08 12:19:53 | 2019-07-09 00:22:06 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 4098798 | 2019-07-06 05:20:37 | 2019-07-08 12:19:54 | 2019-07-08 13:57:54 | 1:38:00 | 0:44:17 | 0:53:43 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098799 | 2019-07-06 05:20:37 | 2019-07-08 12:19:58 | 2019-07-08 12:51:57 | 0:31:59 | 0:19:19 | 0:12:40 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098800 | 2019-07-06 05:20:38 | 2019-07-08 12:21:51 | 2019-07-08 20:13:58 | 7:52:07 | 0:40:25 | 7:11:42 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4098801 | 2019-07-06 05:20:39 | 2019-07-08 12:21:58 | 2019-07-08 14:43:59 | 2:22:01 | 0:46:10 | 1:35:51 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098802 | 2019-07-06 05:20:39 | 2019-07-08 12:23:12 | 2019-07-08 13:51:12 | 1:28:00 | 1:08:52 | 0:19:08 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 12:54:15.879770 mon.a mon.0 172.21.15.37:6789/0 152 : cluster [WRN] Health check failed: 38493 slow requests are blocked > 32 sec. Implicated osds 0,2,3,4,5,6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098803 | 2019-07-06 05:20:40 | 2019-07-08 12:25:03 | 2019-07-08 17:21:08 | 4:56:05 | 0:11:39 | 4:44:26 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4098804 | 2019-07-06 05:20:41 | 2019-07-08 12:25:46 | 2019-07-08 12:57:45 | 0:31:59 | 0:18:51 | 0:13:08 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098805 | 2019-07-06 05:20:41 | 2019-07-08 12:25:49 | 2019-07-08 13:09:49 | 0:44:00 | 0:30:56 | 0:13:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4098806 | 2019-07-06 05:20:42 | 2019-07-08 12:26:41 | 2019-07-08 15:34:43 | 3:08:02 | 0:22:13 | 2:45:49 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4098807 | 2019-07-06 05:20:43 | 2019-07-08 12:26:42 | 2019-07-08 13:24:41 | 0:57:59 | 0:30:34 | 0:27:25 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098808 | 2019-07-06 05:20:43 | 2019-07-08 12:28:08 | 2019-07-08 13:04:07 | 0:35:59 | 0:15:06 | 0:20:53 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098809 | 2019-07-06 05:20:44 | 2019-07-08 12:28:08 | 2019-07-08 18:42:13 | 6:14:05 | 0:24:54 | 5:49:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4098810 | 2019-07-06 05:20:45 | 2019-07-08 12:28:38 | 2019-07-08 14:16:39 | 1:48:01 | 0:17:22 | 1:30:39 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098811 | 2019-07-06 05:20:46 | 2019-07-08 12:30:12 | 2019-07-08 12:58:11 | 0:27:59 | 0:16:31 | 0:11:28 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4098812 | 2019-07-06 05:20:46 | 2019-07-08 12:31:05 | 2019-07-08 20:19:13 | 7:48:08 | 0:16:12 | 7:31:56 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4098813 | 2019-07-06 05:20:47 | 2019-07-08 12:31:24 | 2019-07-08 13:23:24 | 0:52:00 | 0:14:44 | 0:37:16 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098814 | 2019-07-06 05:20:48 | 2019-07-08 12:31:46 | 2019-07-08 13:13:45 | 0:41:59 | 0:15:49 | 0:26:10 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 4098815 | 2019-07-06 05:20:48 | 2019-07-08 12:32:12 | 2019-07-08 13:40:12 | 1:08:00 | 0:29:07 | 0:38:53 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi139 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098816 | 2019-07-06 05:20:49 | 2019-07-08 12:32:55 | 2019-07-08 21:55:04 | 9:22:09 | 0:15:04 | 9:07:05 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4098817 | 2019-07-06 05:20:50 | 2019-07-08 12:34:29 | 2019-07-08 13:50:29 | 1:16:00 | 0:19:36 | 0:56:24 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 4098818 | 2019-07-06 05:20:50 | 2019-07-08 12:34:29 | 2019-07-08 13:26:28 | 0:51:59 | 0:22:46 | 0:29:13 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi168 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098819 | 2019-07-06 05:20:51 | 2019-07-08 12:34:58 | 2019-07-08 16:59:02 | 4:24:04 | 0:36:57 | 3:47:07 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098820 | 2019-07-06 05:20:52 | 2019-07-08 12:36:17 | 2019-07-08 13:34:17 | 0:58:00 | 0:33:14 | 0:24:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098821 | 2019-07-06 05:20:52 | 2019-07-08 12:37:54 | 2019-07-08 13:15:54 | 0:38:00 | 0:24:53 | 0:13:07 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098822 | 2019-07-06 05:20:53 | 2019-07-08 12:37:54 | 2019-07-09 00:40:07 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
pass | 4098823 | 2019-07-06 05:20:54 | 2019-07-08 12:40:19 | 2019-07-08 14:38:20 | 1:58:01 | 0:48:55 | 1:09:06 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098824 | 2019-07-06 05:20:54 | 2019-07-08 12:40:22 | 2019-07-08 13:20:22 | 0:40:00 | 0:28:10 | 0:11:50 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098825 | 2019-07-06 05:20:55 | 2019-07-08 12:42:20 | 2019-07-08 17:44:24 | 5:02:04 | 0:30:59 | 4:31:05 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4098826 | 2019-07-06 05:20:56 | 2019-07-08 12:42:20 | 2019-07-08 14:42:21 | 2:00:01 | 0:48:43 | 1:11:18 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098827 | 2019-07-06 05:20:56 | 2019-07-08 12:42:28 | 2019-07-08 13:32:28 | 0:50:00 | 0:29:07 | 0:20:53 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 13:14:52.862684 mon.b mon.0 172.21.15.12:6789/0 141 : cluster [WRN] Health check failed: 6 slow requests are blocked > 32 sec. Implicated osds 4,6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 4098828 | 2019-07-06 05:20:57 | 2019-07-08 12:44:54 | 2019-07-09 00:47:08 | 12:02:14 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | — | |||||
pass | 4098829 | 2019-07-06 05:20:58 | 2019-07-08 12:46:17 | 2019-07-08 13:50:17 | 1:04:00 | 0:18:40 | 0:45:20 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098830 | 2019-07-06 05:20:58 | 2019-07-08 12:46:56 | 2019-07-08 13:38:56 | 0:52:00 | 0:31:32 | 0:20:28 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4098831 | 2019-07-06 05:20:59 | 2019-07-08 12:48:24 | 2019-07-08 15:00:26 | 2:12:02 | 0:15:23 | 1:56:39 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4098832 | 2019-07-06 05:21:00 | 2019-07-08 12:50:08 | 2019-07-08 13:50:08 | 1:00:00 | 0:17:50 | 0:42:10 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098833 | 2019-07-06 05:21:01 | 2019-07-08 12:52:09 | 2019-07-08 13:34:09 | 0:42:00 | 0:16:25 | 0:25:35 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
fail | 4098834 | 2019-07-06 05:21:01 | 2019-07-08 12:52:09 | 2019-07-08 16:20:12 | 3:28:03 | 0:27:29 | 3:00:34 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
Failure Reason:
"2019-07-08 16:09:59.372891 mon.b mon.0 172.21.15.137:6789/0 962 : cluster [WRN] Health check failed: 20 slow requests are blocked > 32 sec. Implicated osds 0,4 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098835 | 2019-07-06 05:21:02 | 2019-07-08 12:52:09 | 2019-07-08 14:00:09 | 1:08:00 | 0:17:30 | 0:50:30 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098836 | 2019-07-06 05:21:03 | 2019-07-08 12:52:46 | 2019-07-08 13:20:45 | 0:27:59 | 0:16:14 | 0:11:45 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098837 | 2019-07-06 05:21:03 | 2019-07-08 12:53:34 | 2019-07-09 00:55:46 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 4098838 | 2019-07-06 05:21:04 | 2019-07-08 12:54:13 | 2019-07-08 13:20:12 | 0:25:59 | 0:15:11 | 0:10:48 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098839 | 2019-07-06 05:21:05 | 2019-07-08 12:54:26 | 2019-07-08 13:22:25 | 0:27:59 | 0:15:55 | 0:12:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4098840 | 2019-07-06 05:21:05 | 2019-07-08 12:56:40 | 2019-07-08 14:30:40 | 1:34:00 | 0:46:42 | 0:47:18 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4098841 | 2019-07-06 05:21:06 | 2019-07-08 12:58:02 | 2019-07-08 19:26:09 | 6:28:07 | 0:12:16 | 6:15:51 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
fail | 4098842 | 2019-07-06 05:21:07 | 2019-07-08 12:58:02 | 2019-07-08 14:34:03 | 1:36:01 | 0:57:08 | 0:38:53 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 13:52:27.902747 mon.a mon.0 172.21.15.39:6789/0 1286 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 4098843 | 2019-07-06 05:21:07 | 2019-07-08 12:58:13 | 2019-07-08 14:12:13 | 1:14:00 | 0:23:38 | 0:50:22 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi160 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098844 | 2019-07-06 05:21:08 | 2019-07-08 12:59:56 | 2019-07-08 18:06:01 | 5:06:05 | 0:11:11 | 4:54:54 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4098845 | 2019-07-06 05:21:09 | 2019-07-08 12:59:56 | 2019-07-08 13:49:56 | 0:50:00 | 0:38:43 | 0:11:17 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098846 | 2019-07-06 05:21:09 | 2019-07-08 13:01:34 | 2019-07-08 13:49:34 | 0:48:00 | 0:24:39 | 0:23:21 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4098847 | 2019-07-06 05:21:10 | 2019-07-08 13:01:54 | 2019-07-08 16:03:56 | 3:02:02 | 0:17:26 | 2:44:36 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | 6 | |||
pass | 4098848 | 2019-07-06 05:21:11 | 2019-07-08 13:02:07 | 2019-07-08 14:02:07 | 1:00:00 | 0:47:50 | 0:12:10 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098849 | 2019-07-06 05:21:11 | 2019-07-08 13:02:21 | 2019-07-08 13:40:21 | 0:38:00 | 0:17:58 | 0:20:02 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098850 | 2019-07-06 05:21:12 | 2019-07-08 13:04:24 | 2019-07-08 19:22:30 | 6:18:06 | 0:38:32 | 5:39:34 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4098851 | 2019-07-06 05:21:13 | 2019-07-08 13:06:09 | 2019-07-08 14:18:09 | 1:12:00 | 0:49:09 | 0:22:51 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098852 | 2019-07-06 05:21:14 | 2019-07-08 13:06:09 | 2019-07-08 14:02:09 | 0:56:00 | 0:29:27 | 0:26:33 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 13:44:30.770455 mon.b mon.0 172.21.15.107:6789/0 125 : cluster [WRN] Health check failed: 7018 slow requests are blocked > 32 sec. Implicated osds 0,4,5,6 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098853 | 2019-07-06 05:21:14 | 2019-07-08 13:06:39 | 2019-07-08 18:08:44 | 5:02:05 | 0:12:01 | 4:50:04 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4098854 | 2019-07-06 05:21:15 | 2019-07-08 13:08:17 | 2019-07-08 13:38:16 | 0:29:59 | 0:19:26 | 0:10:33 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098855 | 2019-07-06 05:21:16 | 2019-07-08 13:10:08 | 2019-07-08 13:46:07 | 0:35:59 | 0:22:52 | 0:13:07 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 4098856 | 2019-07-06 05:21:16 | 2019-07-08 13:10:08 | 2019-07-08 16:02:09 | 2:52:01 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | |||||
Failure Reason:
reached maximum tries (60) after waiting for 900 seconds |
||||||||||||||
pass | 4098857 | 2019-07-06 05:21:17 | 2019-07-08 13:12:35 | 2019-07-08 14:34:35 | 1:22:00 | 1:04:52 | 0:17:08 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098858 | 2019-07-06 05:21:18 | 2019-07-08 13:14:03 | 2019-07-08 13:46:02 | 0:31:59 | 0:16:03 | 0:15:56 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098859 | 2019-07-06 05:21:18 | 2019-07-08 13:14:19 | 2019-07-08 19:40:25 | 6:26:06 | 0:24:35 | 6:01:31 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4098860 | 2019-07-06 05:21:19 | 2019-07-08 13:16:11 | 2019-07-08 14:02:11 | 0:46:00 | 0:18:56 | 0:27:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098861 | 2019-07-06 05:21:20 | 2019-07-08 13:18:19 | 2019-07-08 13:48:18 | 0:29:59 | 0:17:17 | 0:12:42 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098862 | 2019-07-06 05:21:20 | 2019-07-08 13:18:29 | 2019-07-09 01:20:41 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | — | |||||
pass | 4098863 | 2019-07-06 05:21:21 | 2019-07-08 13:20:26 | 2019-07-08 13:48:25 | 0:27:59 | 0:15:06 | 0:12:53 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098864 | 2019-07-06 05:21:22 | 2019-07-08 13:20:26 | 2019-07-08 13:50:26 | 0:30:00 | 0:15:06 | 0:14:54 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 4098865 | 2019-07-06 05:21:22 | 2019-07-08 13:20:26 | 2019-07-08 15:22:27 | 2:02:01 | 0:27:37 | 1:34:24 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi093 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
dead | 4098866 | 2019-07-06 05:21:23 | 2019-07-08 13:20:26 | 2019-07-09 01:22:39 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
pass | 4098867 | 2019-07-06 05:21:24 | 2019-07-08 13:20:27 | 2019-07-08 14:40:27 | 1:20:00 | 0:28:26 | 0:51:34 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 4098868 | 2019-07-06 05:21:24 | 2019-07-08 13:20:47 | 2019-07-08 14:24:47 | 1:04:00 | 0:23:13 | 0:40:47 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi143 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098869 | 2019-07-06 05:21:25 | 2019-07-08 13:20:51 | 2019-07-08 14:30:51 | 1:10:00 | 0:39:47 | 0:30:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098870 | 2019-07-06 05:21:26 | 2019-07-08 13:22:30 | 2019-07-08 14:12:30 | 0:50:00 | 0:38:25 | 0:11:35 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098871 | 2019-07-06 05:21:27 | 2019-07-08 13:22:30 | 2019-07-08 14:02:30 | 0:40:00 | 0:24:45 | 0:15:15 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098872 | 2019-07-06 05:21:27 | 2019-07-08 13:22:30 | 2019-07-09 01:24:42 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
pass | 4098873 | 2019-07-06 05:21:28 | 2019-07-08 13:23:26 | 2019-07-08 15:01:26 | 1:38:00 | 1:01:40 | 0:36:20 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098874 | 2019-07-06 05:21:29 | 2019-07-08 13:24:09 | 2019-07-08 14:12:08 | 0:47:59 | 0:29:15 | 0:18:44 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098875 | 2019-07-06 05:21:29 | 2019-07-08 13:24:43 | 2019-07-08 20:12:49 | 6:48:06 | 0:30:47 | 6:17:19 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4098876 | 2019-07-06 05:21:30 | 2019-07-08 13:26:46 | 2019-07-08 14:42:47 | 1:16:01 | 0:43:31 | 0:32:30 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098877 | 2019-07-06 05:21:31 | 2019-07-08 13:30:15 | 2019-07-08 14:28:15 | 0:58:00 | 0:29:13 | 0:28:47 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 14:11:22.819553 mon.a mon.0 172.21.15.145:6789/0 146 : cluster [WRN] Health check failed: 14240 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098878 | 2019-07-06 05:21:31 | 2019-07-08 13:32:40 | 2019-07-08 20:20:46 | 6:48:06 | 0:12:12 | 6:35:54 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4098879 | 2019-07-06 05:21:32 | 2019-07-08 13:32:40 | 2019-07-08 14:08:39 | 0:35:59 | 0:20:18 | 0:15:41 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098880 | 2019-07-06 05:21:33 | 2019-07-08 13:34:28 | 2019-07-08 14:30:28 | 0:56:00 | 0:26:33 | 0:29:27 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 4098881 | 2019-07-06 05:21:34 | 2019-07-08 13:34:28 | 2019-07-09 01:36:40 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | — | |||||
pass | 4098882 | 2019-07-06 05:21:34 | 2019-07-08 15:54:50 | 2019-07-08 16:26:50 | 0:32:00 | 0:19:07 | 0:12:53 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098883 | 2019-07-06 05:21:35 | 2019-07-08 15:56:18 | 2019-07-08 16:30:18 | 0:34:00 | 0:15:41 | 0:18:19 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098884 | 2019-07-06 05:21:36 | 2019-07-08 15:56:42 | 2019-07-09 03:02:53 | 11:06:11 | 0:26:55 | 10:39:16 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
fail | 4098885 | 2019-07-06 05:21:36 | 2019-07-08 15:56:43 | 2019-07-08 16:36:43 | 0:40:00 | 0:22:19 | 0:17:41 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2019-07-08 16:26:51.888505 mon.b mon.0 172.21.15.38:6789/0 146 : cluster [WRN] Health check failed: 546 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098886 | 2019-07-06 05:21:37 | 2019-07-08 15:56:45 | 2019-07-08 16:26:45 | 0:30:00 | 0:16:14 | 0:13:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098887 | 2019-07-06 05:21:38 | 2019-07-08 15:56:47 | 2019-07-09 03:59:02 | 12:02:15 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 4098888 | 2019-07-06 05:21:38 | 2019-07-08 15:58:16 | 2019-07-08 16:28:15 | 0:29:59 | 0:14:47 | 0:15:12 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098889 | 2019-07-06 05:21:39 | 2019-07-08 15:58:16 | 2019-07-08 16:30:16 | 0:32:00 | 0:16:04 | 0:15:56 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4098890 | 2019-07-06 05:21:40 | 2019-07-08 15:58:33 | 2019-07-08 16:56:32 | 0:57:59 | 0:43:19 | 0:14:40 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4098891 | 2019-07-06 05:21:41 | 2019-07-08 15:58:47 | 2019-07-08 21:56:55 | 5:58:08 | 0:12:39 | 5:45:29 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098892 | 2019-07-06 05:21:41 | 2019-07-08 15:59:01 | 2019-07-08 16:53:01 | 0:54:00 | 0:28:52 | 0:25:08 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 4098893 | 2019-07-06 05:21:42 | 2019-07-08 16:01:01 | 2019-07-08 16:37:01 | 0:36:00 | 0:24:30 | 0:11:30 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi001 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098894 | 2019-07-06 05:21:43 | 2019-07-08 16:01:01 | 2019-07-08 20:27:06 | 4:26:05 | 0:11:57 | 4:14:08 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4098895 | 2019-07-06 05:21:43 | 2019-07-08 16:01:02 | 2019-07-08 16:59:02 | 0:58:00 | 0:31:56 | 0:26:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098896 | 2019-07-06 05:21:44 | 2019-07-08 16:02:27 | 2019-07-08 16:42:27 | 0:40:00 | 0:25:45 | 0:14:15 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098897 | 2019-07-06 05:21:45 | 2019-07-08 16:02:45 | 2019-07-09 04:05:00 | 12:02:15 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 4098898 | 2019-07-06 05:21:45 | 2019-07-08 16:03:54 | 2019-07-08 17:17:54 | 1:14:00 | 0:58:19 | 0:15:41 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098899 | 2019-07-06 05:21:46 | 2019-07-08 16:03:58 | 2019-07-08 16:53:57 | 0:49:59 | 0:17:57 | 0:32:02 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098900 | 2019-07-06 05:21:47 | 2019-07-08 16:04:01 | 2019-07-08 22:26:06 | 6:22:05 | 0:39:56 | 5:42:09 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4098901 | 2019-07-06 05:21:47 | 2019-07-08 16:04:39 | 2019-07-08 17:26:40 | 1:22:01 | 0:45:32 | 0:36:29 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098902 | 2019-07-06 05:21:48 | 2019-07-08 16:05:09 | 2019-07-08 17:47:10 | 1:42:01 | 1:07:40 | 0:34:21 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 16:51:22.365128 mon.b mon.0 172.21.15.44:6789/0 138 : cluster [WRN] Health check failed: 29136 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 4098903 | 2019-07-06 05:21:49 | 2019-07-08 16:56:38 | 2019-07-09 04:58:50 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | — | |||||
pass | 4098904 | 2019-07-06 05:21:49 | 2019-07-08 16:56:38 | 2019-07-08 17:26:37 | 0:29:59 | 0:18:13 | 0:11:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098905 | 2019-07-06 05:21:50 | 2019-07-08 16:56:40 | 2019-07-08 17:34:40 | 0:38:00 | 0:25:08 | 0:12:52 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 4098906 | 2019-07-06 05:21:51 | 2019-07-08 16:57:00 | 2019-07-09 04:59:11 | 12:02:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | — | |||||
pass | 4098907 | 2019-07-06 05:21:51 | 2019-07-08 16:58:48 | 2019-07-08 17:44:47 | 0:45:59 | 0:29:18 | 0:16:41 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098908 | 2019-07-06 05:21:52 | 2019-07-08 16:58:48 | 2019-07-08 17:32:47 | 0:33:59 | 0:15:58 | 0:18:01 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098909 | 2019-07-06 05:21:53 | 2019-07-08 16:59:03 | 2019-07-08 19:07:04 | 2:08:01 | 0:24:24 | 1:43:37 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4098910 | 2019-07-06 05:21:54 | 2019-07-08 16:59:03 | 2019-07-08 17:29:03 | 0:30:00 | 0:17:57 | 0:12:03 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098911 | 2019-07-06 05:21:54 | 2019-07-08 16:59:04 | 2019-07-08 17:31:03 | 0:31:59 | 0:16:08 | 0:15:51 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4098912 | 2019-07-06 05:21:55 | 2019-07-08 17:00:49 | 2019-07-09 02:48:59 | 9:48:10 | 0:16:02 | 9:32:08 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4098913 | 2019-07-06 05:21:56 | 2019-07-08 17:01:04 | 2019-07-08 17:35:03 | 0:33:59 | 0:13:55 | 0:20:04 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098914 | 2019-07-06 05:21:56 | 2019-07-08 17:04:52 | 2019-07-08 17:32:51 | 0:27:59 | 0:15:28 | 0:12:31 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
dead | 4098915 | 2019-07-06 05:21:57 | 2019-07-08 17:05:05 | 2019-07-09 05:07:32 | 12:02:27 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||||
dead | 4098916 | 2019-07-06 05:21:58 | 2019-07-08 17:05:18 | 2019-07-09 05:07:29 | 12:02:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | — | |||||
pass | 4098917 | 2019-07-06 05:21:58 | 2019-07-08 17:06:30 | 2019-07-08 17:36:29 | 0:29:59 | 0:18:00 | 0:11:59 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 4098918 | 2019-07-06 05:21:59 | 2019-07-08 17:06:47 | 2019-07-08 17:52:47 | 0:46:00 | 0:24:11 | 0:21:49 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi100 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098919 | 2019-07-06 05:22:00 | 2019-07-08 17:06:53 | 2019-07-08 19:56:54 | 2:50:01 | 0:37:40 | 2:12:21 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098920 | 2019-07-06 05:22:00 | 2019-07-08 17:07:02 | 2019-07-08 17:51:02 | 0:44:00 | 0:32:39 | 0:11:21 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098921 | 2019-07-06 05:22:01 | 2019-07-08 17:09:07 | 2019-07-08 17:53:06 | 0:43:59 | 0:25:13 | 0:18:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
pass | 4098922 | 2019-07-06 05:22:02 | 2019-07-08 17:09:07 | 2019-07-08 18:31:07 | 1:22:00 | 0:11:02 | 1:10:58 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | 6 | |||
pass | 4098923 | 2019-07-06 05:22:02 | 2019-07-08 17:09:07 | 2019-07-08 18:09:06 | 0:59:59 | 0:44:37 | 0:15:22 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
fail | 4098924 | 2019-07-06 05:22:03 | 2019-07-08 17:11:27 | 2019-07-08 18:31:27 | 1:20:00 | 1:07:07 | 0:12:53 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 17:40:56.015613 mon.a mon.0 172.21.15.51:6789/0 353 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
pass | 4098925 | 2019-07-06 05:22:04 | 2019-07-08 17:12:01 | 2019-07-08 20:02:03 | 2:50:02 | 0:23:34 | 2:26:28 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4098926 | 2019-07-06 05:22:04 | 2019-07-08 17:13:02 | 2019-07-08 18:11:02 | 0:58:00 | 0:45:15 | 0:12:45 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098927 | 2019-07-06 05:22:05 | 2019-07-08 17:13:03 | 2019-07-08 18:55:03 | 1:42:00 | 1:23:06 | 0:18:54 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 17:44:14.055069 mon.b mon.0 172.21.15.39:6789/0 141 : cluster [WRN] Health check failed: 569 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098928 | 2019-07-06 05:22:06 | 2019-07-08 17:13:03 | 2019-07-08 20:45:05 | 3:32:02 | 0:12:07 | 3:19:55 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | 6 | |||
pass | 4098929 | 2019-07-06 05:22:07 | 2019-07-08 17:13:03 | 2019-07-08 17:47:02 | 0:33:59 | 0:18:30 | 0:15:29 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098930 | 2019-07-06 05:22:07 | 2019-07-08 17:15:17 | 2019-07-08 17:57:16 | 0:41:59 | 0:30:53 | 0:11:06 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4098931 | 2019-07-06 05:22:08 | 2019-07-08 17:15:17 | 2019-07-08 19:03:18 | 1:48:01 | 0:15:22 | 1:32:39 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | 6 | |||
pass | 4098932 | 2019-07-06 05:22:09 | 2019-07-08 17:17:03 | 2019-07-08 17:47:02 | 0:29:59 | 0:18:44 | 0:11:15 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098933 | 2019-07-06 05:22:09 | 2019-07-08 17:17:03 | 2019-07-08 17:45:02 | 0:27:59 | 0:15:28 | 0:12:31 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098934 | 2019-07-06 05:22:10 | 2019-07-08 17:17:17 | 2019-07-08 20:51:20 | 3:34:03 | 0:27:24 | 3:06:39 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | 6 | |||
pass | 4098935 | 2019-07-06 05:22:11 | 2019-07-08 17:17:29 | 2019-07-08 17:47:28 | 0:29:59 | 0:16:57 | 0:13:02 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098936 | 2019-07-06 05:22:11 | 2019-07-08 17:17:56 | 2019-07-08 17:51:55 | 0:33:59 | 0:16:23 | 0:17:36 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098937 | 2019-07-06 05:22:12 | 2019-07-08 17:21:16 | 2019-07-09 05:23:28 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 4098938 | 2019-07-06 05:22:13 | 2019-07-08 17:21:29 | 2019-07-08 17:57:28 | 0:35:59 | 0:14:32 | 0:21:27 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098939 | 2019-07-06 05:22:13 | 2019-07-08 17:23:27 | 2019-07-08 18:01:27 | 0:38:00 | 0:15:40 | 0:22:20 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4098940 | 2019-07-06 05:22:14 | 2019-07-08 17:23:28 | 2019-07-08 18:35:28 | 1:12:00 | 0:49:41 | 0:22:19 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
dead | 4098941 | 2019-07-06 05:22:15 | 2019-07-08 17:25:22 | 2019-07-09 05:27:34 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | — | |||||
fail | 4098942 | 2019-07-06 05:22:15 | 2019-07-08 17:26:54 | 2019-07-08 18:28:54 | 1:02:00 | 0:51:25 | 0:10:35 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 17:57:21.668580 mon.a mon.0 172.21.15.25:6789/0 1760 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log |
||||||||||||||
fail | 4098943 | 2019-07-06 05:22:16 | 2019-07-08 17:26:54 | 2019-07-08 18:00:54 | 0:34:00 | 0:21:24 | 0:12:36 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi160 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098944 | 2019-07-06 05:22:17 | 2019-07-08 17:29:20 | 2019-07-08 19:05:20 | 1:36:00 | 0:10:57 | 1:25:03 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4098945 | 2019-07-06 05:22:17 | 2019-07-08 17:31:20 | 2019-07-08 18:29:20 | 0:58:00 | 0:33:25 | 0:24:35 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098946 | 2019-07-06 05:22:18 | 2019-07-08 17:31:29 | 2019-07-08 18:17:28 | 0:45:59 | 0:24:37 | 0:21:22 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098947 | 2019-07-06 05:22:19 | 2019-07-08 17:36:47 | 2019-07-09 05:38:58 | 12:02:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 4098948 | 2019-07-06 05:22:20 | 2019-07-08 17:43:39 | 2019-07-08 18:41:39 | 0:58:00 | 0:46:14 | 0:11:46 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098949 | 2019-07-06 05:22:20 | 2019-07-08 17:44:26 | 2019-07-08 18:22:26 | 0:38:00 | 0:24:38 | 0:13:22 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098950 | 2019-07-06 05:22:21 | 2019-07-08 17:45:05 | 2019-07-08 20:59:07 | 3:14:02 | 0:43:14 | 2:30:48 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4098951 | 2019-07-06 05:22:22 | 2019-07-08 17:45:05 | 2019-07-08 18:53:05 | 1:08:00 | 0:48:13 | 0:19:47 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098952 | 2019-07-06 05:22:22 | 2019-07-08 17:46:30 | 2019-07-08 18:24:30 | 0:38:00 | 0:27:05 | 0:10:55 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 18:08:45.803491 mon.a mon.0 172.21.15.80:6789/0 229 : cluster [WRN] Health check failed: 98 slow requests are blocked > 32 sec. Implicated osds 0,1,3 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4098953 | 2019-07-06 05:22:23 | 2019-07-08 17:47:03 | 2019-07-09 02:19:11 | 8:32:08 | 0:12:20 | 8:19:48 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4098954 | 2019-07-06 05:22:24 | 2019-07-08 17:47:03 | 2019-07-08 18:21:03 | 0:34:00 | 0:18:55 | 0:15:05 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098955 | 2019-07-06 05:22:24 | 2019-07-08 17:47:12 | 2019-07-08 18:33:11 | 0:45:59 | 0:30:58 | 0:15:01 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4098956 | 2019-07-06 05:22:25 | 2019-07-08 17:47:45 | 2019-07-08 21:31:48 | 3:44:03 | 0:20:13 | 3:23:50 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4098957 | 2019-07-06 05:22:26 | 2019-07-08 17:51:20 | 2019-07-08 18:31:20 | 0:40:00 | 0:28:30 | 0:11:30 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098958 | 2019-07-06 05:22:26 | 2019-07-08 17:51:57 | 2019-07-08 18:21:56 | 0:29:59 | 0:15:29 | 0:14:30 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4098959 | 2019-07-06 05:22:27 | 2019-07-08 17:53:04 | 2019-07-09 03:23:12 | 9:30:08 | 0:24:19 | 9:05:49 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
pass | 4098960 | 2019-07-06 05:22:28 | 2019-07-08 17:53:07 | 2019-07-08 18:29:07 | 0:36:00 | 0:17:17 | 0:18:43 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098961 | 2019-07-06 05:22:28 | 2019-07-08 17:56:09 | 2019-07-08 18:34:09 | 0:38:00 | 0:16:21 | 0:21:39 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098962 | 2019-07-06 05:22:29 | 2019-07-08 17:57:34 | 2019-07-09 05:59:45 | 12:02:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | — | |||||
pass | 4098963 | 2019-07-06 05:22:30 | 2019-07-08 17:57:34 | 2019-07-08 18:25:34 | 0:28:00 | 0:14:27 | 0:13:33 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098964 | 2019-07-06 05:22:31 | 2019-07-08 18:01:10 | 2019-07-08 18:47:10 | 0:46:00 | 0:16:02 | 0:29:58 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
fail | 4098965 | 2019-07-06 05:22:31 | 2019-07-08 18:01:28 | 2019-07-08 18:57:28 | 0:56:00 | 0:25:38 | 0:30:22 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi106 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098966 | 2019-07-06 05:22:32 | 2019-07-08 18:06:20 | 2019-07-08 20:02:20 | 1:56:00 | 0:14:16 | 1:41:44 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4098967 | 2019-07-06 05:22:33 | 2019-07-08 18:09:01 | 2019-07-08 18:53:01 | 0:44:00 | 0:20:47 | 0:23:13 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 4098968 | 2019-07-06 05:22:33 | 2019-07-08 18:09:08 | 2019-07-08 18:41:07 | 0:31:59 | 0:22:09 | 0:09:50 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi143 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098969 | 2019-07-06 05:22:34 | 2019-07-08 18:11:19 | 2019-07-09 00:39:24 | 6:28:05 | 0:38:02 | 5:50:03 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098970 | 2019-07-06 05:22:35 | 2019-07-08 18:15:20 | 2019-07-08 19:05:20 | 0:50:00 | 0:39:08 | 0:10:52 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098971 | 2019-07-06 05:22:35 | 2019-07-08 18:17:19 | 2019-07-08 18:53:18 | 0:35:59 | 0:24:42 | 0:11:17 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098972 | 2019-07-06 05:22:36 | 2019-07-08 18:17:29 | 2019-07-09 06:19:41 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
pass | 4098973 | 2019-07-06 05:22:37 | 2019-07-08 18:21:23 | 2019-07-08 19:21:22 | 0:59:59 | 0:46:41 | 0:13:18 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098974 | 2019-07-06 05:22:37 | 2019-07-08 18:21:58 | 2019-07-08 19:03:57 | 0:41:59 | 0:29:18 | 0:12:41 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4098975 | 2019-07-06 05:22:38 | 2019-07-08 18:22:44 | 2019-07-08 20:46:45 | 2:24:01 | 0:32:11 | 1:51:50 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 | |||
pass | 4098976 | 2019-07-06 05:22:39 | 2019-07-08 18:24:47 | 2019-07-08 19:26:47 | 1:02:00 | 0:49:24 | 0:12:36 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4098977 | 2019-07-06 05:22:39 | 2019-07-08 18:25:35 | 2019-07-08 19:07:35 | 0:42:00 | 0:29:04 | 0:12:56 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 18:49:53.523839 mon.a mon.0 172.21.15.129:6789/0 236 : cluster [WRN] Health check failed: 538 slow requests are blocked > 32 sec. Implicated osds 1 (REQUEST_SLOW)" in cluster log |
||||||||||||||
dead | 4098978 | 2019-07-06 05:22:40 | 2019-07-08 18:28:49 | 2019-07-09 06:31:00 | 12:02:11 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/pool-perm.yaml whitelist_health.yaml} | — | |||||
pass | 4098979 | 2019-07-06 05:22:41 | 2019-07-08 18:28:49 | 2019-07-08 19:10:48 | 0:41:59 | 0:19:25 | 0:22:34 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4098980 | 2019-07-06 05:22:41 | 2019-07-08 18:28:56 | 2019-07-08 19:02:55 | 0:33:59 | 0:22:45 | 0:11:14 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
dead | 4098981 | 2019-07-06 05:22:42 | 2019-07-08 18:29:08 | 2019-07-09 06:31:20 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/sessionmap.yaml whitelist_health.yaml} | — | |||||
pass | 4098982 | 2019-07-06 05:22:43 | 2019-07-08 18:29:20 | 2019-07-08 18:57:20 | 0:28:00 | 0:17:59 | 0:10:01 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098983 | 2019-07-06 05:22:43 | 2019-07-08 18:29:22 | 2019-07-08 19:07:22 | 0:38:00 | 0:15:45 | 0:22:15 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
dead | 4098984 | 2019-07-06 05:22:44 | 2019-07-08 18:31:26 | 2019-07-08 19:43:26 | 1:12:00 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/strays.yaml whitelist_health.yaml} | — | |||||
Failure Reason:
reached maximum tries (60) after waiting for 900 seconds |
||||||||||||||
pass | 4098985 | 2019-07-06 05:22:45 | 2019-07-08 18:31:26 | 2019-07-08 19:23:25 | 0:51:59 | 0:18:57 | 0:33:02 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4098986 | 2019-07-06 05:22:46 | 2019-07-08 18:31:28 | 2019-07-08 18:59:28 | 0:28:00 | 0:16:07 | 0:11:53 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
dead | 4098987 | 2019-07-06 05:22:46 | 2019-07-08 18:33:28 | 2019-07-09 06:35:40 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/volume-client.yaml whitelist_health.yaml} | — | |||||
pass | 4098988 | 2019-07-06 05:22:47 | 2019-07-08 18:34:10 | 2019-07-08 19:18:09 | 0:43:59 | 0:15:05 | 0:28:54 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4098989 | 2019-07-06 05:22:48 | 2019-07-08 18:34:57 | 2019-07-08 19:20:57 | 0:46:00 | 0:15:23 | 0:30:37 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
pass | 4098990 | 2019-07-06 05:22:48 | 2019-07-08 18:35:29 | 2019-07-08 20:11:29 | 1:36:00 | 0:49:54 | 0:46:06 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_dbench_iozone.yaml} | 4 | |||
pass | 4098991 | 2019-07-06 05:22:49 | 2019-07-08 18:41:24 | 2019-07-08 21:07:26 | 2:26:02 | 0:11:25 | 2:14:37 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/auto-repair.yaml whitelist_health.yaml} | 6 | |||
pass | 4098992 | 2019-07-06 05:22:50 | 2019-07-08 18:41:40 | 2019-07-08 19:23:40 | 0:42:00 | 0:27:42 | 0:14:18 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
fail | 4098993 | 2019-07-06 05:22:50 | 2019-07-08 18:42:16 | 2019-07-08 19:26:15 | 0:43:59 | 0:22:48 | 0:21:11 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi151 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
pass | 4098994 | 2019-07-06 05:22:51 | 2019-07-08 18:47:30 | 2019-07-08 21:59:32 | 3:12:02 | 0:10:57 | 3:01:05 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/backtrace.yaml whitelist_health.yaml} | 6 | |||
pass | 4098995 | 2019-07-06 05:22:52 | 2019-07-08 18:53:21 | 2019-07-08 19:51:21 | 0:58:00 | 0:38:35 | 0:19:25 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4098996 | 2019-07-06 05:22:52 | 2019-07-08 18:53:21 | 2019-07-08 19:29:20 | 0:35:59 | 0:24:24 | 0:11:35 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4098997 | 2019-07-06 05:22:53 | 2019-07-08 18:53:21 | 2019-07-09 06:55:33 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-limits.yaml whitelist_health.yaml} | — | |||||
pass | 4098998 | 2019-07-06 05:22:54 | 2019-07-08 18:55:11 | 2019-07-08 20:03:11 | 1:08:00 | 0:57:07 | 0:10:53 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4098999 | 2019-07-06 05:22:55 | 2019-07-08 18:55:11 | 2019-07-08 19:25:10 | 0:29:59 | 0:17:34 | 0:12:25 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
pass | 4099000 | 2019-07-06 05:22:55 | 2019-07-08 18:57:37 | 2019-07-09 03:53:45 | 8:56:08 | 0:39:25 | 8:16:43 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-recovery.yaml whitelist_health.yaml} | 6 | |||
pass | 4099001 | 2019-07-06 05:22:56 | 2019-07-08 18:57:37 | 2019-07-08 19:53:36 | 0:55:59 | 0:43:21 | 0:12:38 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_dbench.yaml} | 3 | |||
fail | 4099002 | 2019-07-06 05:22:57 | 2019-07-08 18:59:44 | 2019-07-08 19:41:44 | 0:42:00 | 0:28:34 | 0:13:26 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_ffsb.yaml} | 3 | |||
Failure Reason:
"2019-07-08 19:25:40.518012 mon.a mon.0 172.21.15.112:6789/0 145 : cluster [WRN] Health check failed: 7545 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4099003 | 2019-07-06 05:22:57 | 2019-07-08 19:03:17 | 2019-07-08 20:03:17 | 1:00:00 | 0:11:58 | 0:48:02 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/config-commands.yaml whitelist_health.yaml} | 6 | |||
pass | 4099004 | 2019-07-06 05:22:58 | 2019-07-08 19:03:17 | 2019-07-08 19:37:16 | 0:33:59 | 0:20:08 | 0:13:51 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsstress.yaml} | 3 | |||
pass | 4099005 | 2019-07-06 05:22:59 | 2019-07-08 19:03:19 | 2019-07-08 19:41:19 | 0:38:00 | 0:25:34 | 0:12:26 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsx.yaml} | 3 | |||
pass | 4099006 | 2019-07-06 05:22:59 | 2019-07-08 19:03:59 | 2019-07-08 20:54:00 | 1:50:01 | 0:21:39 | 1:28:22 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/damage.yaml whitelist_health.yaml} | 6 | |||
pass | 4099007 | 2019-07-06 05:23:00 | 2019-07-08 19:05:40 | 2019-07-08 20:25:40 | 1:20:00 | 1:06:05 | 0:13:55 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mon.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4099008 | 2019-07-06 05:23:01 | 2019-07-08 19:05:40 | 2019-07-08 19:35:39 | 0:29:59 | 0:15:37 | 0:14:22 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_fsync.yaml} | 3 | |||
pass | 4099009 | 2019-07-06 05:23:01 | 2019-07-08 19:07:21 | 2019-07-08 21:11:22 | 2:04:01 | 0:24:30 | 1:39:31 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/data-scan.yaml whitelist_health.yaml} | 6 | |||
fail | 4099010 | 2019-07-06 05:23:02 | 2019-07-08 19:07:24 | 2019-07-08 19:41:23 | 0:33:59 | 0:21:46 | 0:12:13 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_iozone.yaml} | 3 | |||
Failure Reason:
"2019-07-08 19:32:04.588946 mon.a mon.0 172.21.15.37:6789/0 146 : cluster [WRN] Health check failed: 465 slow requests are blocked > 32 sec. Implicated osds 0,1,3,4,5 (REQUEST_SLOW)" in cluster log |
||||||||||||||
pass | 4099011 | 2019-07-06 05:23:03 | 2019-07-08 19:07:36 | 2019-07-08 19:43:36 | 0:36:00 | 0:17:01 | 0:18:59 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_suites_pjd.yaml} | 3 | |||
pass | 4099012 | 2019-07-06 05:23:04 | 2019-07-08 19:09:32 | 2019-07-08 23:51:35 | 4:42:03 | 0:15:36 | 4:26:27 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/failover.yaml whitelist_health.yaml} | 6 | |||
pass | 4099013 | 2019-07-06 05:23:04 | 2019-07-08 19:11:05 | 2019-07-08 19:45:04 | 0:33:59 | 0:14:53 | 0:19:06 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_trivial_sync.yaml} | 3 | |||
pass | 4099014 | 2019-07-06 05:23:05 | 2019-07-08 20:19:41 | 2019-07-08 21:19:41 | 1:00:00 | 0:15:16 | 0:44:44 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_direct_io.yaml} | 3 | |||
dead | 4099015 | 2019-07-06 05:23:06 | 2019-07-08 20:21:04 | 2019-07-09 08:23:29 | 12:02:25 | smithi | master | kcephfs/mixed-clients/{clusters/1-mds-2-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml} | 4 | |||||
pass | 4099016 | 2019-07-06 05:23:06 | 2019-07-08 20:25:58 | 2019-07-08 22:21:59 | 1:56:01 | 0:14:31 | 1:41:30 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/forward-scrub.yaml whitelist_health.yaml} | 6 | |||
pass | 4099017 | 2019-07-06 05:23:07 | 2019-07-08 20:27:23 | 2019-07-08 21:27:23 | 1:00:00 | 0:18:52 | 0:41:08 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/default.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_iozone.yaml} | 3 | |||
fail | 4099018 | 2019-07-06 05:23:08 | 2019-07-08 20:34:13 | 2019-07-08 21:26:12 | 0:51:59 | 0:23:36 | 0:28:23 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_kernel_untar_build.yaml} | 3 | |||
Failure Reason:
Command failed (workunit test kernel_untar_build.sh) on smithi066 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=09c5881d464bedc73214c200c9f9cda0eda34fe9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' |
||||||||||||||
dead | 4099019 | 2019-07-06 05:23:08 | 2019-07-08 20:36:00 | 2019-07-09 08:38:12 | 12:02:12 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/journal-repair.yaml whitelist_health.yaml} | — | |||||
pass | 4099020 | 2019-07-06 05:23:09 | 2019-07-08 20:36:01 | 2019-07-08 21:28:00 | 0:51:59 | 0:32:19 | 0:19:40 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_misc.yaml} | 3 | |||
pass | 4099021 | 2019-07-06 05:23:10 | 2019-07-08 20:36:24 | 2019-07-08 21:56:25 | 1:20:01 | 0:24:49 | 0:55:12 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_o_trunc.yaml} | 3 | |||
dead | 4099022 | 2019-07-06 05:23:10 | 2019-07-08 20:37:34 | 2019-07-09 08:39:47 | 12:02:13 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-flush.yaml whitelist_health.yaml} | — | |||||
pass | 4099023 | 2019-07-06 05:23:11 | 2019-07-08 20:44:30 | 2019-07-08 21:56:30 | 1:12:00 | 0:58:36 | 0:13:24 | smithi | master | kcephfs/cephfs/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/kclient_workunit_snaps.yaml} | 3 | |||
pass | 4099024 | 2019-07-06 05:23:12 | 2019-07-08 20:45:06 | 2019-07-08 22:31:07 | 1:46:01 | 1:04:35 | 0:41:26 | smithi | master | kcephfs/thrash/{clusters/1-mds-1-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml log-config.yaml ms-die-on-skipped.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrashers/mds.yaml thrashosds-health.yaml whitelist_health.yaml workloads/kclient_workunit_suites_ffsb.yaml} | 3 | |||
pass | 4099025 | 2019-07-06 05:23:13 | 2019-07-08 20:47:01 | 2019-07-08 21:35:01 | 0:48:00 | 0:31:01 | 0:16:59 | smithi | master | kcephfs/recovery/{clusters/1-mds-4-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mounts/kmounts.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml log-config.yaml osd-asserts.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/mds-full.yaml whitelist_health.yaml} | 6 |