Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 2524241 2018-05-12 02:00:46 2018-05-12 17:10:00 2018-05-12 17:54:00 0:44:00 0:34:57 0:09:03 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524242 2018-05-12 17:10:43 2018-05-12 17:10:43 2018-05-12 17:32:42 0:21:59 0:16:19 0:05:40 ovh master rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524243 2018-05-12 02:00:47 2018-05-12 17:14:45 2018-05-12 17:44:45 0:30:00 0:21:36 0:08:24 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/repair_test.yaml} 2
fail 2524244 2018-05-12 02:00:47 2018-05-12 17:20:10 2018-05-12 18:02:10 0:42:00 0:35:14 0:06:46 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-12 17:41:14.628423 osd.1 158.69.72.204:6804/8679 54 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.720065 secs" in cluster log

fail 2524245 2018-05-12 02:00:47 2018-05-12 17:21:10 2018-05-12 17:55:10 0:34:00 0:26:46 0:07:14 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-05-12 17:48:12.562738 osd.5 158.69.72.210:6812/9303 111 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.000972 secs" in cluster log

pass 2524246 2018-05-12 02:00:48 2018-05-12 17:21:43 2018-05-12 17:55:43 0:34:00 0:26:31 0:07:29 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync-many.yaml workloads/pool-create-delete.yaml} 2
pass 2524247 2018-05-12 02:00:48 2018-05-12 17:21:56 2018-05-12 17:39:56 0:18:00 0:12:42 0:05:18 ovh master rados/singleton/{all/reg11184.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524248 2018-05-12 02:00:48 2018-05-12 17:22:56 2018-05-12 17:38:55 0:15:59 0:11:56 0:04:03 ovh master rados/objectstore/alloc-hint.yaml 1
pass 2524249 2018-05-12 02:00:49 2018-05-12 17:24:23 2018-05-12 17:42:22 0:17:59 0:10:55 0:07:04 ovh master rados/objectstore/alloc-hint.yaml 1
pass 2524250 2018-05-12 02:00:49 2018-05-12 17:26:30 2018-05-12 18:38:31 1:12:01 1:06:30 0:05:31 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/16113.yaml rados.yaml} 1
pass 2524251 2018-05-12 02:00:50 2018-05-12 17:26:56 2018-05-12 18:38:57 1:12:01 1:07:09 0:04:52 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/16113.yaml rados.yaml} 1
fail 2524252 2018-05-12 02:00:50 2018-05-12 17:28:45 2018-05-12 18:50:46 1:22:01 1:02:39 0:19:22 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 18:03:43.446566 osd.5 158.69.72.27:6808/8802 54 : cluster [WRN] 7 slow requests, 5 included below; oldest blocked for > 30.861224 secs" in cluster log

fail 2524253 2018-05-12 02:00:50 2018-05-12 17:29:45 2018-05-12 18:19:45 0:50:00 0:31:08 0:18:52 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 18:13:02.804775 osd.5 158.69.72.255:6808/8768 105 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.919640 secs" in cluster log

fail 2524254 2018-05-12 02:00:51 2018-05-12 17:32:52 2018-05-12 18:28:52 0:56:00 0:42:31 0:13:29 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 18:10:09.095791 osd.5 158.69.72.252:6808/8851 141 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.334708 secs" in cluster log

fail 2524255 2018-05-12 02:00:51 2018-05-12 17:39:05 2018-05-12 18:27:05 0:48:00 0:39:31 0:08:29 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 18:18:07.268992 osd.1 158.69.72.254:6804/11622 99 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.089675 secs" in cluster log

pass 2524256 2018-05-12 02:00:51 2018-05-12 17:40:15 2018-05-12 18:52:16 1:12:01 0:16:14 0:55:47 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524257 2018-05-12 02:00:52 2018-05-12 17:41:53 2018-05-12 18:27:53 0:46:00 0:15:53 0:30:07 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
fail 2524258 2018-05-12 02:00:52 2018-05-12 17:42:23 2018-05-12 18:24:23 0:42:00 0:33:51 0:08:09 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/default.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 18:17:29.170670 osd.0 158.69.72.40:6812/16870 193 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.870514 secs" in cluster log

fail 2524259 2018-05-12 02:00:52 2018-05-12 17:44:44 2018-05-12 19:18:45 1:34:01 1:11:35 0:22:26 ovh master centos rados/upgrade/{hammer-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-hammer-install/hammer.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-mon/mona.yaml 5-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 6-next-mon/monb.yaml 7-workload/{radosbench.yaml rbd_api.yaml} 8-next-mon/monc.yaml 9-workload/{ec-rados-plugin=jerasure-k=3-m=1.yaml rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml test_cache-pool-snaps.yaml}} rados.yaml} 3
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op write 100 --op delete 50 --pool unique_pool_8'

pass 2524260 2018-05-12 02:00:53 2018-05-12 17:44:47 2018-05-12 18:40:47 0:56:00 0:15:49 0:40:11 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 2524261 2018-05-12 02:00:53 2018-05-12 17:54:18 2018-05-12 18:42:18 0:48:00 0:38:43 0:09:17 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
fail 2524262 2018-05-12 02:00:53 2018-05-12 17:55:29 2018-05-12 20:03:31 2:08:02 1:51:36 0:16:26 ovh master centos rados/upgrade/{hammer-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-hammer-install/hammer.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-mon/mona.yaml 5-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 6-next-mon/monb.yaml 7-workload/{radosbench.yaml rbd_api.yaml} 8-next-mon/monc.yaml 9-workload/{ec-rados-plugin=jerasure-k=3-m=1.yaml rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml test_cache-pool-snaps.yaml}} rados.yaml} 3
Failure Reason:

"2018-05-12 18:32:38.577948 osd.0 158.69.73.108:6800/31809 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.120095 secs" in cluster log

pass 2524263 2018-05-12 02:00:54 2018-05-12 17:55:44 2018-05-12 18:41:44 0:46:00 0:24:56 0:21:04 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rgw_snaps.yaml} 2
pass 2524264 2018-05-12 02:00:54 2018-05-12 18:02:30 2018-05-12 18:40:30 0:38:00 0:25:06 0:12:54 ovh master rados/verify/{1thrash/none.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} 2
pass 2524265 2018-05-12 02:00:55 2018-05-12 18:10:41 2018-05-12 18:28:40 0:17:59 0:13:12 0:04:47 ovh master rados/singleton/{all/rest-api.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524266 2018-05-12 02:00:55 2018-05-12 18:12:10 2018-05-12 19:38:11 1:26:01 1:17:18 0:08:43 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 18:31:09.430456 osd.5 158.69.73.138:6808/8812 3 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.854066 secs" in cluster log

pass 2524267 2018-05-12 02:00:55 2018-05-12 18:15:56 2018-05-12 18:47:56 0:32:00 0:24:05 0:07:55 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
fail 2524268 2018-05-12 02:00:56 2018-05-12 18:19:55 2018-05-12 18:37:55 0:18:00 0:13:34 0:04:26 ovh master rados/singleton/{all/osd-recovery.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
Failure Reason:

"2018-05-12 18:35:13.716132 osd.1 158.69.73.150:6812/9457 1 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.906536 secs" in cluster log

fail 2524269 2018-05-12 02:00:56 2018-05-12 18:24:42 2018-05-12 19:14:43 0:50:01 0:42:38 0:07:23 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-12 18:51:25.155447 osd.4 158.69.73.156:6804/8801 82 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.372215 secs" in cluster log

fail 2524270 2018-05-12 02:00:56 2018-05-12 18:27:14 2018-05-12 19:33:15 1:06:01 0:58:14 0:07:47 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 18:50:17.930453 osd.5 158.69.73.164:6808/8795 13 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.531572 secs" in cluster log

pass 2524271 2018-05-12 02:00:57 2018-05-12 18:28:03 2018-05-12 19:08:02 0:39:59 0:30:03 0:09:56 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524272 2018-05-12 02:00:57 2018-05-12 18:28:51 2018-05-12 19:26:52 0:58:01 0:46:29 0:11:32 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-12 18:57:03.228283 osd.5 158.69.73.183:6808/8744 46 : cluster [WRN] 16 slow requests, 5 included below; oldest blocked for > 30.169189 secs" in cluster log

pass 2524273 2018-05-12 02:00:57 2018-05-12 18:28:53 2018-05-12 19:02:53 0:34:00 0:23:24 0:10:36 ovh master rados/singleton/{all/thrash-rados.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
pass 2524274 2018-05-12 02:00:58 2018-05-12 18:35:08 2018-05-12 19:39:08 1:04:00 0:52:10 0:11:50 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524275 2018-05-12 02:00:58 2018-05-12 18:38:12 2018-05-12 19:06:11 0:27:59 0:20:06 0:07:53 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml} 2
pass 2524276 2018-05-12 02:00:58 2018-05-12 18:38:32 2018-05-12 19:56:33 1:18:01 1:11:28 0:06:33 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
fail 2524277 2018-05-12 02:00:59 2018-05-12 18:38:58 2018-05-12 19:08:58 0:30:00 0:23:38 0:06:22 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-small-objects-fast-read.yaml} 2
Failure Reason:

"2018-05-12 18:56:26.790613 osd.0 158.69.73.188:6800/8724 5 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.747584 secs" in cluster log

pass 2524278 2018-05-12 02:00:59 2018-05-12 18:40:47 2018-05-12 18:56:47 0:16:00 0:10:51 0:05:09 ovh master rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524279 2018-05-12 02:01:00 2018-05-12 18:40:48 2018-05-12 19:10:48 0:30:00 0:16:42 0:13:18 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524280 2018-05-12 02:01:00 2018-05-12 18:42:02 2018-05-12 19:18:02 0:36:00 0:25:37 0:10:23 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 2524281 2018-05-12 02:01:00 2018-05-12 18:42:20 2018-05-12 19:08:19 0:25:59 0:16:35 0:09:24 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/scrub_test.yaml} 2
fail 2524282 2018-05-12 02:01:01 2018-05-12 18:48:13 2018-05-12 19:26:13 0:38:00 0:29:33 0:08:27 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 19:10:25.171880 osd.3 158.69.73.222:6800/8684 35 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.920697 secs" in cluster log

pass 2524283 2018-05-12 02:01:01 2018-05-12 18:50:56 2018-05-12 19:28:56 0:38:00 0:14:59 0:23:01 ovh master rados/multimon/{clusters/9.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 3
pass 2524284 2018-05-12 02:01:01 2018-05-12 18:52:34 2018-05-12 19:34:34 0:42:00 0:25:24 0:16:36 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 2524285 2018-05-12 02:01:02 2018-05-12 18:56:56 2018-05-12 19:22:56 0:26:00 0:16:08 0:09:52 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 2524286 2018-05-12 02:01:02 2018-05-12 19:01:28 2018-05-12 19:15:28 0:14:00 0:10:31 0:03:29 ovh master rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524287 2018-05-12 02:01:03 2018-05-12 19:03:03 2018-05-12 20:41:05 1:38:02 1:27:56 0:10:06 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 2524288 2018-05-12 02:01:03 2018-05-12 19:06:29 2018-05-12 19:20:28 0:13:59 0:10:52 0:03:07 ovh master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml} 1
pass 2524289 2018-05-12 02:01:03 2018-05-12 19:08:21 2018-05-12 19:22:20 0:13:59 0:10:34 0:03:25 ovh master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml} 1
pass 2524290 2018-05-12 02:01:04 2018-05-12 19:08:21 2018-05-12 19:38:21 0:30:00 0:23:03 0:06:57 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rgw_snaps.yaml} 2
dead 2524291 2018-05-12 02:01:04 2018-05-12 19:08:59 2018-05-13 07:11:37 12:02:38 ovh master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
pass 2524292 2018-05-12 02:01:04 2018-05-12 19:11:00 2018-05-12 19:37:00 0:26:00 0:21:42 0:04:18 ovh master rados/objectstore/ceph_objectstore_tool.yaml 1
fail 2524293 2018-05-12 02:01:05 2018-05-12 19:15:04 2018-05-12 20:15:04 1:00:00 0:51:58 0:08:02 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-12 19:43:52.404892 osd.5 158.69.73.59:6808/8738 39 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.994256 secs" in cluster log

fail 2524294 2018-05-12 02:01:05 2018-05-12 19:15:29 2018-05-12 19:47:29 0:32:00 0:23:06 0:08:54 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-12 19:36:07.124436 osd.0 158.69.73.68:6800/8726 7 : cluster [WRN] 24 slow requests, 5 included below; oldest blocked for > 30.572459 secs" in cluster log

pass 2524295 2018-05-12 02:01:06 2018-05-12 19:18:19 2018-05-12 19:46:19 0:28:00 0:22:20 0:05:40 ovh master rados/objectstore/ceph_objectstore_tool.yaml 1
pass 2524296 2018-05-12 02:01:06 2018-05-12 19:18:46 2018-05-12 20:10:46 0:52:00 0:39:52 0:12:08 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
fail 2524297 2018-05-12 02:01:06 2018-05-12 19:20:47 2018-05-12 20:34:48 1:14:01 1:06:19 0:07:42 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 19:38:25.675330 osd.5 158.69.73.82:6808/8802 1 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.867523 secs" in cluster log

pass 2524298 2018-05-12 02:01:07 2018-05-12 19:22:39 2018-05-12 19:44:38 0:21:59 0:15:35 0:06:24 ovh master rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524299 2018-05-12 02:01:07 2018-05-12 19:22:57 2018-05-12 19:48:57 0:26:00 0:15:15 0:10:45 ovh master rados/verify/{1thrash/default.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} 2
fail 2524300 2018-05-12 02:01:07 2018-05-12 19:26:31 2018-05-12 19:50:30 0:23:59 0:14:28 0:09:31 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync.yaml workloads/rados_5925.yaml} 2
Failure Reason:

"2018-05-12 19:44:43.519705 osd.0 158.69.73.93:6800/8698 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.024469 secs" in cluster log

fail 2524301 2018-05-12 02:01:08 2018-05-12 19:26:53 2018-05-12 20:34:53 1:08:00 1:00:20 0:07:40 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 19:46:02.758526 osd.5 158.69.73.90:6808/8837 7 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.455930 secs" in cluster log

fail 2524302 2018-05-12 02:01:08 2018-05-12 19:29:13 2018-05-12 20:39:13 1:10:00 0:44:31 0:25:29 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2018-05-12 19:51:12.418048 osd.3 158.69.74.0:6800/8612 1 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.229126 secs" in cluster log

pass 2524303 2018-05-12 02:01:09 2018-05-12 19:33:25 2018-05-12 19:57:24 0:23:59 0:10:45 0:13:14 ovh master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524304 2018-05-12 02:01:09 2018-05-12 19:33:37 2018-05-12 20:07:38 0:34:01 0:16:29 0:17:32 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524305 2018-05-12 02:01:09 2018-05-12 19:34:53 2018-05-12 20:28:53 0:54:00 0:45:51 0:08:09 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 19:56:36.172052 osd.5 158.69.74.116:6808/8769 1 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.935822 secs" in cluster log

pass 2524306 2018-05-12 02:01:10 2018-05-12 19:37:09 2018-05-12 20:11:09 0:34:00 0:27:05 0:06:55 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524307 2018-05-12 02:01:10 2018-05-12 19:38:28 2018-05-12 20:32:29 0:54:01 0:40:46 0:13:15 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-12 20:15:38.972800 osd.2 158.69.74.142:6808/8867 22 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.831804 secs" in cluster log

pass 2524308 2018-05-12 02:01:11 2018-05-12 19:38:28 2018-05-12 20:12:28 0:34:00 0:20:36 0:13:24 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/repair_test.yaml} 2
pass 2524309 2018-05-12 02:01:11 2018-05-12 19:39:09 2018-05-12 20:51:10 1:12:01 0:49:28 0:22:33 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524310 2018-05-12 02:01:12 2018-05-12 19:44:47 2018-05-12 20:08:47 0:24:00 0:14:55 0:09:05 ovh master rados/multimon/{clusters/6.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml} 2
pass 2524311 2018-05-12 02:01:12 2018-05-12 19:46:36 2018-05-12 20:36:37 0:50:01 0:25:08 0:24:53 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml} 2
pass 2524312 2018-05-12 02:01:12 2018-05-12 19:47:46 2018-05-12 20:13:46 0:26:00 0:15:55 0:10:05 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 2524313 2018-05-12 02:01:13 2018-05-12 19:49:06 2018-05-12 20:07:06 0:18:00 0:10:50 0:07:10 ovh master rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524314 2018-05-12 02:01:13 2018-05-12 19:50:48 2018-05-12 20:12:48 0:22:00 0:12:42 0:09:18 ovh master rados/singleton/{all/reg11184.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524315 2018-05-12 02:01:13 2018-05-12 19:55:11 2018-05-12 20:43:10 0:47:59 0:40:32 0:07:27 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 20:16:49.388930 osd.5 158.69.74.174:6808/26883 35 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.634206 secs" in cluster log

fail 2524316 2018-05-12 02:01:14 2018-05-12 19:56:50 2018-05-12 21:00:50 1:04:00 0:54:53 0:09:07 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-12 20:17:15.244140 osd.5 158.69.74.187:6808/8790 5 : cluster [WRN] 8 slow requests, 5 included below; oldest blocked for > 30.457202 secs" in cluster log

pass 2524317 2018-05-12 02:01:14 2018-05-12 19:57:25 2018-05-12 20:29:25 0:32:00 0:23:07 0:08:53 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
fail 2524318 2018-05-12 02:01:15 2018-05-12 19:59:02 2018-05-12 20:45:03 0:46:01 0:34:43 0:11:18 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 20:24:15.014532 osd.3 158.69.74.194:6812/28351 37 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.555706 secs" in cluster log

pass 2524319 2018-05-12 02:01:15 2018-05-12 20:03:42 2018-05-12 20:17:41 0:13:59 0:09:04 0:04:55 ovh master rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml} 1
fail 2524320 2018-05-12 02:01:15 2018-05-12 20:07:15 2018-05-12 21:07:15 1:00:00 0:51:45 0:08:15 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-12 20:28:58.639578 osd.2 158.69.74.195:6808/8802 5 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.052426 secs" in cluster log

fail 2524321 2018-05-12 02:01:15 2018-05-12 20:07:42 2018-05-12 20:45:41 0:37:59 0:31:06 0:06:53 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 20:26:19.371504 osd.4 158.69.74.200:6804/8671 15 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.337406 secs" in cluster log

pass 2524322 2018-05-12 02:01:16 2018-05-12 20:09:05 2018-05-12 20:21:05 0:12:00 0:09:04 0:02:56 ovh master rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml} 1
fail 2524323 2018-05-12 02:01:16 2018-05-12 20:10:58 2018-05-12 20:54:58 0:44:00 0:35:10 0:08:50 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2018-05-12 20:48:00.988821 osd.3 158.69.74.231:6800/15114 111 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.751033 secs" in cluster log

fail 2524324 2018-05-12 02:01:17 2018-05-12 20:11:10 2018-05-12 21:23:11 1:12:01 1:02:17 0:09:44 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 20:34:01.479923 osd.5 158.69.74.225:6808/8788 16 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.003563 secs" in cluster log

pass 2524325 2018-05-12 02:01:17 2018-05-12 20:12:45 2018-05-12 21:04:45 0:52:00 0:43:19 0:08:41 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/many.yaml workloads/snaps-few-objects.yaml} 2
pass 2524326 2018-05-12 02:01:18 2018-05-12 20:12:49 2018-05-12 20:30:48 0:17:59 0:14:09 0:03:50 ovh master rados/singleton/{all/rest-api.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524327 2018-05-12 02:01:18 2018-05-12 20:14:03 2018-05-12 20:50:03 0:36:00 0:23:38 0:12:22 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-small-objects.yaml} 2
Failure Reason:

"2018-05-12 20:38:10.434088 osd.1 158.69.74.255:6804/8762 1 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.417784 secs" in cluster log

fail 2524328 2018-05-12 02:01:18 2018-05-12 20:14:03 2018-05-12 21:52:04 1:38:01 1:15:13 0:22:48 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 20:47:46.714495 osd.5 158.69.74.45:6808/8796 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.311366 secs" in cluster log

pass 2524329 2018-05-12 02:01:18 2018-05-12 20:15:23 2018-05-12 21:35:24 1:20:01 0:15:26 1:04:35 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524330 2018-05-12 02:01:19 2018-05-12 20:17:51 2018-05-12 21:35:52 1:18:01 1:09:36 0:08:25 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 20:39:26.212556 osd.5 158.69.74.244:6808/9012 15 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.661642 secs" in cluster log

pass 2524331 2018-05-12 02:01:19 2018-05-12 20:21:14 2018-05-12 20:51:14 0:30:00 0:25:36 0:04:24 ovh master rados/singleton/{all/cephtool.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524332 2018-05-12 02:01:20 2018-05-12 20:21:14 2018-05-12 20:37:14 0:16:00 0:09:43 0:06:17 ovh master rados/objectstore/filejournal.yaml 1
pass 2524333 2018-05-12 02:01:20 2018-05-12 20:29:02 2018-05-12 21:01:02 0:32:00 0:22:50 0:09:10 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/rgw_snaps.yaml} 2
pass 2524334 2018-05-12 02:01:21 2018-05-12 20:29:26 2018-05-12 21:13:26 0:44:00 0:33:43 0:10:17 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 2524335 2018-05-12 02:01:21 2018-05-12 20:31:06 2018-05-12 20:45:06 0:14:00 0:10:11 0:03:49 ovh master rados/objectstore/filejournal.yaml 1
pass 2524336 2018-05-12 02:01:21 2018-05-12 20:32:47 2018-05-12 21:06:47 0:34:00 0:24:55 0:09:05 ovh master rados/verify/{1thrash/default.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
fail 2524337 2018-05-12 02:01:21 2018-05-12 20:34:57 2018-05-12 21:14:57 0:40:00 0:23:29 0:16:31 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-12 20:57:16.753436 osd.3 158.69.74.60:6800/8635 23 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.867408 secs" in cluster log

pass 2524338 2018-05-12 02:01:22 2018-05-12 20:34:57 2018-05-12 21:04:57 0:30:00 0:22:31 0:07:29 ovh master rados/singleton/{all/thrash-rados.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
pass 2524339 2018-05-12 02:01:22 2018-05-12 20:36:55 2018-05-12 21:30:55 0:54:00 0:38:52 0:15:08 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 2524340 2018-05-12 02:01:23 2018-05-12 20:37:15 2018-05-12 21:39:15 1:02:00 0:36:05 0:25:55 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524341 2018-05-12 02:01:23 2018-05-12 20:39:23 2018-05-12 20:57:23 0:18:00 0:12:53 0:05:07 ovh master rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524342 2018-05-12 02:01:23 2018-05-12 20:41:15 2018-05-12 21:07:14 0:25:59 0:15:33 0:10:26 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/scrub_test.yaml} 2
pass 2524343 2018-05-12 02:01:24 2018-05-12 20:43:20 2018-05-12 21:25:20 0:42:00 0:33:39 0:08:21 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524344 2018-05-12 02:01:24 2018-05-12 20:45:20 2018-05-12 21:23:20 0:38:00 0:30:32 0:07:28 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-05-12 21:12:35.722599 osd.3 158.69.75.111:6800/8726 231 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.636717 secs" in cluster log

pass 2524345 2018-05-12 02:01:24 2018-05-12 20:45:20 2018-05-12 21:09:20 0:24:00 0:13:46 0:10:14 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml} 2
fail 2524346 2018-05-12 02:01:25 2018-05-12 20:45:42 2018-05-12 21:25:43 0:40:01 0:30:04 0:09:57 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 21:10:14.444245 osd.3 158.69.75.119:6800/8656 23 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.533665 secs" in cluster log

pass 2524347 2018-05-12 02:01:25 2018-05-12 20:50:13 2018-05-12 21:32:13 0:42:00 0:28:29 0:13:31 ovh master rados/multimon/{clusters/21.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml} 3
pass 2524348 2018-05-12 02:01:26 2018-05-12 20:51:28 2018-05-12 22:21:29 1:30:01 1:20:35 0:09:26 ovh master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
fail 2524349 2018-05-12 02:01:26 2018-05-12 20:51:28 2018-05-12 21:17:28 0:26:00 0:15:05 0:10:55 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-12 21:12:25.152573 osd.0 158.69.75.128:6800/8761 3 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.480347 secs" in cluster log

fail 2524350 2018-05-12 02:01:26 2018-05-12 20:55:07 2018-05-12 21:31:07 0:36:00 0:27:45 0:08:15 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-05-12 21:13:03.718801 osd.3 158.69.75.135:6800/8756 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.293689 secs" in cluster log

pass 2524351 2018-05-12 02:01:27 2018-05-12 20:55:08 2018-05-12 21:11:08 0:16:00 0:11:59 0:04:01 ovh master rados/singleton/{all/divergent_priors2.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524352 2018-05-12 02:01:27 2018-05-12 20:57:33 2018-05-12 21:37:33 0:40:00 0:29:35 0:10:25 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2018-05-12 21:28:34.863414 osd.1 158.69.75.144:6804/8904 47 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.749394 secs" in cluster log

fail 2524353 2018-05-12 02:01:27 2018-05-12 21:01:06 2018-05-12 23:13:08 2:12:02 2:04:08 0:07:54 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-12 21:21:38.141397 osd.5 158.69.75.153:6808/8807 16 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.071959 secs" in cluster log

pass 2524354 2018-05-12 02:01:28 2018-05-12 21:01:06 2018-05-12 21:35:06 0:34:00 0:15:39 0:18:21 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524355 2018-05-12 02:01:28 2018-05-12 21:05:03 2018-05-12 21:21:03 0:16:00 0:10:58 0:05:02 ovh master rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml} 1
fail 2524356 2018-05-12 02:01:29 2018-05-12 21:05:03 2018-05-12 21:45:03 0:40:00 0:32:31 0:07:29 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2018-05-12 21:31:14.459748 osd.3 158.69.75.199:6801/10542 9 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.836621 secs" in cluster log

fail 2524357 2018-05-12 02:01:29 2018-05-12 21:07:04 2018-05-12 22:05:05 0:58:01 0:49:41 0:08:20 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op copy_from 50 --op write_excl 50 --op delete 50 --pool base'

pass 2524358 2018-05-12 02:01:29 2018-05-12 21:07:15 2018-05-12 21:23:15 0:16:00 0:10:34 0:05:26 ovh master rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml} 1
fail 2524359 2018-05-12 02:01:29 2018-05-12 21:07:17 2018-05-12 22:39:18 1:32:01 1:07:43 0:24:18 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 21:40:00.401166 osd.3 158.69.75.240:6800/8547 1 : cluster [WRN] 6 slow requests, 5 included below; oldest blocked for > 30.407297 secs" in cluster log

pass 2524360 2018-05-12 02:01:30 2018-05-12 21:07:48 2018-05-12 21:25:47 0:17:59 0:11:55 0:06:04 ovh master rados/singleton/{all/dump-stuck.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524361 2018-05-12 02:01:30 2018-05-12 21:09:29 2018-05-12 21:39:29 0:30:00 0:22:04 0:07:56 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rgw_snaps.yaml} 2
fail 2524362 2018-05-12 02:01:31 2018-05-12 21:11:26 2018-05-12 22:35:27 1:24:01 1:16:30 0:07:31 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 21:30:49.812870 osd.2 158.69.75.224:6808/8942 1 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.723460 secs" in cluster log

fail 2524363 2018-05-12 02:01:31 2018-05-12 21:13:37 2018-05-12 21:47:36 0:33:59 0:24:44 0:09:15 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-12 21:33:13.347540 osd.0 158.69.75.227:6800/9914 1 : cluster [WRN] 14 slow requests, 5 included below; oldest blocked for > 30.028202 secs" in cluster log

pass 2524364 2018-05-12 02:01:32 2018-05-12 21:15:15 2018-05-12 21:39:15 0:24:00 0:11:47 0:12:13 ovh master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524365 2018-05-12 02:01:32 2018-05-12 21:17:37 2018-05-12 22:27:38 1:10:01 0:57:42 0:12:19 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 21:45:41.217299 osd.5 158.69.75.245:6808/8836 20 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.146714 secs" in cluster log

fail 2524366 2018-05-12 02:01:32 2018-05-12 21:21:22 2018-05-12 22:17:22 0:56:00 0:47:53 0:08:07 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-12 21:53:48.733404 osd.0 158.69.75.242:6805/16593 24 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.077529 secs" in cluster log

pass 2524367 2018-05-12 02:01:32 2018-05-12 21:23:21 2018-05-12 22:13:22 0:50:01 0:40:09 0:09:52 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/many.yaml workloads/pool-create-delete.yaml} 2
fail 2524368 2018-05-12 02:01:33 2018-05-12 21:23:21 2018-05-12 22:07:21 0:44:00 0:34:29 0:09:31 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2018-05-12 22:02:49.639707 osd.1 158.69.75.249:6804/8780 126 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.452211 secs" in cluster log

pass 2524369 2018-05-12 02:01:33 2018-05-12 21:23:21 2018-05-12 22:01:21 0:38:00 0:33:03 0:04:57 ovh master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
pass 2524370 2018-05-12 02:01:34 2018-05-12 21:25:29 2018-05-12 22:01:29 0:36:00 0:24:02 0:11:58 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml} 2
fail 2524371 2018-05-12 02:01:34 2018-05-12 21:25:52 2018-05-12 22:15:52 0:50:00 0:38:30 0:11:30 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-12 21:46:34.237753 osd.0 158.69.75.31:6800/8778 8 : cluster [WRN] 15 slow requests, 5 included below; oldest blocked for > 30.317467 secs" in cluster log

pass 2524372 2018-05-12 02:01:35 2018-05-12 21:25:52 2018-05-12 21:55:51 0:29:59 0:12:08 0:17:51 ovh master rados/multimon/{clusters/9.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 3
fail 2524373 2018-05-12 02:01:35 2018-05-12 21:31:12 2018-05-12 21:55:12 0:24:00 0:15:36 0:08:24 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op setattr 25 --op read 100 --op copy_from 50 --op write 0 --op rmattr 25 --op append 100 --op delete 50 --pool unique_pool_0'

pass 2524374 2018-05-12 02:01:35 2018-05-12 21:31:12 2018-05-12 22:07:12 0:36:00 0:31:11 0:04:49 ovh master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
pass 2524375 2018-05-12 02:01:35 2018-05-12 21:32:30 2018-05-12 22:06:30 0:34:00 0:16:32 0:17:28 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524376 2018-05-12 02:01:36 2018-05-12 21:35:21 2018-05-12 21:49:20 0:13:59 0:10:09 0:03:50 ovh master rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
dead 2524377 2018-05-12 02:01:36 2018-05-12 21:35:25 2018-05-13 09:38:01 12:02:36 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/default.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 2524378 2018-05-12 02:01:37 2018-05-12 21:35:54 2018-05-12 22:01:53 0:25:59 0:18:00 0:07:59 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-12 21:54:19.774511 osd.1 158.69.75.51:6804/8473 7 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.974530 secs" in cluster log

pass 2524379 2018-05-12 02:01:37 2018-05-12 21:37:42 2018-05-12 22:09:41 0:31:59 0:16:33 0:15:26 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
fail 2524380 2018-05-12 02:01:38 2018-05-12 21:39:24 2018-05-12 22:53:24 1:14:00 0:37:46 0:36:14 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 22:13:33.968886 osd.1 158.69.75.106:6804/11646 43 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.931599 secs" in cluster log

pass 2524381 2018-05-12 02:01:38 2018-05-12 21:39:24 2018-05-12 22:17:24 0:38:00 0:25:23 0:12:37 ovh master rados/verify/{1thrash/none.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} 2
pass 2524383 2018-05-12 02:01:38 2018-05-12 21:39:30 2018-05-12 22:33:30 0:54:00 0:16:41 0:37:19 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 2524384 2018-05-12 02:01:38 2018-05-12 21:45:22 2018-05-12 22:07:22 0:22:00 0:13:15 0:08:45 ovh master centos rados/singleton/{all/ec-lost-unfound-upgrade.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
fail 2524385 2018-05-12 02:01:39 2018-05-12 21:47:45 2018-05-12 22:59:46 1:12:01 0:55:29 0:16:32 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-12 22:09:35.703784 osd.3 158.69.75.102:6800/9510 11 : cluster [WRN] 6 slow requests, 5 included below; oldest blocked for > 30.372260 secs" in cluster log

fail 2524386 2018-05-12 02:01:39 2018-05-12 21:49:30 2018-05-12 22:51:30 1:02:00 0:53:13 0:08:47 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-12 22:43:39.529252 osd.4 158.69.76.102:6804/8594 372 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.488231 secs" in cluster log

fail 2524388 2018-05-12 02:01:40 2018-05-12 21:52:23 2018-05-12 22:58:23 1:06:00 0:55:01 0:10:59 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-12 22:16:55.274900 osd.1 158.69.76.115:6812/10471 13 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.827025 secs" in cluster log

pass 2524389 2018-05-12 02:01:40 2018-05-12 21:54:09 2018-05-12 22:18:08 0:23:59 0:16:29 0:07:30 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_python.yaml} 2
fail 2524391 2018-05-12 02:01:41 2018-05-12 21:55:30 2018-05-12 22:13:30 0:18:00 0:12:37 0:05:23 ovh master rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml} 1
Failure Reason:

"2018-05-12 22:08:44.902723 osd.1 158.69.76.113:6804/8634 2 : cluster [WRN] 16 slow requests, 5 included below; oldest blocked for > 30.988580 secs" in cluster log

pass 2524392 2018-05-12 02:01:41 2018-05-12 21:55:52 2018-05-12 22:33:52 0:38:00 0:25:46 0:12:14 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
pass 2524394 2018-05-12 02:01:41 2018-05-12 22:01:31 2018-05-12 22:31:30 0:29:59 0:23:55 0:06:04 ovh master rados/singleton/{all/cephtool.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524396 2018-05-12 02:01:41 2018-05-12 22:01:31 2018-05-12 22:19:30 0:17:59 0:12:03 0:05:56 ovh master rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml} 1
Failure Reason:

"2018-05-12 22:15:48.072662 osd.1 158.69.76.15:6804/8715 2 : cluster [WRN] 16 slow requests, 5 included below; oldest blocked for > 30.929594 secs" in cluster log

fail 2524397 2018-05-12 02:01:42 2018-05-12 22:01:55 2018-05-12 23:11:56 1:10:01 0:59:55 0:10:06 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 22:23:57.012541 osd.5 158.69.76.152:6808/8774 1 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.343642 secs" in cluster log

fail 2524399 2018-05-12 02:01:42 2018-05-12 22:05:14 2018-05-12 22:41:14 0:36:00 0:28:16 0:07:44 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 22:21:59.856697 osd.1 158.69.76.155:6804/8833 3 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.834146 secs" in cluster log

fail 2524400 2018-05-12 02:01:43 2018-05-12 22:06:49 2018-05-12 23:16:50 1:10:01 1:01:25 0:08:36 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 22:27:17.263672 osd.5 158.69.76.175:6808/8913 7 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.386921 secs" in cluster log

pass 2524402 2018-05-12 02:01:43 2018-05-12 22:46:07 2018-05-12 23:20:06 0:33:59 0:29:21 0:04:38 ovh master rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524403 2018-05-12 02:01:43 2018-05-12 22:46:07 2018-05-12 23:50:07 1:04:00 0:57:05 0:06:55 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 23:09:58.403258 osd.0 158.69.77.119:6800/8775 23 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.814896 secs" in cluster log

pass 2524405 2018-05-12 02:01:43 2018-05-12 22:47:53 2018-05-12 23:21:53 0:34:00 0:26:20 0:07:40 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 2524406 2018-05-12 02:01:44 2018-05-12 22:48:19 2018-05-12 23:16:18 0:27:59 0:20:46 0:07:13 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/many.yaml workloads/rados_mon_workunits.yaml} 2
pass 2524408 2018-05-12 02:01:44 2018-05-12 22:49:50 2018-05-12 23:19:50 0:30:00 0:21:03 0:08:57 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/rgw_snaps.yaml} 2
pass 2524410 2018-05-12 02:01:45 2018-05-12 22:49:51 2018-05-12 23:21:51 0:32:00 0:22:47 0:09:13 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-small-objects-fast-read.yaml} 2
fail 2524411 2018-05-12 02:01:45 2018-05-12 22:51:29 2018-05-12 23:35:29 0:44:00 0:22:55 0:21:05 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-12 23:07:43.435387 osd.1 158.69.77.132:6804/8885 1 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.076142 secs" in cluster log

pass 2524412 2018-05-12 02:01:45 2018-05-12 22:51:31 2018-05-12 23:27:31 0:36:00 0:15:49 0:20:11 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524414 2018-05-12 02:01:46 2018-05-12 22:53:33 2018-05-12 23:15:33 0:22:00 0:14:50 0:07:10 ovh master centos rados/singleton/{all/lost-unfound-delete-upgrade.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
Failure Reason:

"2018-05-12 23:12:19.930748 osd.0 158.69.77.145:6800/32660 21 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.264458 secs" in cluster log

pass 2524415 2018-05-12 02:01:46 2018-05-12 22:53:34 2018-05-12 23:11:33 0:17:59 0:12:48 0:05:11 ovh master rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524417 2018-05-12 02:01:46 2018-05-12 22:55:33 2018-05-12 23:35:33 0:40:00 0:33:34 0:06:26 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-12 23:18:53.834364 osd.0 158.69.77.150:6800/8795 84 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.104422 secs" in cluster log

pass 2524419 2018-05-12 02:01:47 2018-05-12 22:55:33 2018-05-12 23:39:33 0:44:00 0:35:45 0:08:15 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 2524420 2018-05-12 02:01:47 2018-05-12 22:55:33 2018-05-12 23:37:33 0:42:00 0:34:04 0:07:56 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524421 2018-05-12 02:01:48 2018-05-12 22:55:54 2018-05-12 23:19:54 0:24:00 0:14:28 0:09:32 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool unique_pool_0'

pass 2524423 2018-05-12 02:01:48 2018-05-12 22:57:27 2018-05-12 23:29:27 0:32:00 0:23:33 0:08:27 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_stress_watch.yaml} 2
pass 2524424 2018-05-12 02:01:48 2018-05-12 22:57:33 2018-05-12 23:25:33 0:28:00 0:13:18 0:14:42 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml} 2
pass 2524426 2018-05-12 02:01:48 2018-05-12 22:58:43 2018-05-12 23:14:43 0:16:00 0:10:32 0:05:28 ovh master rados/multimon/{clusters/3.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 1
pass 2524427 2018-05-12 02:01:49 2018-05-12 22:59:19 2018-05-12 23:35:19 0:36:00 0:23:11 0:12:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
pass 2524429 2018-05-12 02:01:49 2018-05-12 22:59:26 2018-05-12 23:43:26 0:44:00 0:34:34 0:09:26 ovh master rados/objectstore/filestore-idempotent.yaml 1
pass 2524430 2018-05-12 02:01:50 2018-05-12 22:59:29 2018-05-12 23:17:29 0:18:00 0:12:11 0:05:49 ovh master rados/singleton/{all/divergent_priors2.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524431 2018-05-12 02:01:50 2018-05-12 23:00:03 2018-05-12 23:32:03 0:32:00 0:24:36 0:07:24 ovh master rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524433 2018-05-12 02:01:51 2018-05-12 23:01:35 2018-05-12 23:41:35 0:40:00 0:34:56 0:05:04 ovh master rados/objectstore/filestore-idempotent.yaml 1
fail 2524434 2018-05-12 02:01:51 2018-05-12 23:03:42 2018-05-12 23:31:41 0:27:59 0:14:57 0:13:02 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-12 23:26:32.221438 osd.4 158.69.77.198:6804/8729 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.399946 secs" in cluster log

fail 2524436 2018-05-12 02:01:51 2018-05-12 23:03:43 2018-05-12 23:45:43 0:42:00 0:30:22 0:11:38 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-12 23:35:36.181163 osd.1 158.69.77.192:6815/17128 40 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.770336 secs" in cluster log

pass 2524437 2018-05-12 02:01:52 2018-05-12 23:05:30 2018-05-12 23:25:29 0:19:59 0:15:07 0:04:52 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/lfn-upgrade-hammer.yaml rados.yaml} 1
pass 2524439 2018-05-12 02:01:52 2018-05-12 23:09:42 2018-05-12 23:45:42 0:36:00 0:27:19 0:08:41 ovh master rados/verify/{1thrash/none.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
fail 2524440 2018-05-12 02:01:52 2018-05-12 23:11:43 2018-05-13 00:25:43 1:14:00 1:05:51 0:08:09 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-12 23:44:29.477324 osd.5 158.69.77.202:6812/9541 15 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.769454 secs" in cluster log

pass 2524442 2018-05-12 02:01:53 2018-05-12 23:11:58 2018-05-12 23:31:58 0:20:00 0:15:26 0:04:34 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/lfn-upgrade-hammer.yaml rados.yaml} 1
pass 2524443 2018-05-12 02:01:53 2018-05-12 23:12:22 2018-05-12 23:34:22 0:22:00 0:12:24 0:09:36 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/one.yaml workloads/rados_5925.yaml} 2
pass 2524445 2018-05-12 02:01:54 2018-05-12 23:13:17 2018-05-12 23:57:17 0:44:00 0:33:27 0:10:33 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 2524446 2018-05-12 02:01:54 2018-05-12 23:15:01 2018-05-13 00:03:06 0:48:05 0:39:41 0:08:24 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 2524448 2018-05-12 02:01:54 2018-05-12 23:15:34 2018-05-12 23:45:34 0:30:00 0:22:29 0:07:31 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/rgw_snaps.yaml} 2
fail 2524449 2018-05-12 02:01:54 2018-05-12 23:16:36 2018-05-12 23:48:36 0:32:00 0:15:24 0:16:36 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
Failure Reason:

"2018-05-12 23:44:36.604687 osd.1 158.69.77.236:6804/8615 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.472832 secs" in cluster log

pass 2524451 2018-05-12 02:01:55 2018-05-12 23:16:51 2018-05-12 23:32:51 0:16:00 0:12:22 0:03:38 ovh master rados/singleton/{all/dump-stuck.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524452 2018-05-12 02:01:55 2018-05-12 23:17:47 2018-05-13 00:11:47 0:54:00 0:47:15 0:06:45 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-12 23:37:31.213173 osd.2 158.69.77.237:6808/8823 18 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.465808 secs" in cluster log

pass 2524454 2018-05-12 02:01:56 2018-05-12 23:19:42 2018-05-12 23:51:42 0:32:00 0:24:28 0:07:32 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
fail 2524455 2018-05-12 02:01:56 2018-05-12 23:19:51 2018-05-12 23:41:51 0:22:00 0:15:08 0:06:52 ovh master centos rados/singleton/{all/lost-unfound-upgrade.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
Failure Reason:

"2018-05-12 23:38:32.024765 osd.0 158.69.77.241:6800/32136 35 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.173499 secs" in cluster log

fail 2524457 2018-05-12 02:01:56 2018-05-12 23:19:55 2018-05-12 23:51:55 0:32:00 0:21:56 0:10:04 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool unique_pool_0'

fail 2524458 2018-05-12 02:01:57 2018-05-12 23:20:09 2018-05-13 00:40:10 1:20:01 1:12:02 0:07:59 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-12 23:42:32.645368 osd.2 158.69.77.247:6808/8936 27 : cluster [WRN] 6 slow requests, 5 included below; oldest blocked for > 30.878667 secs" in cluster log

fail 2524460 2018-05-12 02:01:57 2018-05-12 23:22:11 2018-05-13 00:10:13 0:48:02 0:34:42 0:13:20 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op setattr 25 --op read 100 --op copy_from 50 --op write 0 --op rmattr 25 --op append 100 --op delete 50 --pool unique_pool_0'

fail 2524461 2018-05-12 02:01:57 2018-05-12 23:22:11 2018-05-13 00:44:14 1:22:03 1:13:33 0:08:30 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-12 23:41:56.602465 osd.5 158.69.77.28:6808/8844 1 : cluster [WRN] 10 slow requests, 5 included below; oldest blocked for > 30.646135 secs" in cluster log

pass 2524463 2018-05-12 02:01:58 2018-05-12 23:25:48 2018-05-13 00:03:48 0:38:00 0:31:15 0:06:45 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524464 2018-05-12 02:01:58 2018-05-12 23:25:48 2018-05-13 00:05:48 0:40:00 0:31:11 0:08:49 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-12 23:41:54.985900 osd.4 158.69.77.49:6804/26782 5 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.456345 secs" in cluster log

pass 2524466 2018-05-12 02:01:59 2018-05-12 23:25:57 2018-05-12 23:55:57 0:30:00 0:14:16 0:15:44 ovh master centos rados/singleton/{all/ec-lost-unfound-upgrade.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
fail 2524467 2018-05-12 02:01:59 2018-05-12 23:27:49 2018-05-13 00:27:49 1:00:00 0:53:11 0:06:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-12 23:45:14.828830 osd.5 158.69.77.5:6808/8780 3 : cluster [WRN] 9 slow requests, 5 included below; oldest blocked for > 30.864400 secs" in cluster log

pass 2524469 2018-05-12 02:01:59 2018-05-12 23:27:49 2018-05-12 23:53:48 0:25:59 0:20:52 0:05:07 ovh master rados/singleton/{all/lost-unfound.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524470 2018-05-12 02:01:59 2018-05-12 23:29:46 2018-05-13 00:09:46 0:40:00 0:16:36 0:23:24 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_python.yaml} 2
fail 2524472 2018-05-12 02:02:00 2018-05-12 23:29:54 2018-05-13 00:17:54 0:48:00 0:38:41 0:09:19 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-12 23:51:20.239239 osd.3 158.69.77.66:6800/8655 18 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.140049 secs" in cluster log

pass 2524473 2018-05-12 02:02:00 2018-05-12 23:31:58 2018-05-12 23:55:58 0:24:00 0:12:42 0:11:18 ovh master rados/multimon/{clusters/21.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 3
pass 2524475 2018-05-12 02:02:01 2018-05-12 23:31:59 2018-05-12 23:55:58 0:23:59 0:15:48 0:08:11 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 2524476 2018-05-12 02:02:01 2018-05-12 23:32:04 2018-05-13 00:36:04 1:04:00 0:55:16 0:08:44 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-12 23:59:35.657196 osd.3 158.69.77.71:6800/8691 25 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.902791 secs" in cluster log

pass 2524478 2018-05-12 02:02:02 2018-05-12 23:33:08 2018-05-13 00:41:09 1:08:01 0:59:47 0:08:14 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
pass 2524479 2018-05-12 02:02:02 2018-05-12 23:34:39 2018-05-13 00:20:39 0:46:00 0:36:01 0:09:59 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_big.yaml} 2
pass 2524481 2018-05-12 02:02:02 2018-05-12 23:35:20 2018-05-13 00:49:21 1:14:01 0:51:19 0:22:42 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/one.yaml workloads/snaps-few-objects.yaml} 2
fail 2524482 2018-05-12 02:02:02 2018-05-12 23:35:31 2018-05-13 00:13:31 0:38:00 0:27:02 0:10:58 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-05-13 00:04:09.164000 osd.4 158.69.77.96:6804/8672 118 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.181525 secs" in cluster log

fail 2524484 2018-05-12 02:02:03 2018-05-12 23:35:50 2018-05-13 00:05:50 0:30:00 0:22:58 0:07:02 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-small-objects.yaml} 2
Failure Reason:

"2018-05-13 00:02:49.601971 osd.4 158.69.77.10:6804/10805 23 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.127201 secs" in cluster log

pass 2524485 2018-05-12 02:02:03 2018-05-12 23:35:50 2018-05-12 23:57:50 0:22:00 0:14:54 0:07:06 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/lfn-upgrade-infernalis.yaml rados.yaml} 1
pass 2524487 2018-05-12 02:02:04 2018-05-12 23:37:50 2018-05-13 00:11:50 0:34:00 0:16:06 0:17:54 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524488 2018-05-12 02:02:04 2018-05-12 23:39:51 2018-05-13 00:03:51 0:24:00 0:18:01 0:05:59 ovh master rados/singleton/{all/mon-config-keys.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524490 2018-05-12 02:02:04 2018-05-12 23:41:52 2018-05-13 00:13:52 0:32:00 0:27:07 0:04:53 ovh master rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524491 2018-05-12 02:02:04 2018-05-12 23:41:52 2018-05-12 23:55:52 0:14:00 0:09:35 0:04:25 ovh master rados/objectstore/fusestore.yaml 1
fail 2524493 2018-05-12 02:02:05 2018-05-12 23:43:43 2018-05-13 00:31:44 0:48:01 0:40:14 0:07:47 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-13 00:02:38.535135 osd.5 158.69.78.112:6808/8791 36 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.137345 secs" in cluster log

fail 2524494 2018-05-12 02:02:05 2018-05-12 23:45:51 2018-05-13 00:19:51 0:34:00 0:26:10 0:07:50 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-13 00:11:01.319741 osd.4 158.69.78.121:6804/8640 61 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.275012 secs" in cluster log

pass 2524496 2018-05-12 02:02:06 2018-05-12 23:45:51 2018-05-13 00:07:51 0:22:00 0:16:11 0:05:49 ovh master ubuntu 14.04 rados/singleton-nomsgr/{all/lfn-upgrade-infernalis.yaml rados.yaml} 1
pass 2524497 2018-05-12 02:02:06 2018-05-12 23:45:51 2018-05-13 00:21:51 0:36:00 0:27:30 0:08:30 ovh master rados/verify/{1thrash/default.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
pass 2524499 2018-05-12 02:02:06 2018-05-12 23:48:53 2018-05-13 00:02:52 0:13:59 0:09:53 0:04:06 ovh master rados/objectstore/fusestore.yaml 1
fail 2524500 2018-05-12 02:02:06 2018-05-12 23:50:27 2018-05-13 00:32:27 0:42:00 0:35:32 0:06:28 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2018-05-13 00:08:42.939254 osd.3 158.69.78.124:6800/8645 3 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.172600 secs" in cluster log

fail 2524502 2018-05-12 02:02:07 2018-05-12 23:52:00 2018-05-13 01:24:01 1:32:01 1:14:12 0:17:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-13 00:31:44.577980 osd.3 158.69.78.144:6800/8597 37 : cluster [WRN] 15 slow requests, 5 included below; oldest blocked for > 30.492936 secs" in cluster log

pass 2524503 2018-05-12 02:02:07 2018-05-12 23:52:00 2018-05-13 00:26:00 0:34:00 0:22:40 0:11:20 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rgw_snaps.yaml} 2
fail 2524505 2018-05-12 02:02:08 2018-05-12 23:54:06 2018-05-13 01:14:07 1:20:01 1:12:01 0:08:00 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-13 00:12:32.929604 osd.1 158.69.78.137:6804/8540 1 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.608089 secs" in cluster log

pass 2524506 2018-05-12 02:02:08 2018-05-12 23:56:10 2018-05-13 00:22:10 0:26:00 0:21:47 0:04:13 ovh master rados/singleton/{all/mon-thrasher.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524508 2018-05-12 02:02:08 2018-05-12 23:56:10 2018-05-13 00:24:10 0:28:00 0:14:32 0:13:28 ovh master centos rados/singleton/{all/lost-unfound-delete-upgrade.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
Failure Reason:

"2018-05-13 00:19:55.325124 osd.0 158.69.78.179:6800/31761 37 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.879568 secs" in cluster log

fail 2524509 2018-05-12 02:02:08 2018-05-12 23:56:10 2018-05-13 00:26:10 0:30:00 0:23:02 0:06:58 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-13 00:16:46.161752 osd.5 158.69.78.145:6808/9363 29 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.736454 secs" in cluster log

fail 2524511 2018-05-12 02:02:09 2018-05-12 23:56:10 2018-05-13 00:46:10 0:50:00 0:41:11 0:08:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-13 00:19:15.297499 osd.2 158.69.78.155:6808/8911 25 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.233328 secs" in cluster log

fail 2524512 2018-05-12 02:02:09 2018-05-12 23:57:35 2018-05-13 00:43:35 0:46:00 0:30:54 0:15:06 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync-many.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-05-13 00:23:29.330218 osd.1 158.69.78.185:6804/8868 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.118015 secs" in cluster log

pass 2524514 2018-05-12 02:02:10 2018-05-12 23:57:51 2018-05-13 00:53:51 0:56:00 0:38:24 0:17:36 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
pass 2524515 2018-05-12 02:02:10 2018-05-13 01:20:24 2018-05-13 02:02:24 0:42:00 0:31:24 0:10:36 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-radosbench.yaml} 2
pass 2524517 2018-05-12 02:02:10 2018-05-13 01:20:24 2018-05-13 02:32:25 1:12:01 0:55:49 0:16:12 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524518 2018-05-12 02:02:11 2018-05-13 01:20:30 2018-05-13 02:42:33 1:22:03 0:19:19 1:02:44 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524520 2018-05-12 02:02:11 2018-05-13 01:22:34 2018-05-13 01:52:33 0:29:59 0:22:41 0:07:18 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_stress_watch.yaml} 2
fail 2524521 2018-05-12 02:02:11 2018-05-13 01:24:07 2018-05-13 02:10:08 0:46:01 0:38:10 0:07:51 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-13 02:03:01.533039 osd.4 158.69.79.49:6804/8758 191 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.100128 secs" in cluster log

pass 2524523 2018-05-12 02:02:12 2018-05-13 01:24:08 2018-05-13 01:54:07 0:29:59 0:24:34 0:05:25 ovh master rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524524 2018-05-12 02:02:12 2018-05-13 01:24:08 2018-05-13 01:58:08 0:34:00 0:26:01 0:07:59 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524526 2018-05-12 02:02:13 2018-05-13 01:26:10 2018-05-13 02:02:10 0:36:00 0:26:49 0:09:11 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on ovh083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=jewel TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

pass 2524527 2018-05-12 02:02:13 2018-05-13 01:26:10 2018-05-13 01:46:10 0:20:00 0:14:38 0:05:22 ovh master rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524529 2018-05-12 02:02:13 2018-05-13 01:26:12 2018-05-13 02:10:12 0:44:00 0:28:57 0:15:03 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-13 01:44:32.177045 osd.4 158.69.79.65:6804/8722 3 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.585573 secs" in cluster log

pass 2524530 2018-05-12 02:02:13 2018-05-13 01:26:15 2018-05-13 02:02:15 0:36:00 0:25:40 0:10:20 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
pass 2524532 2018-05-12 02:02:14 2018-05-13 01:26:30 2018-05-13 02:02:32 0:36:02 0:24:12 0:11:50 ovh master rados/singleton-nomsgr/{all/msgr.yaml rados.yaml} 1
pass 2524533 2018-05-12 02:02:14 2018-05-13 01:28:05 2018-05-13 01:52:05 0:24:00 0:12:31 0:11:29 ovh master rados/multimon/{clusters/6.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 2
fail 2524535 2018-05-12 02:02:15 2018-05-13 01:28:05 2018-05-13 02:02:05 0:34:00 0:25:23 0:08:37 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 45 --op write 22 --op write_excl 22 --op delete 10 --pool unique_pool_0'

fail 2524536 2018-05-12 02:02:15 2018-05-13 01:28:19 2018-05-13 01:52:19 0:24:00 0:14:50 0:09:10 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-13 01:47:31.550690 osd.4 158.69.79.8:6804/8691 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.330451 secs" in cluster log

fail 2524538 2018-05-12 02:02:15 2018-05-13 01:28:26 2018-05-13 01:54:26 0:26:00 0:14:31 0:11:29 ovh master centos rados/singleton/{all/lost-unfound-upgrade.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
Failure Reason:

"2018-05-13 01:51:14.698171 osd.0 158.69.79.84:6800/32636 21 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.530168 secs" in cluster log

pass 2524539 2018-05-12 02:02:15 2018-05-13 01:28:30 2018-05-13 01:56:30 0:28:00 0:23:23 0:04:37 ovh master rados/singleton-nomsgr/{all/msgr.yaml rados.yaml} 1
pass 2524541 2018-05-12 02:02:16 2018-05-13 01:29:35 2018-05-13 02:01:35 0:32:00 0:21:44 0:10:16 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/rgw_snaps.yaml} 2
pass 2524542 2018-05-12 02:02:16 2018-05-13 01:32:10 2018-05-13 02:36:11 1:04:01 0:55:23 0:08:38 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 2524544 2018-05-12 02:02:17 2018-05-13 01:32:18 2018-05-13 02:04:17 0:31:59 0:25:02 0:06:57 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/one.yaml workloads/pool-create-delete.yaml} 2
pass 2524545 2018-05-12 02:02:17 2018-05-13 01:34:24 2018-05-13 01:54:24 0:20:00 0:15:02 0:04:58 ovh master rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524547 2018-05-12 02:02:17 2018-05-13 01:34:24 2018-05-13 01:48:24 0:14:00 0:09:03 0:04:57 ovh master rados/objectstore/keyvaluedb.yaml 1
fail 2524548 2018-05-12 02:02:17 2018-05-13 01:34:26 2018-05-13 02:28:27 0:54:01 0:46:21 0:07:40 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-13 01:57:35.128177 osd.3 158.69.79.99:6800/8677 39 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.184127 secs" in cluster log

pass 2524550 2018-05-12 02:02:18 2018-05-13 01:34:29 2018-05-13 01:48:29 0:14:00 0:09:41 0:04:19 ovh master rados/objectstore/keyvaluedb.yaml 1
fail 2524551 2018-05-12 02:02:18 2018-05-13 01:36:28 2018-05-13 02:08:29 0:32:01 0:24:07 0:07:54 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-13 01:53:52.279827 osd.0 158.69.80.105:6800/8748 1 : cluster [WRN] 6 slow requests, 5 included below; oldest blocked for > 30.597124 secs" in cluster log

fail 2524553 2018-05-12 02:02:19 2018-05-13 01:36:28 2018-05-13 02:54:30 1:18:02 1:07:55 0:10:07 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-13 01:59:45.305959 osd.5 158.69.80.124:6808/8763 18 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.609905 secs" in cluster log

fail 2524554 2018-05-12 02:02:19 2018-05-13 01:36:42 2018-05-13 02:20:42 0:44:00 0:36:47 0:07:13 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-13 02:01:39.938347 osd.1 158.69.79.100:6804/8803 89 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.444880 secs" in cluster log

pass 2524556 2018-05-12 02:02:20 2018-05-13 01:38:26 2018-05-13 02:22:27 0:44:01 0:36:32 0:07:29 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 2524557 2018-05-12 02:02:20 2018-05-13 01:38:26 2018-05-13 02:08:27 0:30:01 0:17:00 0:13:01 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524559 2018-05-12 02:02:20 2018-05-13 01:40:31 2018-05-13 02:16:32 0:36:01 0:16:56 0:19:05 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
fail 2524560 2018-05-12 02:02:20 2018-05-13 01:40:31 2018-05-13 02:34:33 0:54:02 0:37:09 0:16:53 ovh master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/default.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-13 02:15:57.850434 osd.5 158.69.80.174:6808/11807 31 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.997120 secs" in cluster log

fail 2524562 2018-05-12 02:02:21 2018-05-13 01:40:31 2018-05-13 03:20:34 1:40:03 1:11:08 0:28:55 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-13 02:21:13.874527 osd.5 158.69.80.141:6808/8896 23 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.783001 secs" in cluster log

pass 2524563 2018-05-12 02:02:21 2018-05-13 01:42:32 2018-05-13 02:30:33 0:48:01 0:17:50 0:30:11 ovh master rados/thrash-erasure-code-shec/{clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 2524565 2018-05-12 02:02:22 2018-05-13 01:44:40 2018-05-13 02:10:40 0:26:00 0:17:03 0:08:57 ovh master rados/verify/{1thrash/default.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
fail 2524566 2018-05-12 02:02:22 2018-05-13 01:46:26 2018-05-13 02:04:25 0:17:59 0:13:14 0:04:45 ovh master rados/singleton/{all/osd-recovery.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
Failure Reason:

"2018-05-13 02:00:25.927516 osd.1 158.69.80.139:6806/9569 1 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.894353 secs" in cluster log

fail 2524568 2018-05-12 02:02:22 2018-05-13 01:48:18 2018-05-13 02:20:18 0:32:00 0:18:22 0:13:38 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool unique_pool_0'

fail 2524569 2018-05-12 02:02:22 2018-05-13 01:48:25 2018-05-13 03:00:26 1:12:01 1:03:47 0:08:14 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-13 02:11:10.570012 osd.2 158.69.80.18:6808/8769 27 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.662621 secs" in cluster log

pass 2524571 2018-05-12 02:02:23 2018-05-13 01:48:29 2018-05-13 02:18:30 0:30:01 0:24:30 0:05:31 ovh master rados/singleton/{all/lost-unfound.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524572 2018-05-12 02:02:23 2018-05-13 01:50:20 2018-05-13 02:28:20 0:38:00 0:29:11 0:08:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op copy_from 50 --op write_excl 50 --op delete 50 --pool base'

fail 2524574 2018-05-12 02:02:24 2018-05-13 01:52:21 2018-05-13 02:34:22 0:42:01 0:34:32 0:07:29 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2018-05-13 02:25:33.065035 osd.1 158.69.80.189:6804/8767 117 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.615136 secs" in cluster log

pass 2524575 2018-05-12 02:02:24 2018-05-13 01:52:21 2018-05-13 02:34:22 0:42:01 0:35:36 0:06:25 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 2524577 2018-05-12 02:02:24 2018-05-13 01:52:34 2018-05-13 02:36:35 0:44:01 0:36:03 0:07:58 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_big.yaml} 2
pass 2524578 2018-05-12 02:02:25 2018-05-13 01:53:37 2018-05-13 03:17:38 1:24:01 1:16:49 0:07:12 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
pass 2524580 2018-05-12 02:02:25 2018-05-13 01:54:08 2018-05-13 02:16:08 0:22:00 0:17:47 0:04:13 ovh master rados/multimon/{clusters/3.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml} 1
pass 2524581 2018-05-12 02:02:25 2018-05-13 01:54:25 2018-05-13 02:10:24 0:15:59 0:12:18 0:03:41 ovh master rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524583 2018-05-12 02:02:26 2018-05-13 01:54:27 2018-05-13 02:28:27 0:34:00 0:15:09 0:18:51 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-13 02:14:37.539021 osd.1 158.69.80.204:6804/8768 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.267258 secs" in cluster log

pass 2524584 2018-05-12 02:02:26 2018-05-13 01:56:49 2018-05-13 02:24:49 0:28:00 0:18:04 0:09:56 ovh master rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml} 2
pass 2524586 2018-05-12 02:02:27 2018-05-13 01:58:24 2018-05-13 02:22:24 0:24:00 0:18:19 0:05:41 ovh master rados/singleton/{all/mon-config-keys.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524587 2018-05-12 02:02:27 2018-05-13 01:59:42 2018-05-13 02:35:42 0:36:00 0:27:44 0:08:16 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
pass 2524589 2018-05-12 02:02:27 2018-05-13 02:01:52 2018-05-13 03:41:54 1:40:02 1:32:24 0:07:38 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 2524590 2018-05-12 02:02:27 2018-05-13 02:02:06 2018-05-13 02:28:06 0:26:00 0:17:26 0:08:34 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync.yaml workloads/rados_mon_workunits.yaml} 2
pass 2524592 2018-05-12 02:02:28 2018-05-13 02:02:11 2018-05-13 02:28:11 0:26:00 0:18:42 0:07:18 ovh master rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml} 2
pass 2524593 2018-05-12 02:02:28 2018-05-13 02:02:16 2018-05-13 02:38:16 0:36:00 0:25:26 0:10:34 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-small-objects-fast-read.yaml} 2
fail 2524595 2018-05-12 02:02:29 2018-05-13 02:02:25 2018-05-13 02:56:26 0:54:01 0:44:03 0:09:58 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-13 02:25:05.743350 osd.0 158.69.80.243:6800/8588 35 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.270346 secs" in cluster log

pass 2524596 2018-05-12 02:02:29 2018-05-13 02:02:33 2018-05-13 02:48:33 0:46:00 0:15:58 0:30:02 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524598 2018-05-12 02:02:30 2018-05-13 02:03:49 2018-05-13 03:27:50 1:24:01 1:13:49 0:10:12 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-13 02:26:53.982416 osd.5 158.69.80.28:6808/8835 1 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.639768 secs" in cluster log

fail 2524599 2018-05-12 02:02:30 2018-05-13 02:04:26 2018-05-13 02:44:26 0:40:00 0:30:19 0:09:41 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-13 02:24:51.657762 osd.1 158.69.80.36:6804/8747 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.977536 secs" in cluster log

pass 2524601 2018-05-12 02:02:30 2018-05-13 02:04:26 2018-05-13 02:34:26 0:30:00 0:25:40 0:04:20 ovh master rados/singleton/{all/mon-thrasher.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524602 2018-05-12 02:02:30 2018-05-13 02:06:20 2018-05-13 02:50:20 0:44:00 0:34:41 0:09:19 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2018-05-13 02:29:09.151670 osd.3 158.69.80.38:6800/8752 5 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.584145 secs" in cluster log

fail 2524604 2018-05-12 02:02:31 2018-05-13 02:08:45 2018-05-13 03:36:46 1:28:01 1:11:32 0:16:29 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-13 02:37:20.702530 osd.0 158.69.80.60:6800/8631 1 : cluster [WRN] 15 slow requests, 5 included below; oldest blocked for > 30.856470 secs" in cluster log

pass 2524605 2018-05-12 02:02:31 2018-05-13 02:08:45 2018-05-13 02:24:44 0:15:59 0:11:12 0:04:47 ovh master rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524607 2018-05-12 02:02:32 2018-05-13 02:10:25 2018-05-13 03:12:26 1:02:01 0:47:25 0:14:36 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-13 02:40:42.369318 osd.5 158.69.80.57:6808/8542 54 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.763202 secs" in cluster log

pass 2524608 2018-05-12 02:02:32 2018-05-13 02:10:25 2018-05-13 02:44:25 0:34:00 0:21:43 0:12:17 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rgw_snaps.yaml} 2
pass 2524610 2018-05-12 02:02:33 2018-05-13 02:10:26 2018-05-13 02:36:25 0:25:59 0:21:22 0:04:37 ovh master rados/objectstore/objectcacher-stress.yaml 1
pass 2524611 2018-05-12 02:02:33 2018-05-13 02:10:41 2018-05-13 02:44:41 0:34:00 0:25:57 0:08:03 ovh master rados/objectstore/objectcacher-stress.yaml 1
fail 2524613 2018-05-12 02:02:33 2018-05-13 02:16:38 2018-05-13 03:04:36 0:47:58 0:37:25 0:10:33 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-13 02:38:12.117155 osd.5 158.69.80.65:6808/8725 12 : cluster [WRN] 25 slow requests, 5 included below; oldest blocked for > 30.653391 secs" in cluster log

fail 2524614 2018-05-12 02:02:33 2018-05-13 02:16:39 2018-05-13 02:52:36 0:35:57 0:24:04 0:11:53 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-13 02:40:38.628656 osd.1 158.69.80.66:6804/8896 3 : cluster [WRN] 12 slow requests, 5 included below; oldest blocked for > 30.224643 secs" in cluster log

pass 2524616 2018-05-12 02:02:34 2018-05-13 02:18:47 2018-05-13 02:36:42 0:17:55 0:13:25 0:04:30 ovh master rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524617 2018-05-12 02:02:34 2018-05-13 02:20:42 2018-05-13 02:56:41 0:35:59 0:25:36 0:10:23 ovh master rados/verify/{1thrash/none.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
fail 2524619 2018-05-12 02:02:35 2018-05-13 02:20:44 2018-05-13 03:12:44 0:52:00 0:41:50 0:10:10 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool unique_pool_0'

pass 2524620 2018-05-12 02:02:35 2018-05-13 02:22:42 2018-05-13 03:20:42 0:58:00 0:46:30 0:11:30 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 2524622 2018-05-12 02:02:35 2018-05-13 02:22:42 2018-05-13 03:02:42 0:40:00 0:25:05 0:14:55 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
pass 2524623 2018-05-12 02:02:36 2018-05-13 02:22:55 2018-05-13 02:42:55 0:20:00 0:16:10 0:03:50 ovh master rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524625 2018-05-12 02:02:36 2018-05-13 02:25:01 2018-05-13 02:49:01 0:24:00 0:14:24 0:09:36 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync-many.yaml workloads/rados_5925.yaml} 2
fail 2524626 2018-05-12 02:02:36 2018-05-13 02:25:01 2018-05-13 03:09:01 0:44:00 0:35:50 0:08:10 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2018-05-13 02:54:11.650959 osd.0 158.69.80.82:6812/9596 45 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.625592 secs" in cluster log

fail 2524628 2018-05-12 02:02:37 2018-05-13 02:37:08 2018-05-13 03:29:09 0:52:01 0:44:17 0:07:44 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2018-05-13 03:07:10.869948 osd.3 158.69.81.15:6812/10912 37 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.507273 secs" in cluster log

pass 2524629 2018-05-12 02:02:37 2018-05-13 02:38:34 2018-05-13 03:08:33 0:29:59 0:20:55 0:09:04 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/repair_test.yaml} 2
pass 2524631 2018-05-12 02:02:38 2018-05-13 02:42:52 2018-05-13 08:04:59 5:22:07 0:15:47 5:06:20 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 2524632 2018-05-12 02:02:38 2018-05-13 02:42:56 2018-05-13 03:14:56 0:32:00 0:19:54 0:12:06 ovh master rados/multimon/{clusters/9.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml} 3
pass 2524634 2018-05-12 02:02:38 2018-05-13 02:44:43 2018-05-13 03:30:43 0:46:00 0:26:28 0:19:32 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 2524635 2018-05-12 02:02:39 2018-05-13 02:44:43 2018-05-13 03:10:43 0:26:00 0:16:44 0:09:16 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 2524637 2018-05-12 02:02:39 2018-05-13 02:44:43 2018-05-13 03:32:44 0:48:01 0:33:09 0:14:52 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-13 03:11:37.895440 osd.4 158.69.81.196:6804/8727 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.884954 secs" in cluster log

pass 2524638 2018-05-12 02:02:39 2018-05-13 02:48:50 2018-05-13 03:04:50 0:16:00 0:10:50 0:05:10 ovh master rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml} 1
pass 2524640 2018-05-12 02:02:40 2018-05-13 02:49:02 2018-05-13 03:07:01 0:17:59 0:11:58 0:06:01 ovh master rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml} 1
pass 2524641 2018-05-12 02:02:40 2018-05-13 02:50:37 2018-05-13 03:12:37 0:22:00 0:13:45 0:08:15 ovh master rados/singleton/{all/reg11184.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524643 2018-05-12 02:02:41 2018-05-13 02:52:54 2018-05-13 03:14:53 0:21:59 0:14:26 0:07:33 ovh master rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524644 2018-05-12 02:02:41 2018-05-13 02:54:48 2018-05-13 04:38:49 1:44:01 1:35:15 0:08:46 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-13 03:16:17.326204 osd.3 158.69.81.211:6800/8604 9 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.563661 secs" in cluster log

fail 2524646 2018-05-12 02:02:41 2018-05-13 02:56:43 2018-05-13 03:16:42 0:19:59 0:12:35 0:07:24 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 45 --op write 22 --op write_excl 22 --op delete 10 --pool unique_pool_0'

fail 2524647 2018-05-12 02:02:42 2018-05-13 02:56:43 2018-05-13 03:54:43 0:58:00 0:45:51 0:12:09 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op setattr 25 --op read 100 --op copy_from 50 --op write 0 --op rmattr 25 --op append 100 --op delete 50 --pool unique_pool_0'

fail 2524649 2018-05-12 02:02:42 2018-05-13 02:58:43 2018-05-13 03:44:43 0:46:00 0:35:13 0:10:47 ovh master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2018-05-13 03:33:13.831091 osd.3 158.69.81.226:6800/29483 55 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.296252 secs" in cluster log

fail 2524650 2018-05-12 02:02:42 2018-05-13 03:00:43 2018-05-13 03:48:43 0:48:00 0:40:24 0:07:36 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-13 03:23:05.261738 osd.5 158.69.81.227:6808/8852 22 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.398901 secs" in cluster log

pass 2524652 2018-05-12 02:02:43 2018-05-13 03:03:00 2018-05-13 03:41:00 0:38:00 0:25:11 0:12:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/rgw_snaps.yaml} 2
fail 2524653 2018-05-12 02:02:43 2018-05-13 03:04:52 2018-05-13 04:32:53 1:28:01 1:16:12 0:11:49 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-05-13 03:30:05.322731 osd.5 158.69.81.239:6809/9622 1 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.396320 secs" in cluster log

pass 2524655 2018-05-12 02:02:44 2018-05-13 03:04:52 2018-05-13 03:24:52 0:20:00 0:14:49 0:05:11 ovh master rados/singleton/{all/rest-api.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524656 2018-05-12 02:02:44 2018-05-13 03:07:18 2018-05-13 03:49:18 0:42:00 0:23:50 0:18:10 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 2524658 2018-05-12 02:02:44 2018-05-13 03:08:50 2018-05-13 03:26:49 0:17:59 0:14:19 0:03:40 ovh master rados/singleton/{all/osd-recovery.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524659 2018-05-12 02:02:44 2018-05-13 03:09:02 2018-05-13 04:37:04 1:28:02 1:17:56 0:10:06 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-13 03:32:50.677733 osd.5 158.69.81.255:6808/8842 12 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.158424 secs" in cluster log

pass 2524661 2018-05-12 02:02:45 2018-05-13 03:10:29 2018-05-13 03:56:31 0:46:02 0:37:50 0:08:12 ovh master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/force-sync-many.yaml workloads/snaps-few-objects.yaml} 2
fail 2524662 2018-05-12 02:02:45 2018-05-13 03:10:44 2018-05-13 03:56:44 0:46:00 0:37:56 0:08:04 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-13 03:46:09.065375 osd.4 158.69.81.248:6807/24955 17 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.271829 secs" in cluster log

pass 2524664 2018-05-12 02:02:46 2018-05-13 03:12:42 2018-05-13 04:00:42 0:48:00 0:37:55 0:10:05 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 2524665 2018-05-12 02:02:46 2018-05-13 03:12:42 2018-05-13 03:44:42 0:32:00 0:23:47 0:08:13 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/ec-small-objects.yaml} 2
Failure Reason:

"2018-05-13 03:30:50.082834 osd.5 158.69.81.3:6808/8727 1 : cluster [WRN] 7 slow requests, 5 included below; oldest blocked for > 30.654780 secs" in cluster log

pass 2524667 2018-05-12 02:02:47 2018-05-13 03:12:45 2018-05-13 04:50:46 1:38:01 1:26:25 0:11:36 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
pass 2524668 2018-05-12 02:02:47 2018-05-13 03:15:13 2018-05-13 04:27:14 1:12:01 0:17:06 0:54:55 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524670 2018-05-12 02:02:47 2018-05-13 03:15:13 2018-05-13 03:49:13 0:34:00 0:25:56 0:08:04 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool base'

pass 2524671 2018-05-12 02:02:47 2018-05-13 03:16:52 2018-05-13 03:54:52 0:38:00 0:12:11 0:25:49 ovh master rados/multimon/{clusters/6.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 2
fail 2524673 2018-05-12 02:02:48 2018-05-13 03:17:47 2018-05-13 03:31:47 0:14:00 0:09:37 0:04:23 ovh master rados/objectstore/objectstore.yaml 1
Failure Reason:

Command failed on ovh094 with status 134: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'mkdir $TESTDIR/ostest && cd $TESTDIR/ostest && ceph_test_objectstore --gtest_filter=-*/2:-*/3'"

fail 2524674 2018-05-12 02:02:48 2018-05-13 03:20:51 2018-05-13 03:38:51 0:18:00 0:10:32 0:07:28 ovh master rados/objectstore/objectstore.yaml 1
Failure Reason:

Command failed on ovh079 with status 139: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'mkdir $TESTDIR/ostest && cd $TESTDIR/ostest && ceph_test_objectstore --gtest_filter=-*/2:-*/3'"

pass 2524676 2018-05-12 02:02:49 2018-05-13 03:20:51 2018-05-13 03:36:51 0:16:00 0:11:52 0:04:08 ovh master rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524677 2018-05-12 02:02:49 2018-05-13 03:25:09 2018-05-13 04:03:09 0:38:00 0:24:55 0:13:05 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op copy_from 50 --op write_excl 50 --op delete 50 --pool base'

fail 2524679 2018-05-12 02:02:49 2018-05-13 03:27:05 2018-05-13 03:49:02 0:21:57 0:15:30 0:06:27 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2018-05-13 03:45:25.413115 osd.4 158.69.81.62:6804/8741 13 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.572424 secs" in cluster log

pass 2524680 2018-05-12 02:02:49 2018-05-13 03:28:04 2018-05-13 04:07:53 0:39:49 0:23:15 0:16:34 ovh master rados/singleton/{all/thrash-rados.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
fail 2524682 2018-05-12 02:02:50 2018-05-13 03:30:05 2018-05-13 04:24:05 0:54:00 0:46:41 0:07:19 ovh master rados/verify/{1thrash/none.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml} 2
Failure Reason:

"2018-05-13 03:55:08.937386 osd.4 158.69.81.84:6800/8905 1 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.976990 secs" in cluster log

fail 2524683 2018-05-12 02:02:50 2018-05-13 03:30:47 2018-05-13 04:46:46 1:15:59 1:00:14 0:15:45 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-13 04:10:08.892273 osd.1 158.69.81.91:6804/8853 118 : cluster [WRN] 5 slow requests, 5 included below; oldest blocked for > 30.706444 secs" in cluster log

pass 2524685 2018-05-12 02:02:51 2018-05-13 03:32:04 2018-05-13 03:58:04 0:26:00 0:17:10 0:08:50 ovh master rados/basic/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/scrub_test.yaml} 2
fail 2524686 2018-05-12 02:02:51 2018-05-13 03:32:45 2018-05-13 04:48:46 1:16:01 1:04:48 0:11:13 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-05-13 04:02:03.399273 osd.2 158.69.81.10:6808/8864 29 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.467858 secs" in cluster log

pass 2524688 2018-05-12 02:02:52 2018-05-13 03:36:56 2018-05-13 03:56:55 0:19:59 0:15:36 0:04:23 ovh master rados/singleton-nomsgr/{all/valgrind-leaks.yaml rados.yaml} 1
pass 2524689 2018-05-12 02:02:52 2018-05-13 03:36:56 2018-05-13 03:56:55 0:19:59 0:16:32 0:03:27 ovh master rados/singleton-nomsgr/{all/valgrind-leaks.yaml rados.yaml} 1
pass 2524691 2018-05-12 02:02:52 2018-05-13 03:39:00 2018-05-13 04:17:00 0:38:00 0:24:27 0:13:33 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
fail 2524692 2018-05-12 02:02:52 2018-05-13 03:41:10 2018-05-13 04:39:10 0:58:00 0:49:44 0:08:16 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-05-13 04:25:01.615523 osd.2 158.69.81.92:6808/8876 125 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.926938 secs" in cluster log

fail 2524694 2018-05-12 02:02:53 2018-05-13 03:42:03 2018-05-13 04:24:03 0:42:00 0:31:22 0:10:38 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-05-13 04:05:14.500626 osd.2 158.69.82.119:6808/8894 9 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.127002 secs" in cluster log

pass 2524695 2018-05-12 02:02:53 2018-05-13 03:44:52 2018-05-13 04:00:51 0:15:59 0:11:25 0:04:34 ovh master rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 2524697 2018-05-12 02:02:54 2018-05-13 03:44:52 2018-05-13 04:50:52 1:06:00 0:54:33 0:11:27 ovh master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
fail 2524698 2018-05-12 02:02:54 2018-05-13 03:44:52 2018-05-13 04:10:51 0:25:59 0:17:19 0:08:40 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool base'

pass 2524700 2018-05-12 02:02:54 2018-05-13 03:48:53 2018-05-13 04:26:53 0:38:00 0:29:21 0:08:39 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 2524701 2018-05-12 02:02:54 2018-05-13 03:49:03 2018-05-13 04:27:03 0:38:00 0:26:11 0:11:49 ovh master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync.yaml workloads/rados_api_tests.yaml} 2
fail 2524702 2018-05-12 02:02:55 2018-05-13 03:49:14 2018-05-13 04:51:15 1:02:01 0:32:12 0:29:49 ovh master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2018-05-13 04:27:22.882656 osd.3 158.69.82.142:6800/8700 91 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.645795 secs" in cluster log

pass 2524703 2018-05-12 02:02:55 2018-05-13 03:49:19 2018-05-13 04:25:19 0:36:00 0:22:45 0:13:15 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/rgw_snaps.yaml} 2
pass 2524704 2018-05-12 02:02:56 2018-05-13 03:50:44 2018-05-13 05:06:45 1:16:01 0:16:14 0:59:47 ovh master rados/thrash-erasure-code-big/{cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
fail 2524705 2018-05-12 02:02:56 2018-05-13 03:54:53 2018-05-13 04:26:53 0:32:00 0:23:55 0:08:05 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-05-13 04:19:51.409424 osd.3 158.69.82.146:6800/8583 15 : cluster [WRN] 3 slow requests, 3 included below; oldest blocked for > 30.233419 secs" in cluster log

fail 2524706 2018-05-12 02:02:57 2018-05-13 03:54:53 2018-05-13 05:14:54 1:20:01 1:12:50 0:07:11 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-05-13 04:15:24.731685 osd.0 158.69.82.152:6801/11994 1 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.852228 secs" in cluster log

pass 2524707 2018-05-12 02:02:57 2018-05-13 03:56:49 2018-05-13 04:14:49 0:18:00 0:12:54 0:05:06 ovh master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524708 2018-05-12 02:02:57 2018-05-13 03:56:49 2018-05-13 04:48:50 0:52:01 0:40:06 0:11:55 ovh master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2018-05-13 04:35:15.542684 osd.3 158.69.82.187:6800/8700 53 : cluster [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.893796 secs" in cluster log

fail 2524709 2018-05-12 02:02:57 2018-05-13 03:56:56 2018-05-13 04:58:57 1:02:01 0:51:20 0:10:41 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2018-05-13 04:19:23.343342 osd.1 158.69.82.162:6804/9748 15 : cluster [WRN] 4 slow requests, 4 included below; oldest blocked for > 30.301943 secs" in cluster log

fail 2524710 2018-05-12 02:02:58 2018-05-13 03:56:56 2018-05-13 04:38:57 0:42:01 0:32:11 0:09:50 ovh master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.5 restart

pass 2524711 2018-05-12 02:02:58 2018-05-13 03:58:21 2018-05-13 04:18:22 0:20:01 0:15:15 0:04:46 ovh master rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 2524713 2018-05-12 02:02:59 2018-05-13 04:01:01 2018-05-13 05:03:02 1:02:01 0:45:25 0:16:36 ovh master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} hobj-sort.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-05-13 04:42:32.913185 osd.5 158.69.82.242:6808/8895 86 : cluster [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.570136 secs" in cluster log