Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 1617210 2017-09-10 22:28:52 2017-09-10 22:31:17 2017-09-10 22:47:16 0:15:59 0:11:42 0:04:17 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617211 2017-09-10 22:28:53 2017-09-10 22:31:44 2017-09-10 22:53:43 0:21:59 0:17:08 0:04:51 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/repair_test.yaml} 2
pass 1617212 2017-09-10 22:28:53 2017-09-10 22:33:29 2017-09-10 22:51:29 0:18:00 0:12:20 0:05:40 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617213 2017-09-10 22:28:54 2017-09-10 22:34:58 2017-09-10 22:46:57 0:11:59 0:07:02 0:04:57 smithi master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 1617214 2017-09-10 22:28:55 2017-09-10 22:36:39 2017-09-10 23:02:39 0:26:00 0:18:46 0:07:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.4 restart

dead 1617215 2017-09-10 22:28:56 2017-09-10 22:38:54 2017-09-11 10:45:46 12:06:52 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617216 2017-09-10 22:28:56 2017-09-10 22:38:54 2017-09-11 06:33:06 7:54:12 7:45:37 0:08:35 smithi master rados/objectstore/objectstore.yaml 1
pass 1617217 2017-09-10 22:28:57 2017-09-10 22:39:54 2017-09-10 22:59:53 0:19:59 0:16:01 0:03:58 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617218 2017-09-10 22:28:58 2017-09-10 22:40:18 2017-09-10 23:02:16 0:21:58 0:17:53 0:04:05 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/sync.yaml workloads/pool-create-delete.yaml} 2
pass 1617219 2017-09-10 22:28:58 2017-09-10 22:41:01 2017-09-10 22:49:02 0:08:01 0:05:59 0:02:02 smithi master rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 1617220 2017-09-10 22:28:59 2017-09-10 22:43:22 2017-09-10 22:59:21 0:15:59 0:11:48 0:04:11 smithi master centos rados/singleton-nomsgr/{all/valgrind-leaks.yaml rados.yaml} 1
pass 1617221 2017-09-10 22:29:00 2017-09-10 22:43:22 2017-09-10 23:03:21 0:19:59 0:11:46 0:08:13 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
dead 1617222 2017-09-10 22:29:01 2017-09-10 22:43:22 2017-09-11 10:50:15 12:06:53 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
dead 1617223 2017-09-10 22:29:01 2017-09-10 22:44:52 2017-09-11 10:51:14 12:06:22 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 1617224 2017-09-10 22:29:02 2017-09-10 22:44:52 2017-09-10 23:08:52 0:24:00 0:20:07 0:03:53 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617225 2017-09-10 22:29:03 2017-09-10 22:45:20 2017-09-10 23:09:20 0:24:00 0:12:40 0:11:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 1617226 2017-09-10 22:29:04 2017-09-10 22:45:42 2017-09-11 10:52:48 12:07:06 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
fail 1617227 2017-09-10 22:29:05 2017-09-10 22:46:13 2017-09-10 23:14:13 0:28:00 0:15:07 0:12:53 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rgw_snaps.yaml} 2
Failure Reason:

"2017-09-10 23:05:30.822018 osd.5 osd.5 172.21.15.52:6801/7741 1 : cluster [ERR] osd.5 found snap mapper error on pg 5.6 oid 5:6dd5b705:root::.bucket.meta.rwtest%3acd79e251-f90b-4314-ba44-ca6de1741490.4168.1:8 snaps in mapper: 1,2,3,4,6, oi: 1,2,3,4,6,8...repaired" in cluster log

pass 1617228 2017-09-10 22:29:05 2017-09-10 22:46:58 2017-09-10 22:58:57 0:11:59 0:07:37 0:04:22 smithi master rados/singleton/{all/divergent_priors.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
fail 1617229 2017-09-10 22:29:06 2017-09-10 22:47:29 2017-09-10 23:23:28 0:35:59 0:28:33 0:07:26 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2017-09-10 22:59:43.427248 osd.2 osd.2 172.21.15.11:6806/341247 35 : cluster [ERR] osd.2 found snap mapper error on pg 3.1 oid 3:82392d76:::smithi011347025-75 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:25 snaps in mapper: 23,24, oi: 23,24,25...repaired" in cluster log

dead 1617230 2017-09-10 22:29:07 2017-09-10 22:47:29 2017-09-11 10:53:42 12:06:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 1617231 2017-09-10 22:29:08 2017-09-10 22:49:14 2017-09-10 23:07:13 0:17:59 0:12:27 0:05:32 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
pass 1617232 2017-09-10 22:29:08 2017-09-10 22:49:20 2017-09-10 23:29:20 0:40:00 0:26:54 0:13:06 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617233 2017-09-10 22:29:09 2017-09-10 22:50:52 2017-09-10 23:02:52 0:12:00 0:09:17 0:02:43 smithi master rados/singleton/{all/divergent_priors2.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617234 2017-09-10 22:29:10 2017-09-10 22:51:22 2017-09-10 23:23:22 0:32:00 0:26:42 0:05:18 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
fail 1617235 2017-09-10 22:29:11 2017-09-10 22:51:30 2017-09-11 00:09:31 1:18:01 1:11:57 0:06:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 1617236 2017-09-10 22:29:11 2017-09-10 22:53:25 2017-09-10 23:45:23 0:51:58 0:24:21 0:27:37 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore.yaml rados.yaml supported/ubuntu_latest.yaml thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 1617237 2017-09-10 22:29:12 2017-09-10 22:53:46 2017-09-10 23:21:45 0:27:59 0:13:22 0:14:37 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617238 2017-09-10 22:29:13 2017-09-10 22:53:46 2017-09-10 23:13:46 0:20:00 0:09:31 0:10:29 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_5925.yaml} 2
fail 1617239 2017-09-10 22:29:13 2017-09-10 22:53:49 2017-09-10 23:05:48 0:11:59 0:05:14 0:06:45 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/scrub_test.yaml} 2
Failure Reason:

Command failed on smithi005 with status 1: "sudo yum -y install '' libcephfs-devel"

pass 1617240 2017-09-10 22:29:14 2017-09-10 22:54:36 2017-09-10 23:10:35 0:15:59 0:10:39 0:05:20 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-bitmap.yaml tasks/failover.yaml} 2
pass 1617241 2017-09-10 22:29:15 2017-09-10 22:54:43 2017-09-10 23:04:43 0:10:00 0:06:24 0:03:36 smithi master rados/objectstore/alloc-hint.yaml 1
pass 1617242 2017-09-10 22:29:16 2017-09-10 22:54:46 2017-09-10 23:08:51 0:14:05 0:08:46 0:05:19 smithi master rados/rest/mgr-restful.yaml 1
pass 1617243 2017-09-10 22:29:16 2017-09-10 22:55:53 2017-09-10 23:23:52 0:27:59 0:24:37 0:03:22 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 1617244 2017-09-10 22:29:17 2017-09-10 22:55:53 2017-09-10 23:07:52 0:11:59 0:09:16 0:02:43 smithi master rados/singleton-nomsgr/{all/admin_socket_output.yaml rados.yaml} 1
pass 1617245 2017-09-10 22:29:18 2017-09-10 22:58:35 2017-09-10 23:06:30 0:07:55 0:07:06 0:00:49 smithi master rados/standalone/crush.yaml 1
fail 1617246 2017-09-10 22:29:19 2017-09-10 22:58:35 2017-09-11 00:56:34 1:57:59 0:11:46 1:46:13 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

"2017-09-11 00:50:14.132964 osd.1 osd.1 172.21.15.24:6800/334001 32 : cluster [ERR] osd.1 found snap mapper error on pg 2.8s2 oid 2:1f25d5e7:::smithi1331693155-37 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:31 snaps in mapper: 9,b,d,13,14,15,17,1b,1c,1f,22,25,26,27,2a,2b,2e,2f,31, oi: 9,b,d,13,14,15,17,1b,1c,1f,20,22,25,26,27,2a,2b,2e,2f,31...repaired" in cluster log

pass 1617247 2017-09-10 22:29:19 2017-09-10 22:58:42 2017-09-11 01:02:43 2:04:01 1:53:05 0:10:56 smithi master rados/upgrade/luminous-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/luminous.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-mimic.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} thrashosds-health.yaml} 3
pass 1617248 2017-09-10 22:29:20 2017-09-10 22:59:00 2017-09-10 23:14:58 0:15:58 0:14:06 0:01:52 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617249 2017-09-10 22:29:21 2017-09-10 22:59:01 2017-09-10 23:10:58 0:11:57 0:09:25 0:02:32 smithi master rados/singleton/{all/dump-stuck.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 1617250 2017-09-10 22:29:21 2017-09-10 22:59:02 2017-09-10 23:23:00 0:23:58 0:17:47 0:06:11 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
pass 1617251 2017-09-10 22:29:22 2017-09-10 22:59:10 2017-09-10 23:09:08 0:09:58 0:08:23 0:01:35 smithi master rados/multimon/{clusters/6.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 2
dead 1617252 2017-09-10 22:29:23 2017-09-10 22:59:23 2017-09-11 11:04:56 12:05:33 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
pass 1617253 2017-09-10 22:29:24 2017-09-10 23:00:17 2017-09-10 23:42:17 0:42:00 0:20:00 0:22:00 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} 2
dead 1617254 2017-09-10 22:29:25 2017-09-10 23:00:18 2017-09-11 11:07:16 12:06:58 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617255 2017-09-10 22:29:25 2017-09-10 23:00:56 2017-09-10 23:22:55 0:21:59 0:15:07 0:06:52 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617256 2017-09-10 22:29:26 2017-09-10 23:02:05 2017-09-10 23:26:04 0:23:59 0:22:11 0:01:48 smithi master rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 1617257 2017-09-10 22:29:27 2017-09-10 23:02:05 2017-09-10 23:30:04 0:27:59 0:13:17 0:14:42 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 1617258 2017-09-10 22:29:28 2017-09-10 23:02:19 2017-09-10 23:58:19 0:56:00 0:45:40 0:10:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

Exiting scrub checking -- not all pgs scrubbed.

dead 1617259 2017-09-10 22:29:28 2017-09-10 23:02:41 2017-09-11 11:09:12 12:06:31 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} 2
pass 1617260 2017-09-10 22:29:29 2017-09-10 23:02:53 2017-09-10 23:24:53 0:22:00 0:16:20 0:05:40 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1617261 2017-09-10 22:29:29 2017-09-10 23:03:34 2017-09-10 23:39:34 0:36:00 0:11:46 0:24:14 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
Failure Reason:

"2017-09-10 23:33:00.527034 osd.0 osd.0 172.21.15.10:6812/31590 84 : cluster [ERR] osd.0 found snap mapper error on pg 2.8s2 oid 2:1c15a252:::smithi0102670-14:36 snaps in mapper: 34,35, oi: 32,34,35,36...repaired" in cluster log

fail 1617262 2017-09-10 22:29:30 2017-09-10 23:03:34 2017-09-10 23:35:34 0:32:00 0:21:49 0:10:11 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2017-09-10 23:28:11.376395 osd.0 osd.0 172.21.15.109:6800/9406 2 : cluster [ERR] osd.0 found snap mapper error on pg 3.1 oid 3:9ab49ae5:::smithi10910798-452:245 snaps in mapper: 18e,1ab,1c5,1cc,1e6,1eb,1ef,1f1,1f5,1f8,200,202,217,21a,21f,223,228,229,22b,231,234,235,237,239,23a,23c,23e,242,245, oi: 17d,18e,1ab,1c5,1cc,1e6,1eb,1ef,1f1,1f5,1f8,200,202,217,21a,21f,223,228,229,22b,231,234,235,237,239,23a,23c,23e,241,242,245...repaired" in cluster log

pass 1617263 2017-09-10 22:29:31 2017-09-10 23:04:54 2017-09-10 23:34:54 0:30:00 0:20:54 0:09:06 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_api_tests.yaml} 2
pass 1617264 2017-09-10 22:29:32 2017-09-10 23:05:50 2017-09-10 23:11:49 0:05:59 0:05:03 0:00:56 smithi master rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1617265 2017-09-10 22:29:32 2017-09-10 23:06:42 2017-09-11 11:12:30 12:05:48 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1617266 2017-09-10 22:29:33 2017-09-10 23:07:16 2017-09-11 11:14:06 12:06:50 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 1617267 2017-09-10 22:29:34 2017-09-10 23:08:04 2017-09-10 23:36:03 0:27:59 0:25:54 0:02:05 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml thrashers/many.yaml workloads/rados_api_tests.yaml} 2
pass 1617268 2017-09-10 22:29:34 2017-09-10 23:08:53 2017-09-10 23:26:52 0:17:59 0:11:44 0:06:15 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache.yaml} 2
pass 1617269 2017-09-10 22:29:35 2017-09-10 23:08:53 2017-09-10 23:20:53 0:12:00 0:06:29 0:05:31 smithi master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml} 1
dead 1617270 2017-09-10 22:29:36 2017-09-10 23:09:18 2017-09-11 11:16:20 12:07:02 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617271 2017-09-10 22:29:37 2017-09-10 23:09:18 2017-09-10 23:33:18 0:24:00 0:23:31 0:00:29 smithi master rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617272 2017-09-10 22:29:37 2017-09-10 23:09:21 2017-09-10 23:31:20 0:21:59 0:19:50 0:02:09 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 1617273 2017-09-10 22:29:38 2017-09-10 23:10:53 2017-09-10 23:32:52 0:21:59 0:21:00 0:00:59 smithi master rados/objectstore/ceph_objectstore_tool.yaml 1
pass 1617274 2017-09-10 22:29:39 2017-09-10 23:10:59 2017-09-10 23:36:59 0:26:00 0:23:21 0:02:39 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 1617275 2017-09-10 22:29:39 2017-09-10 23:11:50 2017-09-10 23:27:50 0:16:00 0:11:22 0:04:38 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617276 2017-09-10 22:29:40 2017-09-10 23:13:01 2017-09-10 23:35:01 0:22:00 0:13:51 0:08:09 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617277 2017-09-10 22:29:41 2017-09-10 23:13:48 2017-09-10 23:27:47 0:13:59 0:10:13 0:03:46 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml tasks/rados_cls_all.yaml} 2
pass 1617278 2017-09-10 22:29:41 2017-09-10 23:13:48 2017-09-10 23:41:47 0:27:59 0:26:22 0:01:37 smithi master rados/singleton/{all/lost-unfound.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
fail 1617279 2017-09-10 22:29:42 2017-09-10 23:14:27 2017-09-10 23:46:26 0:31:59 0:20:12 0:11:47 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2017-09-10 23:29:24.919610 osd.2 osd.2 172.21.15.34:6813/7186 10 : cluster [ERR] osd.2 found snap mapper error on pg 2.e oid 2:7390125c:::smithi0341541-712:15 snaps in mapper: 13, oi: b,f,13,14,15...repaired" in cluster log

fail 1617280 2017-09-10 22:29:43 2017-09-10 23:14:41 2017-09-10 23:54:41 0:40:00 0:28:03 0:11:57 smithi master centos 7.3 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml supported/centos_latest.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2017-09-10 23:29:26.585070 osd.4 osd.4 172.21.15.27:6801/11396 33 : cluster [ERR] osd.4 found snap mapper error on pg 2.es0 oid 2:7744aa6a:::smithi031385825-8:31 snaps in mapper: 27,2d,30, oi: 27,2d,30,31...repaired" in cluster log

pass 1617281 2017-09-10 22:29:44 2017-09-10 23:15:00 2017-09-10 23:48:59 0:33:59 0:24:47 0:09:12 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617282 2017-09-10 22:29:44 2017-09-10 23:17:23 2017-09-10 23:35:22 0:17:59 0:14:01 0:03:58 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617283 2017-09-10 22:29:45 2017-09-10 23:17:38 2017-09-10 23:37:37 0:19:59 0:19:13 0:00:46 smithi master rados/standalone/erasure-code.yaml 1
fail 1617284 2017-09-10 22:29:46 2017-09-10 23:20:49 2017-09-11 00:02:49 0:42:00 0:12:22 0:29:38 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

"2017-09-10 23:54:54.639798 osd.12 osd.12 172.21.15.34:6813/25760 26 : cluster [ERR] osd.12 found snap mapper error on pg 2.fs4 oid 2:ff3004f9:::smithi04919677-48:2e snaps in mapper: 21,23,24,2d,2e, oi: 21,23,24,25,2d,2e...repaired" in cluster log

pass 1617285 2017-09-10 22:29:46 2017-09-10 23:20:49 2017-09-10 23:28:48 0:07:59 0:06:59 0:01:00 smithi master rados/singleton/{all/mon-auth-caps.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 1617286 2017-09-10 22:29:47 2017-09-10 23:20:54 2017-09-10 23:56:54 0:36:00 0:32:32 0:03:28 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 1617287 2017-09-10 22:29:48 2017-09-10 23:21:25 2017-09-10 23:57:25 0:36:00 0:16:12 0:19:48 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/one.yaml workloads/rados_mon_workunits.yaml} 2
pass 1617288 2017-09-10 22:29:49 2017-09-10 23:21:47 2017-09-10 23:49:46 0:27:59 0:14:22 0:13:37 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617289 2017-09-10 22:29:49 2017-09-10 23:22:38 2017-09-10 23:44:37 0:21:59 0:13:25 0:08:34 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
pass 1617290 2017-09-10 22:29:50 2017-09-10 23:22:44 2017-09-10 23:32:44 0:10:00 0:06:03 0:03:57 smithi master rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml} 1
dead 1617291 2017-09-10 22:29:51 2017-09-10 23:22:50 2017-09-11 11:29:20 12:06:30 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 1617292 2017-09-10 22:29:51 2017-09-10 23:22:56 2017-09-10 23:40:56 0:18:00 0:11:54 0:06:06 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_python.yaml} 2
pass 1617293 2017-09-10 22:29:52 2017-09-10 23:23:01 2017-09-10 23:45:01 0:22:00 0:06:56 0:15:04 smithi master rados/multimon/{clusters/6.yaml mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 2
pass 1617294 2017-09-10 22:29:53 2017-09-10 23:23:23 2017-09-10 23:35:22 0:11:59 0:11:26 0:00:33 smithi master rados/singleton/{all/mon-config-keys.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1617295 2017-09-10 22:29:54 2017-09-10 23:23:30 2017-09-11 11:34:29 12:10:59 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
dead 1617296 2017-09-10 22:29:54 2017-09-10 23:24:03 2017-09-11 11:31:05 12:07:02 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/few.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} 2
pass 1617297 2017-09-10 22:29:55 2017-09-10 23:24:55 2017-09-10 23:46:54 0:21:59 0:17:59 0:04:00 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
pass 1617298 2017-09-10 22:29:56 2017-09-10 23:26:18 2017-09-10 23:50:17 0:23:59 0:21:39 0:02:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1617299 2017-09-10 22:29:56 2017-09-10 23:26:53 2017-09-11 11:32:13 12:05:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1617300 2017-09-10 22:29:57 2017-09-10 23:28:03 2017-09-11 11:35:51 12:07:48 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 1617301 2017-09-10 22:29:58 2017-09-10 23:28:02 2017-09-10 23:34:01 0:05:59 0:04:48 0:01:11 smithi master rados/objectstore/filejournal.yaml 1
pass 1617302 2017-09-10 22:29:59 2017-09-10 23:28:50 2017-09-10 23:46:49 0:17:59 0:09:31 0:08:28 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1617303 2017-09-10 22:29:59 2017-09-10 23:29:30 2017-09-11 11:35:54 12:06:24 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617304 2017-09-10 22:30:00 2017-09-10 23:30:12 2017-09-10 23:42:11 0:11:59 0:08:03 0:03:56 smithi master rados/singleton/{all/mon-seesaw.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617305 2017-09-10 22:30:01 2017-09-10 23:31:38 2017-09-11 00:19:38 0:48:00 0:21:53 0:26:07 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 1617306 2017-09-10 22:30:01 2017-09-10 23:33:03 2017-09-10 23:49:02 0:15:59 0:10:03 0:05:56 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-comp.yaml tasks/failover.yaml} 2
fail 1617307 2017-09-10 22:30:02 2017-09-10 23:33:03 2017-09-11 00:01:03 0:28:00 0:22:21 0:05:39 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi172 with status 13: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-sage-testing-2017-09-10-1525 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

pass 1617308 2017-09-10 22:30:03 2017-09-10 23:33:19 2017-09-11 00:19:19 0:46:00 0:35:45 0:10:15 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 1617309 2017-09-10 22:30:03 2017-09-10 23:34:12 2017-09-10 23:54:12 0:20:00 0:11:36 0:08:24 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617310 2017-09-10 22:30:04 2017-09-10 23:35:00 2017-09-11 00:41:01 1:06:01 0:31:50 0:34:11 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/sync-many.yaml workloads/snaps-few-objects.yaml} 2
pass 1617311 2017-09-10 22:30:05 2017-09-10 23:35:02 2017-09-10 23:57:02 0:22:00 0:13:56 0:08:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617312 2017-09-10 22:30:06 2017-09-10 23:35:33 2017-09-11 00:17:33 0:42:00 0:39:45 0:02:15 smithi master rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 1617313 2017-09-10 22:30:06 2017-09-10 23:35:33 2017-09-10 23:51:32 0:15:59 0:14:02 0:01:57 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_stress_watch.yaml} 2
dead 1617314 2017-09-10 22:30:07 2017-09-10 23:35:35 2017-09-11 11:40:59 12:05:24 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 1617315 2017-09-10 22:30:08 2017-09-10 23:36:06 2017-09-11 11:42:09 12:06:03 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617316 2017-09-10 22:30:08 2017-09-10 23:37:09 2017-09-10 23:47:08 0:09:59 0:07:12 0:02:47 smithi master rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml} 1
pass 1617317 2017-09-10 22:30:09 2017-09-10 23:37:39 2017-09-11 00:01:39 0:24:00 0:12:38 0:11:22 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617318 2017-09-10 22:30:10 2017-09-10 23:39:44 2017-09-11 00:33:45 0:54:01 0:50:12 0:03:49 smithi master rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 1617319 2017-09-10 22:30:11 2017-09-10 23:40:54 2017-09-10 23:58:53 0:17:59 0:12:50 0:05:09 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
dead 1617320 2017-09-10 22:30:11 2017-09-10 23:40:57 2017-09-11 11:47:47 12:06:50 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 1617321 2017-09-10 22:30:12 2017-09-10 23:41:20 2017-09-11 00:21:20 0:40:00 0:20:14 0:19:46 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617322 2017-09-10 22:30:13 2017-09-10 23:41:49 2017-09-11 00:37:49 0:56:00 0:16:19 0:39:41 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1617323 2017-09-10 22:30:13 2017-09-10 23:42:22 2017-09-11 04:00:27 4:18:05 4:11:49 0:06:16 smithi master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
Failure Reason:

SELinux denials found on ubuntu@smithi130.front.sepia.ceph.com: ['type=AVC msg=audit(1505101501.878:5442): avc: denied { open } for pid=1992 comm="logrotate" path="/var/lib/logrotate/logrotate.status" dev="sda1" ino=29624804 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101502.147:5444): avc: denied { setattr } for pid=1992 comm="logrotate" name="logrotate.status.tmp" dev="sda1" ino=29624518 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101502.147:5443): avc: denied { write } for pid=1992 comm="logrotate" path="/var/lib/logrotate/logrotate.status.tmp" dev="sda1" ino=29624518 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101502.147:5443): avc: denied { create } for pid=1992 comm="logrotate" name="logrotate.status.tmp" scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101502.252:5445): avc: denied { rename } for pid=1992 comm="logrotate" name="logrotate.status.tmp" dev="sda1" ino=29624518 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101501.878:5442): avc: denied { read } for pid=1992 comm="logrotate" name="logrotate.status" dev="sda1" ino=29624804 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file', 'type=AVC msg=audit(1505101502.252:5445): avc: denied { unlink } for pid=1992 comm="logrotate" name="logrotate.status" dev="sda1" ino=29624804 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file']

pass 1617324 2017-09-10 22:30:14 2017-09-10 23:42:22 2017-09-10 23:56:21 0:13:59 0:11:40 0:02:19 smithi master rados/standalone/misc.yaml 1
dead 1617325 2017-09-10 22:30:15 2017-09-10 23:44:50 2017-09-11 11:51:37 12:06:47 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
dead 1617326 2017-09-10 22:30:16 2017-09-10 23:45:02 2017-09-11 11:51:58 12:06:56 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 1617327 2017-09-10 22:30:16 2017-09-10 23:45:24 2017-09-11 00:17:24 0:32:00 0:30:25 0:01:35 smithi master rados/singleton/{all/osd-recovery.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1617328 2017-09-10 22:30:17 2017-09-10 23:46:26 2017-09-11 11:52:48 12:06:22 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
pass 1617329 2017-09-10 22:30:18 2017-09-10 23:46:27 2017-09-11 00:02:27 0:16:00 0:08:18 0:07:42 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_striper.yaml} 2
fail 1617330 2017-09-10 22:30:18 2017-09-10 23:46:50 2017-09-11 00:58:51 1:12:01 0:20:01 0:52:00 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2017-09-11 00:53:38.485970 osd.2 osd.2 172.21.15.34:6801/25845 1 : cluster [ERR] osd.2 found snap mapper error on pg 3.3 oid 3:d47134b7:::smithi03429108-44:1d9 snaps in mapper: 1cb,1cf,1d1,1d5,1d7, oi: 1cb,1cd,1cf,1d1,1d2,1d3,1d5,1d6,1d7...repaired" in cluster log

pass 1617331 2017-09-10 22:30:19 2017-09-10 23:46:56 2017-09-11 00:22:56 0:36:00 0:18:52 0:17:08 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/sync-many.yaml workloads/pool-create-delete.yaml} 2
dead 1617332 2017-09-10 22:30:20 2017-09-10 23:47:10 2017-09-11 11:52:49 12:05:39 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 1617333 2017-09-10 22:30:21 2017-09-10 23:49:09 2017-09-11 00:05:08 0:15:59 0:12:38 0:03:21 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
fail 1617334 2017-09-10 22:30:21 2017-09-10 23:49:09 2017-09-11 02:13:11 2:24:02 0:43:55 1:40:07 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2017-09-11 01:32:18.809162 osd.2 osd.2 172.21.15.15:6801/988 33 : cluster [ERR] osd.2 found snap mapper error on pg 2.e oid 2:7234f015:::smithi0154373-33 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:21 snaps in mapper: 1d, oi: 18,1d,20...repaired" in cluster log

pass 1617335 2017-09-10 22:30:22 2017-09-10 23:49:48 2017-09-10 23:59:47 0:09:59 0:06:47 0:03:12 smithi master rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml} 1
fail 1617336 2017-09-10 22:30:23 2017-09-10 23:50:30 2017-09-11 00:36:30 0:46:00 0:11:39 0:34:21 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
Failure Reason:

"2017-09-11 00:29:03.455642 osd.1 osd.1 172.21.15.151:6809/28085 28 : cluster [ERR] osd.1 found snap mapper error on pg 2.cs5 oid 2:3a0ae382:::smithi15132245-47 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:28 snaps in mapper: 1d,23,28, oi: 1c,1d,1f,23,25,28...repaired" in cluster log

pass 1617337 2017-09-10 22:30:23 2017-09-10 23:51:42 2017-09-11 00:33:42 0:42:00 0:21:41 0:20:19 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
fail 1617338 2017-09-10 22:30:24 2017-09-10 23:54:22 2017-09-11 00:42:22 0:48:00 0:43:38 0:04:22 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

Command failed on smithi201 with status 22: u'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 2.8'

pass 1617339 2017-09-10 22:30:25 2017-09-10 23:54:42 2017-09-11 00:04:41 0:09:59 0:07:40 0:02:19 smithi master rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml} 1
pass 1617340 2017-09-10 22:30:25 2017-09-10 23:56:34 2017-09-11 01:52:35 1:56:01 0:13:31 1:42:30 smithi master rados/multimon/{clusters/9.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/mon_recovery.yaml} 3
dead 1617341 2017-09-10 22:30:26 2017-09-10 23:56:55 2017-09-11 12:03:03 12:06:08 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} 2
pass 1617342 2017-09-10 22:30:27 2017-09-10 23:57:03 2017-09-11 00:23:02 0:25:59 0:21:33 0:04:26 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
pass 1617343 2017-09-10 22:30:27 2017-09-10 23:57:26 2017-09-11 00:17:25 0:19:59 0:11:48 0:08:11 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1617344 2017-09-10 22:30:28 2017-09-10 23:58:29 2017-09-11 12:05:20 12:06:51 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617345 2017-09-10 22:30:29 2017-09-10 23:59:03 2017-09-11 00:07:03 0:08:00 0:05:55 0:02:05 smithi master rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 1617346 2017-09-10 22:30:29 2017-09-10 23:59:59 2017-09-11 00:49:59 0:50:00 0:28:10 0:21:50 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml tasks/rados_workunit_loadgen_big.yaml} 2
fail 1617347 2017-09-10 22:30:30 2017-09-11 00:01:17 2017-09-11 00:35:17 0:34:00 0:26:33 0:07:27 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

Socket is closed

dead 1617348 2017-09-10 22:30:31 2017-09-11 00:01:42 2017-09-11 12:08:04 12:06:22 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617349 2017-09-10 22:30:31 2017-09-11 00:02:44 2017-09-11 00:32:43 0:29:59 0:14:05 0:15:54 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617350 2017-09-10 22:30:32 2017-09-11 00:02:52 2017-09-11 04:52:57 4:50:05 4:48:25 0:01:40 smithi master rados/objectstore/filestore-idempotent.yaml 1
pass 1617351 2017-09-10 22:30:33 2017-09-11 00:04:52 2017-09-11 00:24:51 0:19:59 0:17:12 0:02:47 smithi master rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 1617352 2017-09-10 22:30:33 2017-09-11 00:05:10 2017-09-11 00:41:10 0:36:00 0:10:51 0:25:09 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 1617353 2017-09-10 22:30:34 2017-09-11 00:07:18 2017-09-11 00:31:18 0:24:00 0:09:07 0:14:53 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml thrashers/sync.yaml workloads/rados_5925.yaml} 2
pass 1617354 2017-09-10 22:30:35 2017-09-11 00:09:48 2017-09-11 01:01:48 0:52:00 0:15:10 0:36:50 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617355 2017-09-10 22:30:36 2017-09-11 00:17:48 2017-09-11 00:53:48 0:36:00 0:14:35 0:21:25 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 1617356 2017-09-10 22:30:36 2017-09-11 00:17:49 2017-09-11 12:23:16 12:05:27 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1617357 2017-09-10 22:30:37 2017-09-11 00:17:48 2017-09-11 12:24:10 12:06:22 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml supported/ubuntu_latest.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 1617358 2017-09-10 22:30:38 2017-09-11 00:19:31 2017-09-11 00:53:33 0:34:02 0:21:02 0:13:00 smithi master rados/singleton/{all/random-eio.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
dead 1617359 2017-09-10 22:30:39 2017-09-11 00:19:39 2017-09-11 12:26:14 12:06:35 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
pass 1617360 2017-09-10 22:30:40 2017-09-11 00:21:19 2017-09-11 00:41:18 0:19:59 0:17:54 0:02:05 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
dead 1617361 2017-09-10 22:30:40 2017-09-11 00:21:21 2017-09-11 12:26:48 12:05:27 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 1617362 2017-09-10 22:30:41 2017-09-11 00:23:06 2017-09-11 00:53:06 0:30:00 0:08:53 0:21:07 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore.yaml tasks/failover.yaml} 2
pass 1617363 2017-09-10 22:30:42 2017-09-11 00:23:06 2017-09-11 00:49:06 0:26:00 0:24:56 0:01:04 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 1617364 2017-09-10 22:30:42 2017-09-11 00:25:02 2017-09-11 00:43:01 0:17:59 0:08:05 0:09:54 smithi master rados/singleton-nomsgr/{all/health-warnings.yaml rados.yaml} 1
pass 1617365 2017-09-10 22:30:43 2017-09-11 00:31:33 2017-09-11 00:45:32 0:13:59 0:09:05 0:04:54 smithi master rados/standalone/mon.yaml 1
fail 1617366 2017-09-10 22:30:44 2017-09-11 00:32:54 2017-09-11 02:02:55 1:30:01 0:09:54 1:20:07 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

"2017-09-11 01:53:50.825456 osd.7 osd.7 172.21.15.76:6813/25634 28 : cluster [ERR] osd.7 found snap mapper error on pg 2.bs2 oid 2:dae21110:::smithi07626149-21 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:25 snaps in mapper: 24, oi: 1f,24,25...repaired" in cluster log

pass 1617367 2017-09-10 22:30:44 2017-09-11 00:33:43 2017-09-11 01:37:44 1:04:01 0:11:49 0:52:12 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1617368 2017-09-10 22:30:45 2017-09-11 00:33:46 2017-09-11 12:40:15 12:06:29 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617369 2017-09-10 22:30:46 2017-09-11 00:35:29 2017-09-11 00:57:29 0:22:00 0:10:09 0:11:51 smithi master rados/singleton/{all/rebuild-mondb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617370 2017-09-10 22:30:46 2017-09-11 00:36:44 2017-09-11 01:34:45 0:58:01 0:38:43 0:19:18 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} 2
fail 1617371 2017-09-10 22:30:47 2017-09-11 00:37:59 2017-09-11 01:13:58 0:35:59 0:21:56 0:14:03 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2017-09-11 00:51:37.379360 mon.b mon.0 172.21.15.50:6789/0 10 : cluster [WRN] message from mon.1 was stamped 0.517413s in the future, clocks not synchronized" in cluster log

pass 1617372 2017-09-10 22:30:48 2017-09-11 00:41:13 2017-09-11 01:41:13 1:00:00 0:41:23 0:18:37 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 1617373 2017-09-10 22:30:48 2017-09-11 00:41:13 2017-09-11 00:59:12 0:17:59 0:10:43 0:07:16 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1617374 2017-09-10 22:30:49 2017-09-11 00:41:19 2017-09-11 12:47:59 12:06:40 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 1617375 2017-09-10 22:30:50 2017-09-11 00:42:35 2017-09-11 01:18:35 0:36:00 0:23:57 0:12:03 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_api_tests.yaml} 2
pass 1617376 2017-09-10 22:30:50 2017-09-11 00:42:51 2017-09-11 01:24:51 0:42:00 0:11:39 0:30:21 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617377 2017-09-10 22:30:51 2017-09-11 00:43:02 2017-09-11 00:51:02 0:08:00 0:07:14 0:00:46 smithi master rados/singleton/{all/reg11184.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 1617378 2017-09-10 22:30:51 2017-09-11 00:45:41 2017-09-11 00:51:40 0:05:59 0:04:46 0:01:13 smithi master rados/objectstore/fusestore.yaml 1
fail 1617379 2017-09-10 22:30:52 2017-09-11 00:46:52 2017-09-11 01:24:52 0:38:00 0:18:21 0:19:39 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2017-09-11 01:07:30.787708 osd.7 osd.7 172.21.15.76:6808/28016 50 : cluster [ERR] osd.7 found snap mapper error on pg 2.1 oid 2:82cc61f5:::smithi1059551-200:3f snaps in mapper: 2e,33,34,37,39,3f, oi: 2e,33,34,35,37,39,3a,3f...repaired" in cluster log

pass 1617380 2017-09-10 22:30:53 2017-09-11 00:47:36 2017-09-11 01:13:36 0:26:00 0:21:26 0:04:34 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
dead 1617381 2017-09-10 22:30:53 2017-09-11 00:49:15 2017-09-11 12:54:35 12:05:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 1617382 2017-09-10 22:30:54 2017-09-11 00:50:01 2017-09-11 02:46:02 1:56:01 0:13:13 1:42:48 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617383 2017-09-10 22:30:55 2017-09-11 00:51:12 2017-09-11 01:23:12 0:32:00 0:10:51 0:21:09 smithi master rados/singleton/{all/resolve_stuck_peering.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 2
pass 1617384 2017-09-10 22:30:56 2017-09-11 00:51:41 2017-09-11 01:11:41 0:20:00 0:14:21 0:05:39 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 1617385 2017-09-10 22:30:56 2017-09-11 00:51:55 2017-09-11 01:47:55 0:56:00 0:07:50 0:48:10 smithi master rados/multimon/{clusters/21.yaml mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 3
dead 1617386 2017-09-10 22:30:57 2017-09-11 00:53:18 2017-09-11 12:59:12 12:05:54 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} 2
pass 1617387 2017-09-10 22:30:57 2017-09-11 00:53:34 2017-09-11 01:23:33 0:29:59 0:10:21 0:19:38 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
pass 1617388 2017-09-10 22:30:58 2017-09-11 00:53:52 2017-09-11 01:47:52 0:54:00 0:22:53 0:31:07 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617389 2017-09-10 22:30:59 2017-09-11 00:54:56 2017-09-11 01:26:56 0:32:00 0:25:52 0:06:08 smithi master rados/singleton-nomsgr/{all/msgr.yaml rados.yaml} 1
pass 1617390 2017-09-10 22:31:00 2017-09-11 00:56:44 2017-09-11 01:24:44 0:28:00 0:10:37 0:17:23 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 1617391 2017-09-10 22:31:00 2017-09-11 00:57:30 2017-09-11 13:04:00 12:06:30 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 1617392 2017-09-10 22:31:01 2017-09-11 00:59:01 2017-09-11 01:11:00 0:11:59 0:08:16 0:03:43 smithi master rados/singleton/{all/rest-api.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1617393 2017-09-10 22:31:02 2017-09-11 00:59:14 2017-09-11 13:05:34 12:06:20 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
pass 1617394 2017-09-10 22:31:02 2017-09-11 01:01:58 2017-09-11 02:01:59 1:00:01 0:12:31 0:47:30 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/readwrite.yaml} 2
dead 1617395 2017-09-10 22:31:03 2017-09-11 01:02:45 2017-09-11 13:09:38 12:06:53 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 1617396 2017-09-10 22:31:04 2017-09-11 01:11:11 2017-09-11 01:39:11 0:28:00 0:13:06 0:14:54 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/many.yaml workloads/rados_mon_workunits.yaml} 2
pass 1617397 2017-09-10 22:31:04 2017-09-11 01:11:42 2017-09-11 01:25:42 0:14:00 0:10:04 0:03:56 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1617398 2017-09-10 22:31:05 2017-09-11 01:13:19 2017-09-11 13:18:41 12:05:22 smithi master centos 7.3 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml supported/centos_latest.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 1617399 2017-09-10 22:31:05 2017-09-11 01:13:37 2017-09-11 01:47:37 0:34:00 0:26:30 0:07:30 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617400 2017-09-10 22:31:06 2017-09-11 01:14:01 2017-09-11 04:36:04 3:22:03 3:20:04 0:01:59 smithi master rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617401 2017-09-10 22:31:07 2017-09-11 01:14:44 2017-09-11 01:56:44 0:42:00 0:22:46 0:19:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 1617402 2017-09-10 22:31:07 2017-09-11 01:18:44 2017-09-11 01:30:44 0:12:00 0:06:02 0:05:58 smithi master rados/objectstore/keyvaluedb.yaml 1
pass 1617403 2017-09-10 22:31:08 2017-09-11 01:23:26 2017-09-11 01:39:25 0:15:59 0:13:54 0:02:05 smithi master rados/standalone/osd.yaml 1
dead 1617404 2017-09-10 22:31:09 2017-09-11 01:23:35 2017-09-11 13:30:33 12:06:58 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 1617405 2017-09-10 22:31:09 2017-09-11 01:24:56 2017-09-11 01:54:55 0:29:59 0:21:25 0:08:34 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
fail 1617406 2017-09-10 22:31:10 2017-09-11 01:24:56 2017-09-11 02:52:57 1:28:01 0:18:09 1:09:52 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
Failure Reason:

"2017-09-11 02:38:38.563284 osd.4 osd.4 172.21.15.205:6800/550 54 : cluster [ERR] osd.4 found snap mapper error on pg 2.fs0 oid 2:f4a1ffc7:::smithi18524480-436:29 snaps in mapper: a,c,12,20,21,23,28,29, oi: 4,a,c,12,1e,20,21,23,28,29...repaired" in cluster log

pass 1617407 2017-09-10 22:31:11 2017-09-11 01:24:56 2017-09-11 01:40:55 0:15:59 0:13:45 0:02:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617408 2017-09-10 22:31:11 2017-09-11 01:25:43 2017-09-11 01:53:43 0:28:00 0:14:04 0:13:56 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617409 2017-09-10 22:31:12 2017-09-11 01:27:09 2017-09-11 01:57:08 0:29:59 0:21:02 0:08:57 smithi master rados/singleton/{all/thrash-eio.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
pass 1617410 2017-09-10 22:31:13 2017-09-11 01:28:24 2017-09-11 02:38:25 1:10:01 0:18:25 0:51:36 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
pass 1617411 2017-09-10 22:31:14 2017-09-11 01:30:54 2017-09-11 01:48:53 0:17:59 0:15:48 0:02:11 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml tasks/repair_test.yaml} 2
pass 1617412 2017-09-10 22:31:15 2017-09-11 01:33:16 2017-09-11 01:55:16 0:22:00 0:19:57 0:02:03 smithi master rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml} 2
fail 1617413 2017-09-10 22:31:16 2017-09-11 01:34:57 2017-09-11 02:10:56 0:35:59 0:26:01 0:09:58 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2017-09-11 01:48:07.920957 osd.0 osd.0 172.21.15.143:6801/254211 43 : cluster [ERR] osd.0 found snap mapper error on pg 2.19 oid 2:9cc3d0d6:::smithi143257621-39 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:42 snaps in mapper: 41, oi: 41,42...repaired" in cluster log

fail 1617414 2017-09-10 22:31:16 2017-09-11 01:35:52 2017-09-11 02:27:53 0:52:01 0:11:32 0:40:29 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
Failure Reason:

"2017-09-11 02:19:27.957242 osd.5 osd.5 172.21.15.104:6805/101889 25 : cluster [ERR] osd.5 found snap mapper error on pg 2.9s4 oid 2:9b9e2d69:::smithi024125880-10:27 snaps in mapper: 1e, oi: 1e,20,25,27...repaired" in cluster log

pass 1617415 2017-09-10 22:31:17 2017-09-11 01:36:35 2017-09-11 02:32:35 0:56:00 0:14:34 0:41:26 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 1617416 2017-09-10 22:31:18 2017-09-11 01:37:57 2017-09-11 02:15:56 0:37:59 0:18:29 0:19:30 smithi master rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 2
pass 1617417 2017-09-10 22:31:18 2017-09-11 01:39:21 2017-09-11 02:11:21 0:32:00 0:17:45 0:14:15 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 1617418 2017-09-10 22:31:19 2017-09-11 01:39:22 2017-09-11 02:11:21 0:31:59 0:30:06 0:01:53 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 1617419 2017-09-10 22:31:20 2017-09-11 01:39:26 2017-09-11 03:07:27 1:28:01 0:40:59 0:47:02 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/one.yaml workloads/snaps-few-objects.yaml} 2
pass 1617420 2017-09-10 22:31:20 2017-09-11 01:41:06 2017-09-11 02:05:05 0:23:59 0:12:31 0:11:28 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1617421 2017-09-10 22:31:21 2017-09-11 01:41:14 2017-09-11 02:31:14 0:50:00 0:21:20 0:28:40 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2017-09-11 02:23:44.857026 osd.5 osd.5 172.21.15.50:6800/7070 2 : cluster [ERR] osd.5 found snap mapper error on pg 3.0 oid 3:18832e65:::smithi11498206-352:25f snaps in mapper: 24a,250, oi: 226,246,24a,24b,24c,250,259...repaired" in cluster log

pass 1617422 2017-09-10 22:31:22 2017-09-11 01:47:50 2017-09-11 02:07:50 0:20:00 0:09:41 0:10:19 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/filestore-xfs.yaml tasks/failover.yaml} 2
pass 1617423 2017-09-10 22:31:22 2017-09-11 01:47:54 2017-09-11 02:13:54 0:26:00 0:24:59 0:01:01 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617424 2017-09-10 22:31:23 2017-09-11 01:47:56 2017-09-11 02:09:56 0:22:00 0:19:38 0:02:22 smithi master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
fail 1617425 2017-09-10 22:31:24 2017-09-11 01:48:45 2017-09-11 02:42:45 0:54:00 0:46:55 0:07:05 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

failed to recover before timeout expired

dead 1617426 2017-09-10 22:31:24 2017-09-11 01:49:04 2017-09-11 13:56:10 12:07:06 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
fail 1617427 2017-09-10 22:31:25 2017-09-11 01:49:49 2017-09-11 02:13:49 0:24:00 0:14:51 0:09:09 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rgw_snaps.yaml} 2
Failure Reason:

"2017-09-11 02:06:36.542220 osd.5 osd.5 172.21.15.189:6805/26929 2 : cluster [ERR] osd.5 found snap mapper error on pg 5.7 oid 5:e54c7651:users.uid::foo.client.0:9 snaps in mapper: 1,2,3,4,6,7,8,9, oi: 1,2,3,4,5,6,7,8,9...repaired" in cluster log

fail 1617428 2017-09-10 22:31:26 2017-09-11 01:52:46 2017-09-11 02:58:47 1:06:01 0:52:40 0:13:21 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache.yaml} 2
Failure Reason:

failed to become clean before timeout expired

pass 1617429 2017-09-10 22:31:26 2017-09-11 01:53:44 2017-09-11 02:09:43 0:15:59 0:15:35 0:00:24 smithi master rados/objectstore/objectcacher-stress.yaml 1
dead 1617430 2017-09-10 22:31:27 2017-09-11 01:55:05 2017-09-11 14:00:55 12:05:50 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 1617431 2017-09-10 22:31:28 2017-09-11 01:55:22 2017-09-11 02:11:22 0:16:00 0:10:09 0:05:51 smithi master rados/multimon/{clusters/3.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 2
dead 1617432 2017-09-10 22:31:28 2017-09-11 01:56:54 2017-09-11 14:03:03 12:06:09 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/few.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
pass 1617433 2017-09-10 22:31:29 2017-09-11 01:57:10 2017-09-11 02:37:10 0:40:00 0:29:46 0:10:14 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml} 2
pass 1617434 2017-09-10 22:31:30 2017-09-11 02:01:47 2017-09-11 02:17:47 0:16:00 0:07:22 0:08:38 smithi master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml} 1
pass 1617435 2017-09-10 22:31:30 2017-09-11 02:02:01 2017-09-11 03:08:01 1:06:00 0:19:12 0:46:48 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 1617436 2017-09-10 22:31:31 2017-09-11 02:03:05 2017-09-11 02:17:04 0:13:59 0:06:54 0:07:05 smithi master rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml} 1
fail 1617437 2017-09-10 22:31:31 2017-09-11 02:05:19 2017-09-11 04:29:22 2:24:03 1:38:41 0:45:22 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 1617438 2017-09-10 22:31:32 2017-09-11 02:07:59 2017-09-11 04:12:01 2:04:02 0:23:56 1:40:06 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 1617439 2017-09-10 22:31:33 2017-09-11 02:09:54 2017-09-11 02:25:53 0:15:59 0:11:55 0:04:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 1617440 2017-09-10 22:31:33 2017-09-11 02:09:57 2017-09-11 02:25:56 0:15:59 0:12:32 0:03:27 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 1617441 2017-09-10 22:31:34 2017-09-11 02:11:06 2017-09-11 02:33:05 0:21:59 0:16:55 0:05:04 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml thrashers/one.yaml workloads/pool-create-delete.yaml} 2
pass 1617442 2017-09-10 22:31:35 2017-09-11 02:11:22 2017-09-11 02:17:21 0:05:59 0:05:23 0:00:36 smithi master rados/singleton/{all/admin-socket.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
dead 1617443 2017-09-10 22:31:35 2017-09-11 02:11:22 2017-09-11 14:17:43 12:06:21 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} 2
fail 1617444 2017-09-10 22:31:36 2017-09-11 02:11:23 2017-09-11 03:01:23 0:50:00 0:21:48 0:28:12 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2017-09-11 02:43:18.797546 osd.6 osd.6 172.21.15.181:6812/15697 36 : cluster [ERR] osd.6 found snap mapper error on pg 2.c oid 2:31f90279:::smithi0152396-734:16 snaps in mapper: 15, oi: 8,10,11,15,16...repaired" in cluster log

fail 1617445 2017-09-10 22:31:37 2017-09-11 02:13:23 2017-09-11 02:59:24 0:46:01 0:42:18 0:03:43 smithi master rados/standalone/scrub.yaml 1
Failure Reason:

Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi189 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-sage-testing-2017-09-10-1525 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh'

fail 1617446 2017-09-10 22:31:37 2017-09-11 02:13:50 2017-09-11 11:42:02 9:28:12 0:11:41 9:16:31 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

"2017-09-11 11:33:36.170933 osd.0 osd.0 172.21.15.137:6804/21958 29 : cluster [ERR] osd.0 found snap mapper error on pg 2.ds0 oid 2:bd4cb8bb:::smithi1684697-22:2a snaps in mapper: 9,f,16,23,26, oi: 9,f,16,19,1e,23,26,27,28,29,2a...repaired" in cluster log

pass 1617447 2017-09-10 22:31:38 2017-09-11 02:13:55 2017-09-11 03:11:55 0:58:00 0:10:23 0:47:37 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/scrub_test.yaml} 2
pass 1617448 2017-09-10 22:31:39 2017-09-11 02:16:07 2017-09-11 03:10:07 0:54:00 0:26:43 0:27:17 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2