Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 1673317 2017-09-26 02:25:51 2017-09-26 06:07:14 2017-09-26 17:09:27 11:02:13 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported/ubuntu_latest.yaml thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 1673321 2017-09-26 02:25:52 2017-09-26 06:07:19 2017-09-26 17:09:33 11:02:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
dead 1673324 2017-09-26 02:25:53 2017-09-26 06:07:26 2017-09-26 17:09:40 11:02:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1673328 2017-09-26 02:25:54 2017-09-26 06:07:41 2017-09-26 06:27:40 0:19:59 0:13:36 0:06:23 smithi master rados/singleton/{all/rest-api.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
Failure Reason:

Command failed on smithi045 with status 11: u'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 1.1'

dead 1673332 2017-09-26 02:25:55 2017-09-26 06:07:47 2017-09-26 17:10:01 11:02:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1673333 2017-09-26 02:25:56 2017-09-26 06:07:48 2017-09-26 17:10:01 11:02:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1673334 2017-09-26 02:25:57 2017-09-26 06:08:25 2017-09-26 17:10:40 11:02:15 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1673335 2017-09-26 02:25:58 2017-09-26 06:08:25 2017-09-26 17:10:40 11:02:15 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1673336 2017-09-26 02:25:59 2017-09-26 06:09:39 2017-09-26 17:09:52 11:00:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 1673337 2017-09-26 02:26:00 2017-09-26 06:09:49 2017-09-26 09:33:52 3:24:03 3:18:43 0:05:20 smithi master rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
Failure Reason:

Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi091 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

fail 1673338 2017-09-26 02:26:01 2017-09-26 06:10:03 2017-09-26 09:38:06 3:28:03 3:20:14 0:07:49 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi096 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 1673339 2017-09-26 02:26:02 2017-09-26 06:11:46 2017-09-26 07:29:47 1:18:01 1:10:55 0:07:06 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (500) after waiting for 3000 seconds

fail 1673340 2017-09-26 02:26:04 2017-09-26 06:11:46 2017-09-26 06:43:46 0:32:00 0:24:43 0:07:17 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rgw_snaps.yaml} 2
Failure Reason:

HTTPConnectionPool(host='smithi152.front.sepia.ceph.com', port=7280): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f61160cff10>: Failed to establish a new connection: [Errno 111] Connection refused',))

fail 1673341 2017-09-26 02:26:05 2017-09-26 06:11:46 2017-09-26 06:43:46 0:32:00 0:25:55 0:06:05 smithi master rados/multimon/{clusters/9.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/mon_recovery.yaml} 3
Failure Reason:

Exiting scrub checking -- not all pgs scrubbed.

dead 1673342 2017-09-26 02:26:06 2017-09-26 06:11:56 2017-09-26 17:10:10 10:58:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1673343 2017-09-26 02:26:07 2017-09-26 06:12:07 2017-09-26 17:10:21 10:58:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
dead 1673344 2017-09-26 02:26:08 2017-09-26 06:12:44 2017-09-26 17:10:58 10:58:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 1673345 2017-09-26 02:26:09 2017-09-26 06:13:18 2017-09-26 17:09:33 10:56:15 smithi master rados/singleton/{all/thrash-eio.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
dead 1673346 2017-09-26 02:26:10 2017-09-26 06:13:18 2017-09-26 17:09:33 10:56:15 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-bitmap.yaml tasks/failover.yaml} 2
fail 1673347 2017-09-26 02:26:11 2017-09-26 06:13:47 2017-09-26 09:37:51 3:24:04 3:18:08 0:05:56 smithi master rados/objectstore/alloc-hint.yaml 1
Failure Reason:

Command failed (workunit test rados/test_alloc_hint.sh) on smithi054 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_alloc_hint.sh'

dead 1673348 2017-09-26 02:26:12 2017-09-26 06:14:51 2017-09-26 17:11:05 10:56:14 smithi master rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml settings/optimized.yaml workloads/sample_fio.yaml} 1
dead 1673349 2017-09-26 02:26:13 2017-09-26 06:15:16 2017-09-26 17:09:30 10:54:14 smithi master rados/rest/mgr-restful.yaml 1
fail 1673350 2017-09-26 02:26:14 2017-09-26 06:15:17 2017-09-26 06:41:17 0:26:00 0:21:28 0:04:32 smithi master rados/singleton-nomsgr/{all/admin_socket_output.yaml rados.yaml} 1
Failure Reason:

"2017-09-26 06:33:56.934911 mon.a mon.0 172.21.15.202:6789/0 406 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 1673351 2017-09-26 02:26:15 2017-09-26 06:15:33 2017-09-26 06:25:32 0:09:59 0:05:38 0:04:21 smithi master rados/standalone/crush.yaml 1
Failure Reason:

Command failed (workunit test crush/crush-choose-args.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh'

dead 1673352 2017-09-26 02:26:17 2017-09-26 06:15:47 2017-09-26 17:10:01 10:54:14 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
fail 1673353 2017-09-26 02:26:18 2017-09-26 06:17:20 2017-09-26 09:47:24 3:30:04 3:23:56 0:06:08 smithi master rados/upgrade/luminous-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/luminous.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-mimic.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} thrashosds-health.yaml} 3
Failure Reason:

failed to recover before timeout expired

fail 1673354 2017-09-26 02:26:19 2017-09-26 06:17:20 2017-09-26 07:05:20 0:48:00 0:41:28 0:06:32 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
Failure Reason:

failed to recover before timeout expired

dead 1673355 2017-09-26 02:26:20 2017-09-26 06:17:26 2017-09-26 17:09:40 10:52:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
dead 1673356 2017-09-26 02:26:21 2017-09-26 06:19:41 2017-09-26 17:09:55 10:50:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 1673357 2017-09-26 02:26:22 2017-09-26 06:19:52 2017-09-26 06:29:51 0:09:59 0:06:20 0:03:39 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

Command failed on smithi201 with status 1: 'sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmpm91W1B'

fail 1673358 2017-09-26 02:26:23 2017-09-26 06:21:08 2017-09-26 06:55:08 0:34:00 0:27:54 0:06:06 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml} 2
Failure Reason:

"2017-09-26 06:26:24.559438 mon.a mon.1 172.21.15.70:6789/0 114 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

dead 1673359 2017-09-26 02:26:24 2017-09-26 06:21:47 2017-09-26 17:10:00 10:48:13 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
dead 1673360 2017-09-26 02:26:26 2017-09-26 06:22:29 2017-09-26 17:10:43 10:48:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
fail 1673361 2017-09-26 02:26:27 2017-09-26 06:23:23 2017-09-26 09:37:26 3:14:03 3:08:03 0:06:00 smithi master rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml} 2
Failure Reason:

Command failed (workunit test rados/load-gen-mix-small.sh) on smithi192 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/load-gen-mix-small.sh'

dead 1673362 2017-09-26 02:26:28 2017-09-26 06:24:27 2017-09-26 17:10:41 10:46:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 1673363 2017-09-26 02:26:30 2017-09-26 06:25:35 2017-09-26 17:09:49 10:44:14 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1673364 2017-09-26 02:26:31 2017-09-26 06:25:36 2017-09-26 17:09:49 10:44:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1673365 2017-09-26 02:26:32 2017-09-26 06:25:36 2017-09-26 17:09:49 10:44:13 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1673366 2017-09-26 02:26:33 2017-09-26 06:27:41 2017-09-26 17:09:55 10:42:14 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/scrub_test.yaml} 2
dead 1673367 2017-09-26 02:26:34 2017-09-26 06:29:11 2017-09-26 17:09:24 10:40:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1673368 2017-09-26 02:26:35 2017-09-26 06:29:11 2017-09-26 17:09:25 10:40:14 smithi master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 2
dead 1673369 2017-09-26 02:26:36 2017-09-26 06:29:20 2017-09-26 17:09:33 10:40:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
dead 1673370 2017-09-26 02:26:37 2017-09-26 06:29:52 2017-09-26 17:10:08 10:40:16 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
dead 1673371 2017-09-26 02:26:38 2017-09-26 06:34:11 2017-09-26 17:10:26 10:36:15 smithi master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml supported/centos_latest.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 1673372 2017-09-26 02:26:40 2017-09-26 06:34:11 2017-09-26 10:02:15 3:28:04 3:20:46 0:07:18 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi184 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 1673373 2017-09-26 02:26:41 2017-09-26 06:34:37 2017-09-26 07:52:37 1:18:00 1:11:18 0:06:42 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (500) after waiting for 3000 seconds

dead 1673374 2017-09-26 02:26:42 2017-09-26 06:35:54 2017-09-26 17:10:08 10:34:14 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} 2
dead 1673375 2017-09-26 02:26:43 2017-09-26 06:39:12 2017-09-26 17:09:25 10:30:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1673376 2017-09-26 02:26:44 2017-09-26 06:41:18 2017-09-26 17:09:33 10:28:15 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
dead 1673377 2017-09-26 02:26:45 2017-09-26 06:43:57 2017-09-26 17:10:10 10:26:13 smithi master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
dead 1673378 2017-09-26 02:26:46 2017-09-26 06:43:57 2017-09-26 17:10:10 10:26:13 smithi master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml} 1
dead 1673379 2017-09-26 02:26:47 2017-09-26 06:47:26 2017-09-26 17:09:39 10:22:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
fail 1673380 2017-09-26 02:26:48 2017-09-26 06:49:19 2017-09-26 10:03:22 3:14:03 3:07:57 0:06:06 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi151 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

dead 1673381 2017-09-26 02:26:49 2017-09-26 06:53:36 2017-09-26 17:09:49 10:16:13 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
dead 1673382 2017-09-26 02:26:50 2017-09-26 06:55:09 2017-09-26 17:09:22 10:14:13 smithi master rados/objectstore/ceph_objectstore_tool.yaml 1
dead 1673383 2017-09-26 02:26:51 2017-09-26 07:05:21 2017-09-26 17:09:35 10:04:14 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 1673384 2017-09-26 02:26:52 2017-09-26 07:16:19 2017-09-26 09:04:21 1:48:02 1:41:30 0:06:32 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

reached maximum tries (800) after waiting for 4800 seconds

fail 1673385 2017-09-26 02:26:53 2017-09-26 07:17:42 2017-09-26 07:35:42 0:18:00 0:14:00 0:04:00 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-comp.yaml tasks/workunits.yaml} 2
Failure Reason:

Timed out waiting for MDS daemons to become healthy

dead 1673386 2017-09-26 02:26:54 2017-09-26 07:22:13 2017-09-26 17:10:26 9:48:13 smithi master rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml settings/optimized.yaml workloads/sample_radosbench.yaml} 1
fail 1673387 2017-09-26 02:26:55 2017-09-26 07:39:49 2017-09-26 11:07:53 3:28:04 3:21:01 0:07:03 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi117 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 1673388 2017-09-26 02:26:56 2017-09-26 07:55:44 2017-09-26 08:21:44 0:26:00 0:20:30 0:05:30 smithi master rados/multimon/{clusters/21.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 3
Failure Reason:

"2017-09-26 08:01:24.501007 mon.a mon.0 172.21.15.99:6789/0 75 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1673389 2017-09-26 02:26:58 2017-09-26 07:55:44 2017-09-26 08:19:44 0:24:00 0:19:54 0:04:06 smithi master rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
Failure Reason:

"2017-09-26 07:59:58.868566 mon.a mon.0 172.21.15.200:6789/0 74 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 1673390 2017-09-26 02:26:59 2017-09-26 07:55:44 2017-09-26 17:09:55 9:14:11 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
dead 1673391 2017-09-26 02:27:00 2017-09-26 13:05:51 2017-09-26 17:09:55 4:04:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 1673392 2017-09-26 02:27:01 2017-09-26 13:05:51 2017-09-26 17:09:56 4:04:05 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1673393 2017-09-26 02:27:02 2017-09-26 13:06:14 2017-09-26 17:10:19 4:04:05 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1673394 2017-09-26 02:27:03 2017-09-26 13:06:50 2017-09-26 17:10:54 4:04:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1673395 2017-09-26 02:27:04 2017-09-26 13:06:51 2017-09-26 17:10:56 4:04:05 smithi master rados/singleton/{all/divergent_priors.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml} 1
dead 1673396 2017-09-26 02:27:05 2017-09-26 13:06:59 2017-09-26 17:11:03 4:04:04 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/many.yaml workloads/snaps-few-objects.yaml} 2
fail 1673397 2017-09-26 02:27:06 2017-09-26 13:07:46 2017-09-26 16:21:50 3:14:04 3:06:18 0:07:46 smithi master rados/standalone/erasure-code.yaml 1
Failure Reason:

Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi083 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh'

dead 1673398 2017-09-26 02:27:07 2017-09-26 13:08:36 2017-09-26 17:10:40 4:02:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1673399 2017-09-26 02:27:08 2017-09-26 13:09:34 2017-09-26 17:09:38 4:00:04 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} 2
dead 1673400 2017-09-26 02:27:09 2017-09-26 13:09:45 2017-09-26 17:09:49 4:00:04 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
dead 1673401 2017-09-26 02:27:10 2017-09-26 13:10:15 2017-09-26 17:10:19 4:00:04 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml} 2
dead 1673402 2017-09-26 02:27:11 2017-09-26 13:11:36 2017-09-26 17:09:40 3:58:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 1673403 2017-09-26 02:27:13 2017-09-26 13:11:36 2017-09-26 16:37:39 3:26:03 3:19:54 0:06:09 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi109 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 1673404 2017-09-26 02:27:14 2017-09-26 13:11:36 2017-09-26 14:29:37 1:18:01 1:12:04 0:05:57 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (500) after waiting for 3000 seconds

dead 1673405 2017-09-26 02:27:15 2017-09-26 13:11:38 2017-09-26 17:09:42 3:58:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1673406 2017-09-26 02:27:16 2017-09-26 13:12:01 2017-09-26 17:10:05 3:58:04 smithi master rados/singleton/{all/divergent_priors2.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 1673407 2017-09-26 02:27:17 2017-09-26 13:13:06 2017-09-26 13:19:06 0:06:00 0:05:15 0:00:45 smithi master rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml} 1
dead 1673408 2017-09-26 02:27:18 2017-09-26 13:13:30 2017-09-26 17:09:35 3:56:05 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
fail 1673409 2017-09-26 02:27:19 2017-09-26 13:13:35 2017-09-26 16:39:38 3:26:03 3:20:21 0:05:42 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml tasks/rados_cls_all.yaml} 2
Failure Reason:

Command failed (workunit test cls/test_cls_hello.sh) on smithi010 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_hello.sh'

dead 1673410 2017-09-26 02:27:20 2017-09-26 13:13:38 2017-09-26 17:09:42 3:56:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 1673411 2017-09-26 02:27:22 2017-09-26 13:13:45 2017-09-26 17:09:50 3:56:05 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
fail 1673412 2017-09-26 02:27:23 2017-09-26 13:14:30 2017-09-26 13:38:29 0:23:59 0:17:58 0:06:01 smithi master rados/objectstore/filejournal.yaml 1
Failure Reason:

"2017-09-26 13:18:11.054963 mon.a mon.0 172.21.15.174:6789/0 73 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 1673413 2017-09-26 02:27:24 2017-09-26 13:14:30 2017-09-26 17:10:34 3:56:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
fail 1673414 2017-09-26 02:27:25 2017-09-26 13:14:48 2017-09-26 13:40:48 0:26:00 0:20:05 0:05:55 smithi master rados/singleton/{all/dump-stuck.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
Failure Reason:

Exiting scrub checking -- not all pgs scrubbed.

fail 1673415 2017-09-26 02:27:26 2017-09-26 13:15:04 2017-09-26 15:03:05 1:48:01 1:40:49 0:07:12 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

reached maximum tries (800) after waiting for 4800 seconds

dead 1673416 2017-09-26 02:27:28 2017-09-26 13:15:45 2017-09-26 17:09:49 3:54:04 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
dead 1673417 2017-09-26 02:27:29 2017-09-26 13:15:45 2017-09-26 17:09:49 3:54:04 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore.yaml rados.yaml supported/ubuntu_14.04.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 1673418 2017-09-26 02:27:30 2017-09-26 13:15:48 2017-09-26 17:09:52 3:54:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
dead 1673419 2017-09-26 02:27:31 2017-09-26 13:18:00 2017-09-26 17:10:05 3:52:05 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1673420 2017-09-26 02:27:32 2017-09-26 13:18:00 2017-09-26 13:36:00 0:18:00 0:12:15 0:05:45 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore.yaml tasks/failover.yaml} 2
Failure Reason:

"2017-09-26 13:22:37.002621 mon.a mon.0 172.21.15.103:6789/0 129 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

dead 1673421 2017-09-26 02:27:33 2017-09-26 13:18:10 2017-09-26 17:10:14 3:52:04 smithi master rados/perf/{ceph.yaml objectstore/bluestore.yaml settings/optimized.yaml workloads/sample_fio.yaml} 1
dead 1673422 2017-09-26 02:27:34 2017-09-26 13:19:05 2017-09-26 17:11:09 3:52:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1673423 2017-09-26 02:27:35 2017-09-26 13:19:15 2017-09-26 17:09:19 3:50:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 1673424 2017-09-26 02:27:37 2017-09-26 13:19:41 2017-09-26 17:09:45 3:50:04 smithi master rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1673425 2017-09-26 02:27:38 2017-09-26 13:20:01 2017-09-26 17:10:05 3:50:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1673426 2017-09-26 02:27:39 2017-09-26 13:21:08 2017-09-26 17:11:12 3:50:04 smithi master rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/few.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} 2
dead 1673427 2017-09-26 02:27:40 2017-09-26 13:21:35 2017-09-26 17:09:39 3:48:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1673428 2017-09-26 02:27:41 2017-09-26 13:21:43 2017-09-26 17:09:47 3:48:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 1673429 2017-09-26 02:27:42 2017-09-26 13:21:47 2017-09-26 16:47:50 3:26:03 3:19:16 0:06:47 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_python.yaml} 2
Failure Reason:

Command failed (workunit test rados/test_python.sh) on smithi194 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh'

fail 1673430 2017-09-26 02:27:43 2017-09-26 13:21:48 2017-09-26 13:47:48 0:26:00 0:21:10 0:04:50 smithi master rados/multimon/{clusters/3.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 2
Failure Reason:

"2017-09-26 13:29:10.034695 mon.b mon.0 172.21.15.7:6789/0 58 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1673431 2017-09-26 02:27:45 2017-09-26 13:22:41 2017-09-26 16:48:44 3:26:03 3:18:52 0:07:11 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi054 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 1673432 2017-09-26 02:27:46 2017-09-26 13:23:28 2017-09-26 14:39:29 1:16:01 1:10:13 0:05:48 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (500) after waiting for 3000 seconds

pass 1673433 2017-09-26 02:27:47 2017-09-26 13:23:38 2017-09-26 13:29:37 0:05:59 0:05:26 0:00:33 smithi master rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml} 1
dead 1673434 2017-09-26 02:27:48 2017-09-26 13:24:07 2017-09-26 17:10:11 3:46:04 smithi master rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml} 1
dead 1673435 2017-09-26 02:27:49 2017-09-26 13:24:27 2017-09-26 17:10:31 3:46:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
dead 1673436 2017-09-26 02:27:50 2017-09-26 13:25:37 2017-09-26 17:09:41 3:44:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
dead 1673437 2017-09-26 02:27:51 2017-09-26 13:25:38 2017-09-26 17:09:42 3:44:04 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/many.yaml workloads/pool-create-delete.yaml} 2
dead 1673438 2017-09-26 02:27:52 2017-09-26 13:25:41 2017-09-26 17:09:45 3:44:04 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
dead 1673439 2017-09-26 02:27:53 2017-09-26 13:25:47 2017-09-26 17:09:51 3:44:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 1673440 2017-09-26 02:27:54 2017-09-26 13:26:26 2017-09-26 17:10:30 3:44:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
dead 1673441 2017-09-26 02:27:55 2017-09-26 13:27:00 2017-09-26 17:11:04 3:44:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
dead 1673442 2017-09-26 02:27:56 2017-09-26 13:29:40 2017-09-26 17:09:44 3:40:04 smithi master rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
dead 1673443 2017-09-26 02:27:57 2017-09-26 13:29:40 2017-09-26 17:09:44 3:40:04 smithi master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
fail 1673444 2017-09-26 02:27:58 2017-09-26 13:29:47 2017-09-26 16:39:50 3:10:03 3:05:12 0:04:51 smithi master rados/standalone/misc.yaml 1
Failure Reason:

Command failed (workunit test misc/rados-striper.sh) on smithi025 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh'

dead 1673445 2017-09-26 02:27:59 2017-09-26 13:29:47 2017-09-26 17:09:51 3:40:04 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
fail 1673446 2017-09-26 02:28:00 2017-09-26 13:30:10 2017-09-26 15:18:12 1:48:02 1:42:14 0:05:48 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

reached maximum tries (800) after waiting for 4800 seconds

dead 1673447 2017-09-26 02:28:01 2017-09-26 13:32:07 2017-09-26 17:10:11 3:38:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
fail 1673448 2017-09-26 02:28:02 2017-09-26 13:34:05 2017-09-26 17:00:09 3:26:04 3:19:50 0:06:14 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} 2
Failure Reason:

Command failed (workunit test cls/test_cls_hello.sh) on smithi103 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_hello.sh'

dead 1673449 2017-09-26 02:28:03 2017-09-26 13:36:01 2017-09-26 17:10:05 3:34:04 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 1673450 2017-09-26 02:28:04 2017-09-26 13:38:33 2017-09-26 16:50:36 3:12:03 3:07:40 0:04:23 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi031 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 1673451 2017-09-26 02:28:05 2017-09-26 13:38:33 2017-09-26 17:04:36 3:26:03 3:18:24 0:07:39 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_stress_watch.yaml} 2
Failure Reason:

Command failed (workunit test rados/stress_watch.sh) on smithi174 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-no-lock TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/stress_watch.sh'

dead 1673452 2017-09-26 02:28:06 2017-09-26 13:40:58 2017-09-26 17:11:02 3:30:04 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 1673453 2017-09-26 02:28:07 2017-09-26 13:41:44 2017-09-26 17:09:47 3:28:03 smithi master rados/singleton/{all/lost-unfound.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
dead 1673454 2017-09-26 02:28:08 2017-09-26 15:17:53 2017-09-26 17:09:55 1:52:02 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
fail 1673455 2017-09-26 02:28:09 2017-09-26 15:18:13 2017-09-26 15:46:13 0:28:00 0:11:39 0:16:21 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/filestore-xfs.yaml tasks/workunits.yaml} 2
Failure Reason:

"2017-09-26 15:34:34.510453 mon.b mon.0 172.21.15.50:6789/0 105 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

dead 1673456 2017-09-26 02:28:10 2017-09-26 15:27:57 2017-09-26 17:09:58 1:42:01 smithi master rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml settings/optimized.yaml workloads/sample_radosbench.yaml} 1
dead 1673457 2017-09-26 02:28:11 2017-09-26 16:39:47 2017-09-26 17:09:47 0:30:00 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 1673458 2017-09-26 02:28:12 2017-09-26 16:39:51 2017-09-26 17:09:51 0:30:00 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 1673459 2017-09-26 02:28:13 2017-09-26 16:41:42 2017-09-26 17:09:41 0:27:59 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
dead 1673460 2017-09-26 02:28:14 2017-09-26 16:48:01 2017-09-26 17:10:00 0:21:59 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
dead 1673461 2017-09-26 02:28:15 2017-09-26 16:48:45 2017-09-26 17:10:45 0:22:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml supported/ubuntu_latest.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 1673462 2017-09-26 02:28:16 2017-09-26 16:50:37 2017-09-26 17:10:36 0:19:59 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
dead 1673463 2017-09-26 02:28:17 2017-09-26 17:00:19 2017-09-26 17:10:19 0:10:00 smithi master rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml} 1
dead 1673464 2017-09-26 02:28:18 2017-09-26 17:03:57 2017-09-26 17:09:57 0:06:00 smithi master rados/singleton/{all/mon-auth-caps.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
dead 1673465 2017-09-26 02:28:19 2017-09-26 17:04:37 2017-09-26 17:10:36 0:05:59 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2