Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 3005948 2018-09-11 18:22:49 2018-09-11 19:58:21 2018-09-11 20:18:20 0:19:59 0:09:21 0:10:38 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/bluestore.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/fio_4M_rand_write.yaml} 1
fail 3005949 2018-09-11 18:22:49 2018-09-11 19:58:24 2018-09-11 20:56:24 0:58:00 0:43:16 0:14:44 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3005950 2018-09-11 18:22:50 2018-09-11 19:59:02 2018-09-11 20:53:02 0:54:00 0:44:54 0:09:06 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 3005951 2018-09-11 18:22:50 2018-09-11 20:00:23 2018-09-11 20:20:23 0:20:00 0:12:08 0:07:52 smithi master centos 7.4 rados/singleton/{all/osd-recovery.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3005952 2018-09-11 18:22:51 2018-09-11 20:00:54 2018-09-11 22:12:56 2:12:02 smithi master ubuntu 16.04 rados/objectstore/{backends/filestore-idempotent-aio-journal.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3005953 2018-09-11 18:22:52 2018-09-11 20:01:02 2018-09-11 20:17:01 0:15:59 0:06:33 0:09:26 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3005954 2018-09-11 18:22:52 2018-09-11 20:02:20 2018-09-11 20:26:20 0:24:00 0:14:27 0:09:33 smithi master centos 7.4 rados/standalone/{supported-random-distro$/{centos_latest.yaml} workloads/misc.yaml} 1
fail 3005955 2018-09-11 18:22:53 2018-09-11 20:04:09 2018-09-11 22:04:10 2:00:01 1:45:54 0:14:07 smithi master ubuntu 16.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2018-09-11 20:22:27.294969 mon.a (mon.0) 19 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005956 2018-09-11 18:22:54 2018-09-11 20:04:09 2018-09-11 22:06:10 2:02:01 1:48:15 0:13:46 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
Failure Reason:

"2018-09-11 20:23:59.067349 mon.a (mon.0) 20 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005957 2018-09-11 18:22:54 2018-09-11 20:04:54 2018-09-11 21:00:55 0:56:01 0:43:59 0:12:02 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3005958 2018-09-11 18:22:55 2018-09-11 20:06:21 2018-09-11 20:32:21 0:26:00 0:15:00 0:11:00 smithi master rhel 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} tasks/readwrite.yaml} 2
pass 3005959 2018-09-11 18:22:56 2018-09-11 20:06:21 2018-09-11 20:34:21 0:28:00 0:12:36 0:15:24 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 3005960 2018-09-11 18:22:56 2018-09-11 20:06:55 2018-09-11 20:26:54 0:19:59 0:08:55 0:11:04 smithi master centos 7.4 rados/singleton/{all/peer.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3005961 2018-09-11 18:22:57 2018-09-11 20:06:55 2018-09-11 21:08:55 1:02:00 0:49:40 0:12:20 smithi master rhel 7.5 rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/many.yaml workloads/rados_mon_osdmap_prune.yaml} 2
fail 3005962 2018-09-11 18:22:58 2018-09-11 20:06:56 2018-09-11 22:08:58 2:02:02 1:48:01 0:14:01 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} 2
Failure Reason:

"2018-09-11 20:28:01.866703 mon.a (mon.0) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005963 2018-09-11 18:22:58 2018-09-11 20:09:11 2018-09-11 22:11:13 2:02:02 1:48:14 0:13:48 smithi master ubuntu 16.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml two-plus-three.yaml} d-balancer/crush-compat.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/few.yaml msgr/async.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 3
Failure Reason:

"2018-09-11 20:29:19.677843 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005964 2018-09-11 18:22:59 2018-09-11 20:09:11 2018-09-11 21:27:12 1:18:01 1:01:25 0:16:36 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/module_selftest.yaml} 2
Failure Reason:

"2018-09-11 20:29:25.496529 mon.a (mon.0) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3005965 2018-09-11 18:22:59 2018-09-11 20:12:22 2018-09-11 20:34:22 0:22:00 0:11:42 0:10:18 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4K_rand_read.yaml} 1
fail 3005966 2018-09-11 18:23:00 2018-09-11 20:12:22 2018-09-11 21:08:23 0:56:01 0:42:38 0:13:23 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3005967 2018-09-11 18:23:01 2018-09-11 20:12:54 2018-09-11 20:52:54 0:40:00 0:27:40 0:12:20 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 3005968 2018-09-11 18:23:01 2018-09-11 20:13:00 2018-09-11 20:31:00 0:18:00 0:10:02 0:07:58 smithi master rhel 7.5 rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
fail 3005969 2018-09-11 18:23:02 2018-09-11 20:14:08 2018-09-11 22:10:10 1:56:02 1:45:47 0:10:15 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2018-09-11 20:28:27.453876 mon.a (mon.1) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 3005970 2018-09-11 18:23:03 2018-09-11 20:15:52 2018-09-11 22:11:54 1:56:02 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 3005971 2018-09-11 18:23:03 2018-09-11 20:16:16 2018-09-11 20:40:15 0:23:59 0:17:45 0:06:14 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 3005972 2018-09-11 18:23:04 2018-09-11 20:16:21 2018-09-11 20:30:21 0:14:00 0:06:05 0:07:55 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3005973 2018-09-11 18:23:05 2018-09-11 20:16:24 2018-09-11 20:40:24 0:24:00 0:15:44 0:08:16 smithi master centos 7.4 rados/singleton/{all/radostool.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
fail 3005974 2018-09-11 18:23:05 2018-09-11 20:17:10 2018-09-11 20:45:10 0:28:00 0:21:14 0:06:46 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2018-09-11 20:32:42.075277 mon.a (mon.0) 32 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005975 2018-09-11 18:23:06 2018-09-11 20:17:10 2018-09-11 20:45:10 0:28:00 0:16:43 0:11:17 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/repair_test.yaml} 2
Failure Reason:

"2018-09-11 20:31:23.367030 mon.a (mon.1) 39 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3005976 2018-09-11 18:23:07 2018-09-11 20:18:09 2018-09-11 20:38:08 0:19:59 0:11:40 0:08:19 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4K_seq_read.yaml} 1
fail 3005977 2018-09-11 18:23:07 2018-09-11 20:18:22 2018-09-11 20:52:22 0:34:00 0:23:18 0:10:42 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-09-11 20:33:29.243985 mon.b (mon.1) 16 : cluster [WRN] Health check update: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

fail 3005978 2018-09-11 18:23:08 2018-09-11 20:18:22 2018-09-11 21:28:23 1:10:01 0:58:38 0:11:23 smithi master centos 7.4 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
Failure Reason:

Command failed on smithi064 with status 1: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3005979 2018-09-11 18:23:08 2018-09-11 20:18:22 2018-09-11 22:12:24 1:54:02 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 3005980 2018-09-11 18:23:09 2018-09-11 20:20:22 2018-09-11 21:28:23 1:08:01 0:57:11 0:10:50 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

Command failed on smithi026 with status 1: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3005981 2018-09-11 18:23:10 2018-09-11 20:20:22 2018-09-11 20:52:22 0:32:00 0:22:28 0:09:32 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
dead 3005982 2018-09-11 18:23:10 2018-09-11 20:20:24 2018-09-11 22:12:26 1:52:02 smithi master centos 7.4 rados/multimon/{clusters/3.yaml mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_clock_with_skews.yaml} 2
fail 3005983 2018-09-11 18:23:11 2018-09-11 20:21:02 2018-09-11 21:39:03 1:18:01 1:07:28 0:10:33 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
Failure Reason:

"2018-09-11 20:35:15.908144 mon.a (mon.1) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 3005984 2018-09-11 18:23:12 2018-09-11 20:22:16 2018-09-11 22:12:17 1:50:01 smithi master rhel 7.5 rados/objectstore/{backends/filestore-idempotent.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3005985 2018-09-11 18:23:12 2018-09-11 20:22:16 2018-09-11 22:12:17 1:50:01 1:43:48 0:06:13 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

SSH connection to smithi040 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 3005986 2018-09-11 18:23:13 2018-09-11 20:22:20 2018-09-11 20:56:20 0:34:00 0:22:51 0:11:09 smithi master centos 7.4 rados/singleton/{all/random-eio.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 2
fail 3005987 2018-09-11 18:23:14 2018-09-11 20:24:17 2018-09-11 21:38:18 1:14:01 1:03:02 0:10:59 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-09-11 20:41:18.596560 mon.a (mon.1) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3005988 2018-09-11 18:23:14 2018-09-11 20:24:18 2018-09-11 20:58:18 0:34:00 0:23:58 0:10:02 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
dead 3005989 2018-09-11 18:23:15 2018-09-11 20:25:00 2018-09-11 22:13:02 1:48:02 1:37:46 0:10:16 smithi master centos 7.4 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

SSH connection to smithi141 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 3005990 2018-09-11 18:23:16 2018-09-11 20:26:21 2018-09-11 21:38:21 1:12:00 1:06:22 0:05:38 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-09-11 20:40:40.582031 mon.a (mon.0) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3005991 2018-09-11 18:23:16 2018-09-11 20:26:55 2018-09-11 20:44:55 0:18:00 0:11:52 0:06:08 smithi master rhel 7.5 rados/singleton/{all/rebuild-mondb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3005992 2018-09-11 18:23:17 2018-09-11 20:28:23 2018-09-11 22:12:24 1:44:01 smithi master ubuntu 18.04 rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/one.yaml workloads/rados_mon_workunits.yaml} 2
pass 3005993 2018-09-11 18:23:18 2018-09-11 20:28:24 2018-09-11 20:48:23 0:19:59 0:10:32 0:09:27 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/radosbench_4M_rand_read.yaml} 1
fail 3005994 2018-09-11 18:23:18 2018-09-11 20:29:00 2018-09-11 21:21:01 0:52:01 0:42:41 0:09:20 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

dead 3005995 2018-09-11 18:23:19 2018-09-11 20:29:03 2018-09-11 22:13:04 1:44:01 smithi master rhel 7.5 rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
pass 3005996 2018-09-11 18:23:20 2018-09-11 20:29:12 2018-09-11 20:53:12 0:24:00 0:13:35 0:10:25 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
fail 3005997 2018-09-11 18:23:20 2018-09-11 20:29:13 2018-09-11 20:59:12 0:29:59 0:18:09 0:11:50 smithi master ubuntu 16.04 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/hammer.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml two-plus-three.yaml} d-balancer/off.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/osd-delay.yaml msgr/random.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rbd_cls.yaml} 3
Failure Reason:

"2018-09-11 20:45:41.053471 mon.a (mon.0) 29 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3005998 2018-09-11 18:23:21 2018-09-11 20:29:13 2018-09-11 20:59:12 0:29:59 0:19:19 0:10:40 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_latest.yaml} tasks/prometheus.yaml} 2
Failure Reason:

"2018-09-11 20:47:00.285788 mon.a (mon.1) 41 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3005999 2018-09-11 18:23:22 2018-09-11 20:29:13 2018-09-11 21:03:12 0:33:59 0:26:50 0:07:09 smithi master rhel 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3006000 2018-09-11 18:23:22 2018-09-11 20:29:13 2018-09-11 20:47:12 0:17:59 0:10:38 0:07:21 smithi master rhel 7.5 rados/singleton-nomsgr/{all/health-warnings.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3006001 2018-09-11 18:23:23 2018-09-11 20:29:13 2018-09-11 21:15:13 0:46:00 0:33:16 0:12:44 smithi master centos 7.4 rados/standalone/{supported-random-distro$/{centos_latest.yaml} workloads/mon.yaml} 1
fail 3006002 2018-09-11 18:23:24 2018-09-11 20:29:13 2018-09-11 21:29:13 1:00:00 0:41:19 0:18:41 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rgw_snaps.yaml} 2
Failure Reason:

"2018-09-11 20:51:33.897428 mon.b (mon.0) 95 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive (PG_AVAILABILITY)" in cluster log

dead 3006003 2018-09-11 18:23:24 2018-09-11 20:29:14 2018-09-11 22:11:15 1:42:01 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} 2
pass 3006004 2018-09-11 18:23:25 2018-09-11 20:29:14 2018-09-11 20:55:13 0:25:59 0:18:03 0:07:56 smithi master rhel 7.5 rados/singleton/{all/recovery-preemption.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3006005 2018-09-11 18:23:26 2018-09-11 20:29:14 2018-09-11 22:11:15 1:42:01 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 3006006 2018-09-11 18:23:26 2018-09-11 20:29:14 2018-09-11 21:17:14 0:48:00 0:19:08 0:28:52 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} 2
fail 3006007 2018-09-11 18:23:27 2018-09-11 20:29:14 2018-09-11 21:23:14 0:54:00 0:19:25 0:34:35 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2018-09-11 21:10:11.375316 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006008 2018-09-11 18:23:28 2018-09-11 20:29:15 2018-09-11 20:51:13 0:21:58 0:12:28 0:09:30 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/bluestore.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4M_seq_read.yaml} 1
pass 3006009 2018-09-11 18:23:28 2018-09-11 20:29:15 2018-09-11 20:55:14 0:25:59 0:08:09 0:17:50 smithi master ubuntu 18.04 rados/singleton/{all/reg11184.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
fail 3006010 2018-09-11 18:23:29 2018-09-11 20:29:15 2018-09-11 21:39:15 1:10:00 0:42:18 0:27:42 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3006011 2018-09-11 18:23:30 2018-09-11 20:29:15 2018-09-11 21:09:14 0:39:59 0:12:17 0:27:42 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 3006012 2018-09-11 18:23:30 2018-09-11 20:29:15 2018-09-11 20:47:14 0:17:59 0:05:13 0:12:46 smithi master ubuntu 18.04 rados/objectstore/{backends/fusestore.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
dead 3006013 2018-09-11 18:23:31 2018-09-11 20:29:15 2018-09-11 22:11:16 1:42:01 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 3006014 2018-09-11 18:23:32 2018-09-11 20:29:15 2018-09-11 21:35:15 1:06:00 0:42:10 0:23:50 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

dead 3006015 2018-09-11 18:23:32 2018-09-11 20:29:15 2018-09-11 22:11:16 1:42:01 smithi master ubuntu 18.04 rados/singleton/{all/resolve_stuck_peering.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
pass 3006016 2018-09-11 18:23:33 2018-09-11 20:29:16 2018-09-11 20:55:15 0:25:59 0:13:24 0:12:35 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
pass 3006017 2018-09-11 18:23:34 2018-09-11 20:29:16 2018-09-11 20:57:16 0:28:00 0:12:20 0:15:40 smithi master centos 7.4 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/scrub_test.yaml} 2
pass 3006018 2018-09-11 18:23:35 2018-09-11 20:29:16 2018-09-11 20:49:16 0:20:00 0:05:54 0:14:06 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/large-omap-object-warnings.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
fail 3006019 2018-09-11 18:23:35 2018-09-11 20:29:16 2018-09-11 22:05:17 1:36:01 1:06:00 0:30:01 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

"2018-09-11 21:06:35.973900 mon.a (mon.0) 19 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006020 2018-09-11 18:23:36 2018-09-11 20:29:16 2018-09-11 20:57:16 0:28:00 0:10:50 0:17:10 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4M_write.yaml} 1
fail 3006021 2018-09-11 18:23:37 2018-09-11 20:30:26 2018-09-11 21:48:27 1:18:01 0:46:37 0:31:24 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

dead 3006022 2018-09-11 18:23:37 2018-09-11 20:30:27 2018-09-11 22:12:28 1:42:01 smithi master ubuntu 16.04 rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/sync-many.yaml workloads/snaps-few-objects.yaml} 2
pass 3006023 2018-09-11 18:23:38 2018-09-11 20:30:27 2018-09-11 20:56:26 0:25:59 0:08:47 0:17:12 smithi master centos 7.4 rados/singleton/{all/test-crash.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
fail 3006024 2018-09-11 18:23:39 2018-09-11 20:31:01 2018-09-11 21:41:01 1:10:00 0:57:56 0:12:04 smithi master centos 7.4 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
Failure Reason:

Command failed on smithi200 with status 1: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3006025 2018-09-11 18:23:39 2018-09-11 20:32:26 2018-09-11 22:12:27 1:40:01 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 3006026 2018-09-11 18:23:40 2018-09-11 20:32:26 2018-09-11 22:12:27 1:40:01 smithi master rhel 7.5 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 3006027 2018-09-11 18:23:41 2018-09-11 20:32:26 2018-09-11 21:02:25 0:29:59 0:19:41 0:10:18 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
fail 3006028 2018-09-11 18:23:41 2018-09-11 20:34:18 2018-09-11 21:18:19 0:44:01 0:20:12 0:23:49 smithi master rhel 7.5 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2018-09-11 21:06:24.566140 mon.a (mon.1) 58 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3006029 2018-09-11 18:23:42 2018-09-11 20:34:22 2018-09-11 21:08:21 0:33:59 0:16:21 0:17:38 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache.yaml} 2
Failure Reason:

"2018-09-11 20:55:49.744002 mon.a (mon.1) 38 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006030 2018-09-11 18:23:43 2018-09-11 20:34:23 2018-09-11 21:12:23 0:38:00 0:25:30 0:12:30 smithi master centos 7.4 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml two-plus-three.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/fastclose.yaml msgr/simple.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 3
fail 3006031 2018-09-11 18:23:44 2018-09-11 20:34:56 2018-09-11 21:38:57 1:04:01 0:39:42 0:24:19 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2018-09-11 21:09:20.240520 mon.b (mon.1) 291 : cluster [WRN] Health check update: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

dead 3006032 2018-09-11 18:23:44 2018-09-11 20:36:17 2018-09-11 22:12:18 1:36:01 smithi master centos 7.4 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{centos_latest.yaml} tasks/workunits.yaml} 2
pass 3006033 2018-09-11 18:23:45 2018-09-11 20:36:55 2018-09-11 20:56:55 0:20:00 0:11:51 0:08:09 smithi master ubuntu 16.04 rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3006034 2018-09-11 18:23:46 2018-09-11 20:38:14 2018-09-11 21:14:14 0:36:00 0:24:29 0:11:31 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
dead 3006035 2018-09-11 18:23:47 2018-09-11 20:38:19 2018-09-11 22:12:20 1:34:01 smithi master ubuntu 18.04 rados/multimon/{clusters/6.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/mon_recovery.yaml} 2
fail 3006036 2018-09-11 18:23:48 2018-09-11 20:38:20 2018-09-11 21:48:21 1:10:01 0:54:12 0:15:49 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
Failure Reason:

saw valgrind issues

fail 3006037 2018-09-11 18:23:48 2018-09-11 20:40:25 2018-09-11 21:10:25 0:30:00 0:20:27 0:09:33 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2018-09-11 20:56:43.447119 mon.a (mon.1) 20 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006038 2018-09-11 18:23:49 2018-09-11 20:40:25 2018-09-11 21:04:25 0:24:00 0:11:44 0:12:16 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/sample_fio.yaml} 1
fail 3006039 2018-09-11 18:23:50 2018-09-11 20:43:03 2018-09-11 21:45:03 1:02:00 0:55:12 0:06:48 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3006040 2018-09-11 18:23:50 2018-09-11 20:44:19 2018-09-11 21:26:19 0:42:00 0:19:10 0:22:50 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_api_tests.yaml} 2
dead 3006041 2018-09-11 18:23:51 2018-09-11 20:44:56 2018-09-11 22:12:58 1:28:02 smithi master centos 7.4 rados/singleton/{all/thrash-eio.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 2
pass 3006042 2018-09-11 18:23:52 2018-09-11 20:45:11 2018-09-11 21:21:11 0:36:00 0:14:04 0:21:56 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} 2
pass 3006043 2018-09-11 18:23:53 2018-09-11 20:45:11 2018-09-11 21:03:11 0:18:00 0:07:38 0:10:22 smithi master centos 7.4 rados/objectstore/{backends/keyvaluedb.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3006044 2018-09-11 18:23:53 2018-09-11 20:46:22 2018-09-11 21:24:22 0:38:00 0:24:08 0:13:52 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/msgr.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3006045 2018-09-11 18:23:54 2018-09-11 20:47:11 2018-09-11 21:53:12 1:06:01 0:55:46 0:10:15 smithi master ubuntu 16.04 rados/standalone/{supported-random-distro$/{ubuntu_16.04.yaml} workloads/osd.yaml} 1
dead 3006046 2018-09-11 18:23:55 2018-09-11 20:47:13 2018-09-11 22:11:14 1:24:01 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
dead 3006047 2018-09-11 18:23:55 2018-09-11 20:47:15 2018-09-11 22:11:16 1:24:01 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} 2
pass 3006048 2018-09-11 18:23:56 2018-09-11 20:48:24 2018-09-11 21:20:24 0:32:00 0:20:47 0:11:13 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 3006049 2018-09-11 18:23:57 2018-09-11 20:49:17 2018-09-11 22:11:17 1:22:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} 2
dead 3006050 2018-09-11 18:23:58 2018-09-11 20:51:03 2018-09-11 22:13:04 1:22:01 smithi master ubuntu 16.04 rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 2
dead 3006051 2018-09-11 18:23:58 2018-09-11 20:51:17 2018-09-11 22:13:17 1:22:00 0:44:24 0:37:36 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

SSH connection to smithi012 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 3006052 2018-09-11 18:23:59 2018-09-11 20:52:28 2018-09-11 21:10:27 0:17:59 0:08:42 0:09:17 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/sample_radosbench.yaml} 1
fail 3006053 2018-09-11 18:24:00 2018-09-11 20:52:28 2018-09-11 21:22:27 0:29:59 0:16:39 0:13:20 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2018-09-11 21:09:33.660220 mon.a (mon.0) 18 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006054 2018-09-11 18:24:00 2018-09-11 20:52:28 2018-09-11 21:36:28 0:44:00 0:22:50 0:21:10 smithi master rhel 7.5 rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/sync.yaml workloads/pool-create-delete.yaml} 2
pass 3006055 2018-09-11 18:24:01 2018-09-11 20:52:55 2018-09-11 21:18:55 0:26:00 0:10:47 0:15:13 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 3006056 2018-09-11 18:24:02 2018-09-11 20:53:03 2018-09-11 21:27:03 0:34:00 0:21:48 0:12:12 smithi master centos 7.4 rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 2
fail 3006057 2018-09-11 18:24:02 2018-09-11 20:53:13 2018-09-11 21:25:13 0:32:00 0:20:30 0:11:30 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2018-09-11 21:12:20.435575 mon.a (mon.1) 23 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 3006058 2018-09-11 18:24:03 2018-09-11 20:54:26 2018-09-11 22:12:27 1:18:01 smithi master centos 7.4 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_cls_all.yaml} 2
dead 3006059 2018-09-11 18:24:04 2018-09-11 20:54:26 2018-09-11 22:12:27 1:18:01 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
fail 3006060 2018-09-11 18:24:04 2018-09-11 20:55:14 2018-09-11 21:23:14 0:28:00 0:16:26 0:11:34 smithi master ubuntu 16.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
Failure Reason:

"2018-09-11 21:09:22.462255 mon.a (mon.0) 28 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006061 2018-09-11 18:24:05 2018-09-11 20:55:15 2018-09-11 21:35:15 0:40:00 0:33:10 0:06:50 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 3006062 2018-09-11 18:24:06 2018-09-11 20:55:16 2018-09-11 22:11:17 1:16:01 smithi master centos 7.4 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml two-plus-three.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/few.yaml msgr/async.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} 3
pass 3006063 2018-09-11 18:24:06 2018-09-11 20:56:23 2018-09-11 21:12:22 0:15:59 0:09:53 0:06:06 smithi master rhel 7.5 rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
fail 3006064 2018-09-11 18:24:07 2018-09-11 20:56:23 2018-09-11 21:26:23 0:30:00 0:16:39 0:13:21 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2018-09-11 21:13:39.998044 mon.a (mon.1) 20 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 3006065 2018-09-11 18:24:08 2018-09-11 20:56:25 2018-09-11 22:12:26 1:16:01 smithi master centos 7.4 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore.yaml supported-random-distro$/{centos_latest.yaml} tasks/crash.yaml} 2
pass 3006066 2018-09-11 18:24:09 2018-09-11 20:56:27 2018-09-11 21:26:27 0:30:00 0:20:02 0:09:58 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/bluestore.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/cosbench_64K_read_write.yaml} 1
fail 3006067 2018-09-11 18:24:09 2018-09-11 20:56:56 2018-09-11 21:40:56 0:44:00 0:31:16 0:12:44 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi063 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=wip-sage3-testing-2018-09-10-1637 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

dead 3006068 2018-09-11 18:24:10 2018-09-11 20:57:17 2018-09-11 22:11:18 1:14:01 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 2
dead 3006069 2018-09-11 18:24:11 2018-09-11 20:57:17 2018-09-11 22:13:18 1:16:01 0:43:00 0:33:01 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3006070 2018-09-11 18:24:11 2018-09-11 20:58:26 2018-09-11 21:42:26 0:44:00 0:10:29 0:33:31 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 3006071 2018-09-11 18:24:12 2018-09-11 20:58:26 2018-09-11 22:12:27 1:14:01 smithi master centos 7.4 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
dead 3006072 2018-09-11 18:24:13 2018-09-11 20:58:56 2018-09-11 22:12:57 1:14:01 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 3006073 2018-09-11 18:24:13 2018-09-11 20:59:13 2018-09-11 22:11:14 1:12:01 smithi master ubuntu 16.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 3006074 2018-09-11 18:24:14 2018-09-11 20:59:13 2018-09-11 21:17:13 0:18:00 0:07:47 0:10:13 smithi master centos 7.4 rados/singleton/{all/admin-socket.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3006075 2018-09-11 18:24:15 2018-09-11 21:00:56 2018-09-11 22:12:57 1:12:01 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 3006076 2018-09-11 18:24:15 2018-09-11 21:02:35 2018-09-11 21:28:34 0:25:59 0:15:44 0:10:15 smithi master ubuntu 16.04 rados/objectstore/{backends/objectcacher-stress.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3006077 2018-09-11 18:24:16 2018-09-11 21:03:12 2018-09-11 22:13:12 1:10:00 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 3006078 2018-09-11 18:24:17 2018-09-11 21:03:13 2018-09-11 21:57:13 0:54:00 0:39:35 0:14:25 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
dead 3006079 2018-09-11 18:24:18 2018-09-11 21:04:26 2018-09-11 22:12:27 1:08:01 0:55:16 0:12:45 smithi master centos 7.4 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_python.yaml} 2
Failure Reason:

SSH connection to smithi037 was lost: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0'

dead 3006080 2018-09-11 18:24:18 2018-09-11 21:08:23 2018-09-11 22:12:23 1:04:00 0:39:22 0:24:38 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
Failure Reason:

"2018-09-11 21:40:39.413106 mon.a (mon.0) 22 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006081 2018-09-11 18:24:19 2018-09-11 21:08:24 2018-09-11 21:30:23 0:21:59 0:14:08 0:07:51 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/cosbench_64K_write.yaml} 1
pass 3006082 2018-09-11 18:24:20 2018-09-11 21:08:56 2018-09-11 21:30:56 0:22:00 0:10:26 0:11:34 smithi master ubuntu 18.04 rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
fail 3006083 2018-09-11 18:24:20 2018-09-11 21:09:16 2018-09-11 22:07:16 0:58:00 0:42:48 0:15:12 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3006084 2018-09-11 18:24:21 2018-09-11 21:14:23 2018-09-11 21:42:23 0:28:00 0:10:34 0:17:26 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
dead 3006085 2018-09-11 18:24:22 2018-09-11 21:15:14 2018-09-11 22:11:14 0:56:00 smithi master rhel 7.5 rados/monthrash/{ceph.yaml clusters/3-mons.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/force-sync-many.yaml workloads/rados_5925.yaml} 2
dead 3006086 2018-09-11 18:24:22 2018-09-11 21:17:14 2018-09-11 22:11:14 0:54:00 smithi master ubuntu 18.04 rados/multimon/{clusters/9.yaml mon_kv_backend/rocksdb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/mon_clock_no_skews.yaml}
pass 3006087 2018-09-11 18:24:23 2018-09-11 21:17:15 2018-09-11 21:45:14 0:27:59 0:08:50 0:19:09 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
dead 3006088 2018-09-11 18:24:24 2018-09-11 21:18:24 2018-09-11 22:12:24 0:54:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} 2
pass 3006089 2018-09-11 18:24:24 2018-09-11 21:18:56 2018-09-11 21:36:55 0:17:59 0:08:48 0:09:11 smithi master centos 7.4 rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3006090 2018-09-11 18:24:25 2018-09-11 21:18:58 2018-09-11 22:10:58 0:52:00 0:40:50 0:11:10 smithi master ubuntu 16.04 rados/standalone/{supported-random-distro$/{ubuntu_16.04.yaml} workloads/scrub.yaml} 1
pass 3006091 2018-09-11 18:24:26 2018-09-11 21:20:33 2018-09-11 21:38:32 0:17:59 0:07:42 0:10:17 smithi master ubuntu 16.04 rados/singleton/{all/divergent_priors2.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3006092 2018-09-11 18:24:26 2018-09-11 21:21:02 2018-09-11 22:13:02 0:52:00 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
pass 3006093 2018-09-11 18:24:27 2018-09-11 21:21:12 2018-09-11 22:05:12 0:44:00 0:22:28 0:21:32 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
fail 3006094 2018-09-11 18:24:28 2018-09-11 21:22:29 2018-09-11 22:06:28 0:43:59 0:35:25 0:08:34 smithi master centos 7.4 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/fio_4K_rand_read.yaml} 1
Failure Reason:

failed to become clean before timeout expired

dead 3006095 2018-09-11 18:24:29 2018-09-11 21:23:15 2018-09-11 22:11:15 0:48:00 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 3006096 2018-09-11 18:24:29 2018-09-11 21:23:15 2018-09-11 21:41:14 0:17:59 0:11:48 0:06:11 smithi master rhel 7.5 rados/singleton/{all/dump-stuck.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3006097 2018-09-11 18:24:30 2018-09-11 21:24:31 2018-09-11 22:12:32 0:48:01 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
dead 3006098 2018-09-11 18:24:31 2018-09-11 21:25:14 2018-09-11 22:11:14 0:46:00 smithi master centos 7.4 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/hammer.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml two-plus-three.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/osd-delay.yaml msgr/random.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 3
dead 3006099 2018-09-11 18:24:31 2018-09-11 21:26:28 2018-09-11 22:12:28 0:46:00 smithi master centos 7.4 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-many-deletes.yaml} 2
fail 3006100 2018-09-11 18:24:32 2018-09-11 21:26:29 2018-09-11 21:54:28 0:27:59 0:15:36 0:12:23 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} 2
Failure Reason:

"2018-09-11 21:42:40.714245 mon.a (mon.0) 52 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3006101 2018-09-11 18:24:33 2018-09-11 21:26:30 2018-09-11 22:02:29 0:35:59 0:16:29 0:19:30 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 3006102 2018-09-11 18:24:33 2018-09-11 21:27:04 2018-09-11 22:03:04 0:36:00 0:13:53 0:22:07 smithi master centos 7.4 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_stress_watch.yaml} 2
dead 3006103 2018-09-11 18:24:34 2018-09-11 21:27:13 2018-09-11 22:13:13 0:46:00 smithi master centos 7.4 rados/thrash-erasure-code-overwrites/{bluestore.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} 2
dead 3006104 2018-09-11 18:24:35 2018-09-11 21:28:24 2018-09-11 22:12:24 0:44:00 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 3006105 2018-09-11 18:24:35 2018-09-11 21:28:24 2018-09-11 22:12:24 0:44:00 smithi master ubuntu 16.04 rados/objectstore/{backends/objectstore.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3006106 2018-09-11 18:24:36 2018-09-11 21:28:35 2018-09-11 22:08:35 0:40:00 0:30:34 0:09:26 smithi master rhel 7.5 rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3006107 2018-09-11 18:24:37 2018-09-11 21:29:14 2018-09-11 22:11:14 0:42:00 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 3006108 2018-09-11 18:24:37 2018-09-11 21:30:33 2018-09-11 22:12:33 0:42:00 0:26:05 0:15:55 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

SSH connection to smithi131 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" autoremove'

pass 3006109 2018-09-11 18:24:38 2018-09-11 21:30:57 2018-09-11 21:52:56 0:21:59 0:15:13 0:06:46 smithi master rhel 7.5 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_latest.yaml} workloads/fio_4K_rand_rw.yaml} 1
dead 3006110 2018-09-11 18:24:39 2018-09-11 21:32:25 2018-09-11 22:12:25 0:40:00 smithi master centos 7.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 3006112 2018-09-11 18:24:39 2018-09-11 21:32:25 2018-09-11 22:12:25 0:40:00 smithi master centos 7.4 rados/singleton-nomsgr/{all/recovery-unfound-found.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3006113 2018-09-11 18:24:40 2018-09-11 21:35:16 2018-09-11 22:11:16 0:36:00 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
pass 3006114 2018-09-11 18:24:41 2018-09-11 21:35:16 2018-09-11 21:53:16 0:18:00 0:06:55 0:11:05 smithi master centos 7.4 rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3006115 2018-09-11 18:24:41 2018-09-11 21:36:37 2018-09-11 22:12:37 0:36:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml}
dead 3006116 2018-09-11 18:24:42 2018-09-11 21:36:56 2018-09-11 22:12:56 0:36:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml}
dead 3006117 2018-09-11 18:24:43 2018-09-11 21:38:27 2018-09-11 22:12:27 0:34:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 3006118 2018-09-11 18:24:43 2018-09-11 21:38:28 2018-09-11 22:12:27 0:33:59 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
dead 3006119 2018-09-11 18:24:44 2018-09-11 21:38:34 2018-09-11 22:12:33 0:33:59 0:21:49 0:12:10 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2018-09-11 21:54:08.509029 mon.a (mon.0) 17 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 3006120 2018-09-11 18:24:45 2018-09-11 21:38:58 2018-09-11 22:12:58 0:34:00 smithi master ubuntu 18.04 rados/monthrash/{ceph.yaml clusters/9-mons.yaml mon_kv_backend/leveldb.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/many.yaml workloads/rados_api_tests.yaml} 2
dead 3006121 2018-09-11 18:24:46 2018-09-11 21:39:04 2018-09-11 22:13:04 0:34:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
dead 3006122 2018-09-11 18:24:46 2018-09-11 21:39:16 2018-09-11 22:11:16 0:32:00 smithi master rhel 7.5 rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3006123 2018-09-11 18:24:47 2018-09-11 21:40:57 2018-09-11 22:12:57 0:32:00 0:16:03 0:15:57 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_striper.yaml} 2
Failure Reason:

SSH connection to smithi005 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 3006124 2018-09-11 18:24:48 2018-09-11 21:41:02 2018-09-11 22:03:02 0:22:00 0:13:35 0:08:25 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore.yaml rados.yaml rocksdb.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 3006125 2018-09-11 18:24:48 2018-09-11 21:41:16 2018-09-11 22:09:15 0:27:59 0:08:54 0:19:05 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/bluestore.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/fio_4M_rand_read.yaml} 1
fail 3006163 2018-09-11 18:25:11 2018-09-11 22:10:19 2018-09-11 22:18:19 0:08:00 smithi master rhel 7.5 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_latest.yaml} workloads/fio_4M_rand_write.yaml}
Failure Reason:

machine smithi097.front.sepia.ceph.com is locked by scheduled_jdillaman@dillaman-rh, not scheduled_sage@teuthology