Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 3702998 2019-03-09 23:33:54 2019-03-09 23:35:24 2019-03-09 23:53:23 0:17:59 0:11:04 0:06:55 smithi master rhel 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_latest.yaml} workloads/radosbench_4M_write.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3702999 2019-03-09 23:33:55 2019-03-09 23:35:24 2019-03-10 00:11:24 0:36:00 0:23:14 0:12:46 smithi master centos 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 3703000 2019-03-09 23:33:55 2019-03-09 23:35:24 2019-03-09 23:55:23 0:19:59 0:13:08 0:06:51 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_latest.yaml} tasks/ssh_orchestrator.yaml} 2
pass 3703001 2019-03-09 23:33:56 2019-03-09 23:35:24 2019-03-10 00:11:24 0:36:00 0:26:21 0:09:39 smithi master ubuntu 16.04 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703002 2019-03-09 23:33:57 2019-03-09 23:35:24 2019-03-09 23:55:23 0:19:59 0:13:35 0:06:24 smithi master rhel 7.5 rados/singleton/{all/deduptool.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
fail 3703003 2019-03-09 23:33:58 2019-03-09 23:36:47 2019-03-10 00:02:46 0:25:59 0:14:51 0:11:08 smithi master ubuntu 18.04 rados/multimon/{clusters/6.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/mon_recovery.yaml} 2
Failure Reason:

Command failed on smithi023 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

dead 3703004 2019-03-09 23:33:59 2019-03-09 23:36:53 2019-03-10 01:10:54 1:34:01 smithi master ubuntu 18.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
dead 3703005 2019-03-09 23:34:00 2019-03-09 23:37:05 2019-03-10 01:11:06 1:34:01 1:21:21 0:12:40 smithi master ubuntu 16.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

psutil.NoSuchProcess process no longer exists (pid=17911)

fail 3703006 2019-03-09 23:34:01 2019-03-09 23:37:05 2019-03-10 00:53:06 1:16:01 1:04:55 0:11:06 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} 2
Failure Reason:

"2019-03-09 23:51:18.266034 mon.b (mon.1) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 3703007 2019-03-09 23:34:01 2019-03-09 23:37:10 2019-03-09 23:53:10 0:16:00 0:08:07 0:07:53 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/librados_hello_world.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3703008 2019-03-09 23:34:02 2019-03-09 23:37:14 2019-03-10 01:03:14 1:26:00 1:11:14 0:14:46 smithi master centos 7.5 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/fastclose.yaml msgr/async.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 4
pass 3703009 2019-03-09 23:34:03 2019-03-09 23:37:21 2019-03-10 00:05:20 0:27:59 0:21:59 0:06:00 smithi master rhel 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} tasks/rados_stress_watch.yaml} 2
pass 3703010 2019-03-09 23:34:04 2019-03-09 23:39:23 2019-03-10 00:01:22 0:21:59 0:16:01 0:05:58 smithi master rhel 7.5 rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
fail 3703011 2019-03-09 23:34:05 2019-03-09 23:39:23 2019-03-09 23:55:22 0:15:59 0:05:47 0:10:12 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/sample_fio.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703012 2019-03-09 23:34:05 2019-03-09 23:39:23 2019-03-09 23:57:22 0:17:59 0:09:25 0:08:34 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 3703013 2019-03-09 23:34:06 2019-03-09 23:39:27 2019-03-10 01:09:28 1:30:01 smithi master centos 7.5 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=clay-k=4-m=2.yaml} 2
fail 3703014 2019-03-09 23:34:07 2019-03-09 23:39:27 2019-03-10 00:11:27 0:32:00 0:22:10 0:09:50 smithi master ubuntu 16.04 rados/singleton/{all/divergent_priors2.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
Failure Reason:

Exiting scrub checking -- not all pgs scrubbed.

pass 3703015 2019-03-09 23:34:08 2019-03-09 23:39:28 2019-03-09 23:55:27 0:15:59 0:06:30 0:09:29 smithi master ubuntu 18.04 rados/objectstore/{backends/keyvaluedb.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3703016 2019-03-09 23:34:09 2019-03-09 23:40:55 2019-03-10 00:00:54 0:19:59 0:13:31 0:06:28 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{rhel_latest.yaml} tasks/workunits.yaml} 2
pass 3703017 2019-03-09 23:34:09 2019-03-09 23:40:58 2019-03-10 00:20:58 0:40:00 0:29:59 0:10:01 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 3703018 2019-03-09 23:34:10 2019-03-09 23:41:00 2019-03-09 23:54:59 0:13:59 0:05:37 0:08:22 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/msgr.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
Failure Reason:

Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_msgr'

fail 3703019 2019-03-09 23:34:11 2019-03-09 23:41:07 2019-03-09 23:57:06 0:15:59 0:08:20 0:07:39 smithi master ubuntu 16.04 rados/singleton/{all/dump-stuck.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
Failure Reason:

Command failed on smithi143 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump_stuck inactive 10 --format=json'

fail 3703020 2019-03-09 23:34:12 2019-03-09 23:41:09 2019-03-10 00:05:08 0:23:59 0:11:30 0:12:29 smithi master ubuntu 16.04 rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/sync.yaml workloads/rados_mon_osdmap_prune.yaml} 2
Failure Reason:

Command failed on smithi097 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json'

dead 3703021 2019-03-09 23:34:12 2019-03-09 23:41:14 2019-03-10 01:11:14 1:30:00 1:18:07 0:11:53 smithi master rhel 7.5 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
Failure Reason:

SSH connection to smithi166 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 3703022 2019-03-09 23:34:13 2019-03-09 23:41:14 2019-03-10 00:13:14 0:32:00 0:16:45 0:15:15 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

Command failed on smithi067 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true'

fail 3703023 2019-03-09 23:34:14 2019-03-09 23:41:23 2019-03-09 23:57:22 0:15:59 0:05:48 0:10:11 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/sample_radosbench.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3703024 2019-03-09 23:34:15 2019-03-09 23:43:07 2019-03-10 01:11:07 1:28:00 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_striper.yaml} 2
fail 3703025 2019-03-09 23:34:16 2019-03-09 23:43:07 2019-03-09 23:59:06 0:15:59 0:06:19 0:09:40 smithi master ubuntu 16.04 rados/standalone/{supported-random-distro$/{ubuntu_16.04.yaml} workloads/osd.yaml} 1
Failure Reason:

Command failed (workunit test osd/ec-error-rollforward.sh) on smithi025 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e4f44841fa08929ec727e376a4d8afa1eb7db05 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh'

pass 3703026 2019-03-09 23:34:16 2019-03-09 23:43:13 2019-03-10 00:29:13 0:46:00 0:38:35 0:07:25 smithi master rhel 7.5 rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3703027 2019-03-09 23:34:17 2019-03-09 23:43:19 2019-03-10 01:09:19 1:26:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 3703028 2019-03-09 23:34:18 2019-03-09 23:43:33 2019-03-10 00:01:32 0:17:59 0:09:30 0:08:29 smithi master centos 7.5 rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703029 2019-03-09 23:34:19 2019-03-09 23:44:41 2019-03-10 00:08:40 0:23:59 0:12:40 0:11:19 smithi master centos 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{centos_latest.yaml} tasks/crash.yaml} 2
fail 3703030 2019-03-09 23:34:19 2019-03-09 23:45:19 2019-03-10 00:05:18 0:19:59 0:08:54 0:11:05 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/cosbench_64K_read_write.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703031 2019-03-09 23:34:20 2019-03-09 23:47:20 2019-03-10 00:33:20 0:46:00 0:36:51 0:09:09 smithi master centos 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703032 2019-03-09 23:34:21 2019-03-09 23:47:20 2019-03-10 00:19:20 0:32:00 0:16:58 0:15:02 smithi master centos 7.5 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/hammer.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/few.yaml msgr/random.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rbd_cls.yaml} 4
fail 3703033 2019-03-09 23:34:22 2019-03-09 23:47:20 2019-03-10 00:57:20 1:10:00 1:03:03 0:06:57 smithi master rhel 7.5 rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 3703034 2019-03-09 23:34:22 2019-03-09 23:47:20 2019-03-10 00:23:20 0:36:00 0:26:50 0:09:10 smithi master ubuntu 18.04 rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3703035 2019-03-09 23:34:23 2019-03-09 23:47:23 2019-03-10 00:25:23 0:38:00 0:28:01 0:09:59 smithi master ubuntu 16.04 rados/objectstore/{backends/objectcacher-stress.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703036 2019-03-09 23:34:24 2019-03-09 23:47:23 2019-03-10 00:13:22 0:25:59 0:12:46 0:13:13 smithi master centos 7.5 rados/multimon/{clusters/9.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_clock_no_skews.yaml} 3
dead 3703037 2019-03-09 23:34:25 2019-03-09 23:47:23 2019-03-10 01:09:24 1:22:01 smithi master centos 7.5 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} 2
fail 3703038 2019-03-09 23:34:26 2019-03-09 23:47:58 2019-03-10 00:23:58 0:36:00 0:06:45 0:29:15 smithi master centos 7.5 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

Command failed on smithi202 with status 1: '\n sudo yum -y install ceph-radosgw\n '

pass 3703039 2019-03-09 23:34:27 2019-03-09 23:49:11 2019-03-10 00:27:10 0:37:59 0:24:58 0:13:01 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} 2
dead 3703040 2019-03-09 23:34:27 2019-03-09 23:49:11 2019-03-10 01:11:11 1:22:00 smithi master rhel 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} tasks/rados_workunit_loadgen_big.yaml} 2
fail 3703041 2019-03-09 23:34:28 2019-03-09 23:49:11 2019-03-10 01:05:11 1:16:00 1:04:20 0:11:40 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2019-03-10 00:04:08.076697 mon.b (mon.1) 10 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

dead 3703042 2019-03-09 23:34:29 2019-03-09 23:49:21 2019-03-10 01:09:21 1:20:00 smithi master rhel 7.5 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
fail 3703043 2019-03-09 23:34:30 2019-03-09 23:49:34 2019-03-10 00:07:33 0:17:59 0:08:45 0:09:14 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/cosbench_64K_write.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3703044 2019-03-09 23:34:30 2019-03-09 23:51:04 2019-03-10 01:11:04 1:20:00 smithi master rhel 7.5 rados/singleton/{all/lost-unfound.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3703045 2019-03-09 23:34:31 2019-03-09 23:51:04 2019-03-10 00:19:03 0:27:59 0:15:53 0:12:06 smithi master centos 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
fail 3703046 2019-03-09 23:34:32 2019-03-09 23:51:06 2019-03-10 00:25:05 0:33:59 0:22:13 0:11:46 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} 2
Failure Reason:

Test failure: test_delete_s3 (tasks.mgr.dashboard.test_rgw.RgwUserKeyTest)

pass 3703047 2019-03-09 23:34:33 2019-03-09 23:51:11 2019-03-10 00:07:10 0:15:59 0:06:34 0:09:25 smithi master ubuntu 16.04 rados/singleton/{all/max-pg-per-osd.from-mon.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703048 2019-03-09 23:34:34 2019-03-09 23:51:18 2019-03-10 00:29:17 0:37:59 0:25:58 0:12:01 smithi master ubuntu 18.04 rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml} 2
dead 3703049 2019-03-09 23:34:35 2019-03-09 23:51:18 2019-03-10 01:09:18 1:18:00 smithi master centos 7.5 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
dead 3703050 2019-03-09 23:34:36 2019-03-09 23:53:17 2019-03-10 01:09:17 1:16:00 smithi master rhel 7.5 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 3703051 2019-03-09 23:34:37 2019-03-09 23:53:17 2019-03-10 00:15:16 0:21:59 0:10:55 0:11:04 smithi master centos 7.5 rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703052 2019-03-09 23:34:37 2019-03-09 23:53:17 2019-03-10 00:37:17 0:44:00 0:19:04 0:24:56 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_workunit_loadgen_mix.yaml} 2
fail 3703053 2019-03-09 23:34:38 2019-03-09 23:53:18 2019-03-10 00:11:17 0:17:59 0:05:52 0:12:07 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/fio_4K_rand_read.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

fail 3703054 2019-03-09 23:34:39 2019-03-09 23:53:17 2019-03-10 00:13:16 0:19:59 0:11:54 0:08:05 smithi master rhel 7.5 rados/standalone/{supported-random-distro$/{rhel_latest.yaml} workloads/scrub.yaml} 1
Failure Reason:

Command failed (workunit test scrub/osd-recovery-scrub.sh) on smithi158 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e4f44841fa08929ec727e376a4d8afa1eb7db05 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh'

pass 3703055 2019-03-09 23:34:40 2019-03-09 23:53:17 2019-03-10 00:21:17 0:28:00 0:17:12 0:10:48 smithi master centos 7.5 rados/singleton/{all/max-pg-per-osd.from-primary.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703056 2019-03-09 23:34:41 2019-03-09 23:53:17 2019-03-10 00:21:16 0:27:59 0:18:00 0:09:59 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml} 2
pass 3703057 2019-03-09 23:34:42 2019-03-09 23:53:17 2019-03-10 00:35:17 0:42:00 0:29:03 0:12:57 smithi master ubuntu 16.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/osd-delay.yaml msgr/simple.yaml rados.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 4
pass 3703058 2019-03-09 23:34:42 2019-03-09 23:53:17 2019-03-10 00:15:17 0:22:00 0:10:54 0:11:06 smithi master ubuntu 16.04 rados/singleton/{all/max-pg-per-osd.from-replica.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3703059 2019-03-09 23:34:43 2019-03-09 23:53:21 2019-03-10 01:09:21 1:16:00 smithi master ubuntu 18.04 rados/objectstore/{backends/objectstore.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
pass 3703060 2019-03-09 23:34:44 2019-03-09 23:53:24 2019-03-10 00:29:24 0:36:00 0:13:54 0:22:06 smithi master centos 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_latest.yaml} tasks/failover.yaml} 2
dead 3703061 2019-03-09 23:34:45 2019-03-09 23:53:25 2019-03-10 01:09:25 1:16:00 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
fail 3703062 2019-03-09 23:34:46 2019-03-09 23:55:01 2019-03-10 00:27:00 0:31:59 0:18:43 0:13:16 smithi master centos 7.5 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2019-03-10 00:16:00.519984 mon.a (mon.1) 13 : cluster [WRN] Health check update: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 3703063 2019-03-09 23:34:46 2019-03-09 23:55:01 2019-03-10 01:03:01 1:08:00 0:58:45 0:09:15 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/fio_4K_rand_rw.yaml} 1
Failure Reason:

Command failed on smithi193 with status 1: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703064 2019-03-09 23:34:47 2019-03-09 23:55:01 2019-03-10 00:17:00 0:21:59 0:15:11 0:06:48 smithi master rhel 7.5 rados/singleton/{all/mon-auth-caps.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3703065 2019-03-09 23:34:48 2019-03-09 23:55:18 2019-03-10 00:43:18 0:48:00 0:13:51 0:34:09 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 3703066 2019-03-09 23:34:49 2019-03-09 23:55:18 2019-03-10 00:33:18 0:38:00 0:26:35 0:11:25 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/recovery-unfound-found.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703067 2019-03-09 23:34:50 2019-03-09 23:55:23 2019-03-10 00:37:23 0:42:00 0:29:29 0:12:31 smithi master centos 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_workunit_loadgen_mostlyread.yaml} 2
pass 3703068 2019-03-09 23:34:50 2019-03-09 23:55:24 2019-03-10 00:17:23 0:21:59 0:13:17 0:08:42 smithi master centos 7.5 rados/singleton/{all/mon-config-key-caps.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3703069 2019-03-09 23:34:51 2019-03-09 23:55:24 2019-03-10 01:09:24 1:14:00 smithi master centos 7.5 rados/multimon/{clusters/21.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_clock_with_skews.yaml} 3
dead 3703070 2019-03-09 23:34:52 2019-03-09 23:55:25 2019-03-10 01:09:25 1:14:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} 2
fail 3703071 2019-03-09 23:34:53 2019-03-09 23:55:25 2019-03-10 01:11:25 1:16:00 smithi master rhel 7.5 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
Failure Reason:

psutil.NoSuchProcess process no longer exists (pid=6805)

pass 3703072 2019-03-09 23:34:54 2019-03-09 23:55:28 2019-03-10 00:43:28 0:48:00 0:18:10 0:29:50 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
dead 3703073 2019-03-09 23:34:55 2019-03-09 23:56:48 2019-03-10 01:10:48 1:14:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml} 2
fail 3703074 2019-03-09 23:34:55 2019-03-09 23:57:07 2019-03-10 01:07:07 1:10:00 0:56:31 0:13:29 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/fio_4M_rand_read.yaml} 1
Failure Reason:

Command failed on smithi025 with status 1: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3703075 2019-03-09 23:34:56 2019-03-09 23:57:18 2019-03-10 01:09:18 1:12:00 smithi master ubuntu 16.04 rados/singleton/{all/mon-config-keys.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703076 2019-03-09 23:34:57 2019-03-09 23:57:18 2019-03-10 01:03:18 1:06:00 0:33:11 0:32:49 smithi master ubuntu 18.04 rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/many.yaml workloads/snaps-few-objects.yaml} 2
dead 3703077 2019-03-09 23:34:58 2019-03-09 23:57:23 2019-03-10 01:09:23 1:12:00 smithi master rhel 7.5 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
dead 3703078 2019-03-09 23:34:59 2019-03-09 23:57:23 2019-03-10 01:09:24 1:12:01 smithi master rhel 7.5 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 3703079 2019-03-09 23:34:59 2019-03-09 23:57:24 2019-03-10 00:23:23 0:25:59 0:16:05 0:09:54 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{rhel_latest.yaml} tasks/insights.yaml} 2
pass 3703080 2019-03-09 23:35:00 2019-03-09 23:57:24 2019-03-10 00:21:24 0:24:00 0:11:16 0:12:44 smithi master centos 7.5 rados/singleton/{all/mon-config.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703081 2019-03-09 23:35:01 2019-03-09 23:57:25 2019-03-10 00:17:24 0:19:59 0:08:42 0:11:17 smithi master ubuntu 16.04 rados/objectstore/{backends/alloc-hint.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703082 2019-03-09 23:35:02 2019-03-09 23:59:17 2019-03-10 00:21:17 0:22:00 0:11:20 0:10:40 smithi master centos 7.5 rados/rest/{mgr-restful.yaml supported-random-distro$/{centos_latest.yaml}} 1
pass 3703083 2019-03-09 23:35:03 2019-03-09 23:59:17 2019-03-10 00:29:17 0:30:00 0:16:01 0:13:59 smithi master centos rados/singleton-flat/valgrind-leaks.yaml 1
pass 3703084 2019-03-09 23:35:04 2019-03-09 23:59:17 2019-03-10 00:21:17 0:22:00 0:11:33 0:10:27 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/admin_socket_output.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
fail 3703085 2019-03-09 23:35:04 2019-03-10 00:01:08 2019-03-10 00:21:08 0:20:00 0:09:21 0:10:39 smithi master centos 7.5 rados/standalone/{supported-random-distro$/{centos_latest.yaml} workloads/crush.yaml} 1
Failure Reason:

Command failed (workunit test crush/crush-choose-args.sh) on smithi114 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e4f44841fa08929ec727e376a4d8afa1eb7db05 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/crush/crush-choose-args.sh'

dead 3703086 2019-03-09 23:35:05 2019-03-10 00:01:23 2019-03-10 01:09:23 1:08:00 smithi master centos 7.5 rados/upgrade/luminous-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/luminous.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-nautilus.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} supported-random-distro$/{centos_latest.yaml} thrashosds-health.yaml} 4
pass 3703087 2019-03-09 23:35:06 2019-03-10 00:01:24 2019-03-10 00:31:24 0:30:00 0:14:39 0:15:21 smithi master centos 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/readwrite.yaml} 2
dead 3703088 2019-03-09 23:35:07 2019-03-10 00:01:24 2019-03-10 01:09:24 1:08:00 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
dead 3703089 2019-03-09 23:35:07 2019-03-10 00:01:25 2019-03-10 01:09:25 1:08:00 smithi master centos 7.5 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/fastclose.yaml msgr/async-v1only.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml}
fail 3703090 2019-03-09 23:35:08 2019-03-10 00:01:33 2019-03-10 00:21:33 0:20:00 0:12:08 0:07:52 smithi master rhel 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_latest.yaml} workloads/fio_4M_rand_rw.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703091 2019-03-09 23:35:09 2019-03-10 00:02:59 2019-03-10 00:28:59 0:26:00 0:16:51 0:09:09 smithi master ubuntu 18.04 rados/singleton/{all/osd-backfill.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
fail 3703092 2019-03-09 23:35:10 2019-03-10 00:03:19 2019-03-10 00:45:19 0:42:00 0:13:34 0:28:26 smithi master ubuntu 16.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2019-03-10 00:36:26.260055 mon.a (mon.1) 17 : cluster [WRN] Health check update: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 3703093 2019-03-09 23:35:11 2019-03-10 00:05:22 2019-03-10 00:43:21 0:37:59 0:27:32 0:10:27 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{rhel_latest.yaml} tasks/module_selftest.yaml} 2
dead 3703094 2019-03-09 23:35:12 2019-03-10 00:05:22 2019-03-10 01:09:22 1:04:00 smithi master centos 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
fail 3703095 2019-03-09 23:35:12 2019-03-10 00:05:22 2019-03-10 00:23:21 0:17:59 smithi master centos 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

Command failed on smithi055 with status 1: 'sudo yum install -y kernel'

pass 3703096 2019-03-09 23:35:13 2019-03-10 00:07:23 2019-03-10 00:33:23 0:26:00 0:17:57 0:08:03 smithi master centos 7.5 rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
fail 3703097 2019-03-09 23:35:14 2019-03-10 00:07:35 2019-03-10 00:25:34 0:17:59 0:06:00 0:11:59 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/fio_4M_rand_write.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703098 2019-03-09 23:35:15 2019-03-10 00:08:53 2019-03-10 00:28:52 0:19:59 0:13:06 0:06:53 smithi master rhel 7.5 rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3703099 2019-03-09 23:35:16 2019-03-10 00:11:19 2019-03-10 01:09:19 0:58:00 smithi master centos 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/repair_test.yaml} 2
dead 3703100 2019-03-09 23:35:16 2019-03-10 00:11:19 2019-03-10 01:09:19 0:58:00 smithi master ubuntu 16.04 rados/singleton/{all/osd-recovery.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703101 2019-03-09 23:35:17 2019-03-10 00:11:26 2019-03-10 00:49:25 0:37:59 0:15:33 0:22:26 smithi master centos 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
dead 3703102 2019-03-09 23:35:18 2019-03-10 00:11:26 2019-03-10 01:11:26 1:00:00 0:50:55 0:09:05 smithi master rhel 7.5 rados/singleton/{all/peer.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
Failure Reason:

SSH connection to smithi067 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 3703103 2019-03-09 23:35:19 2019-03-10 00:11:28 2019-03-10 00:31:27 0:19:59 0:11:33 0:08:26 smithi master ubuntu 16.04 rados/objectstore/{backends/ceph_objectstore_tool.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703104 2019-03-09 23:35:19 2019-03-10 00:13:26 2019-03-10 00:59:26 0:46:00 0:18:01 0:27:59 smithi master ubuntu 16.04 rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/one.yaml workloads/pool-create-delete.yaml} 2
pass 3703105 2019-03-09 23:35:20 2019-03-10 00:13:26 2019-03-10 00:57:26 0:44:00 0:12:40 0:31:20 smithi master centos 7.5 rados/multimon/{clusters/3.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_recovery.yaml} 2
dead 3703106 2019-03-09 23:35:21 2019-03-10 00:13:26 2019-03-10 01:09:26 0:56:00 smithi master rhel 7.5 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
dead 3703107 2019-03-09 23:35:22 2019-03-10 00:15:32 2019-03-10 01:09:32 0:54:00 smithi master centos 7.5 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 3703108 2019-03-09 23:35:23 2019-03-10 00:15:32 2019-03-10 01:09:32 0:54:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} 2
dead 3703109 2019-03-09 23:35:24 2019-03-10 00:17:13 2019-03-10 01:11:13 0:54:00 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
pass 3703110 2019-03-09 23:35:24 2019-03-10 00:17:24 2019-03-10 01:03:24 0:46:00 0:31:53 0:14:07 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
pass 3703111 2019-03-09 23:35:25 2019-03-10 00:17:25 2019-03-10 00:39:25 0:22:00 0:09:09 0:12:51 smithi master ubuntu 16.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/orchestrator_cli.yaml} 2
fail 3703112 2019-03-09 23:35:26 2019-03-10 00:19:16 2019-03-10 00:37:15 0:17:59 0:10:56 0:07:03 smithi master rhel 7.5 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_latest.yaml} workloads/radosbench_4K_rand_read.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703113 2019-03-09 23:35:27 2019-03-10 00:19:16 2019-03-10 00:41:15 0:21:59 0:09:59 0:12:00 smithi master ubuntu 18.04 rados/singleton/{all/pg-autoscaler.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
pass 3703114 2019-03-09 23:35:28 2019-03-10 00:19:21 2019-03-10 01:01:21 0:42:00 0:28:13 0:13:47 smithi master centos 7.5 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
dead 3703115 2019-03-09 23:35:29 2019-03-10 00:21:11 2019-03-10 01:11:11 0:50:00 smithi master ubuntu 16.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/hammer.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/few.yaml msgr/async.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}
fail 3703116 2019-03-09 23:35:29 2019-03-10 00:21:11 2019-03-10 00:41:10 0:19:59 0:09:16 0:10:43 smithi master centos 7.5 rados/standalone/{supported-random-distro$/{centos_latest.yaml} workloads/erasure-code.yaml} 1
Failure Reason:

Command failed (workunit test erasure-code/test-erasure-code-plugins.sh) on smithi202 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e4f44841fa08929ec727e376a4d8afa1eb7db05 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh'

dead 3703117 2019-03-09 23:35:30 2019-03-10 00:21:14 2019-03-10 01:11:14 0:50:00 smithi master ubuntu 18.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rgw_snaps.yaml} 2
pass 3703118 2019-03-09 23:35:31 2019-03-10 00:21:17 2019-03-10 00:43:17 0:22:00 0:12:55 0:09:05 smithi master rhel 7.5 rados/singleton-nomsgr/{all/ceph-kvstore-tool.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3703119 2019-03-09 23:35:32 2019-03-10 00:21:18 2019-03-10 01:09:18 0:48:00 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
pass 3703120 2019-03-09 23:35:33 2019-03-10 00:21:18 2019-03-10 00:39:17 0:17:59 0:07:15 0:10:44 smithi master ubuntu 18.04 rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
dead 3703121 2019-03-09 23:35:33 2019-03-10 00:21:18 2019-03-10 01:09:18 0:48:00 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/radosbench_4K_seq_read.yaml} 1
dead 3703122 2019-03-09 23:35:34 2019-03-10 00:21:25 2019-03-10 01:09:25 0:48:00 smithi master rhel 7.5 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 3703123 2019-03-09 23:35:35 2019-03-10 00:21:34 2019-03-10 01:01:34 0:40:00 0:31:16 0:08:44 smithi master ubuntu 16.04 rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
pass 3703124 2019-03-09 23:35:36 2019-03-10 00:23:14 2019-03-10 00:47:14 0:24:00 0:12:58 0:11:02 smithi master ubuntu 16.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/progress.yaml} 2
dead 3703125 2019-03-09 23:35:37 2019-03-10 00:23:21 2019-03-10 01:09:21 0:46:00 smithi master centos 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3703126 2019-03-09 23:35:37 2019-03-10 00:23:22 2019-03-10 01:09:21 0:45:59 smithi master rhel 7.5 rados/singleton/{all/random-eio.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 2
pass 3703127 2019-03-09 23:35:38 2019-03-10 00:23:22 2019-03-10 00:41:21 0:17:59 0:12:03 0:05:56 smithi master rhel 7.5 rados/objectstore/{backends/filejournal.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3703128 2019-03-09 23:35:39 2019-03-10 00:23:24 2019-03-10 00:45:24 0:22:00 0:10:05 0:11:55 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/scrub_test.yaml} 2
pass 3703129 2019-03-09 23:35:40 2019-03-10 00:23:59 2019-03-10 00:41:58 0:17:59 0:11:53 0:06:06 smithi master rhel 7.5 rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
fail 3703130 2019-03-09 23:35:40 2019-03-10 00:24:47 2019-03-10 00:46:46 0:21:59 0:09:15 0:12:44 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4M_rand_read.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3703131 2019-03-09 23:35:41 2019-03-10 00:25:06 2019-03-10 01:11:06 0:46:00 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
dead 3703132 2019-03-09 23:35:42 2019-03-10 00:25:24 2019-03-10 01:09:23 0:43:59 smithi master ubuntu 16.04 rados/singleton/{all/rebuild-mondb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3703133 2019-03-09 23:35:43 2019-03-10 00:25:35 2019-03-10 01:09:35 0:44:00 smithi master rhel 7.5 rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/sync-many.yaml workloads/rados_5925.yaml} 2
dead 3703134 2019-03-09 23:35:44 2019-03-10 00:27:13 2019-03-10 01:11:13 0:44:00 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
dead 3703135 2019-03-09 23:35:44 2019-03-10 00:27:13 2019-03-10 01:11:13 0:44:00 smithi master centos 7.5 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 3703136 2019-03-09 23:35:45 2019-03-10 00:29:04 2019-03-10 01:11:04 0:42:00 smithi master centos 7.5 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/osd-delay.yaml msgr/random.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml} 4
Failure Reason:

psutil.NoSuchProcess process no longer exists (pid=8660)

pass 3703137 2019-03-09 23:35:46 2019-03-10 00:29:04 2019-03-10 00:55:04 0:26:00 0:08:41 0:17:19 smithi master ubuntu 16.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/prometheus.yaml} 2
pass 3703138 2019-03-09 23:35:47 2019-03-10 00:29:14 2019-03-10 00:59:14 0:30:00 0:19:18 0:10:42 smithi master centos 7.5 rados/singleton/{all/recovery-preemption.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}} 1
dead 3703139 2019-03-09 23:35:48 2019-03-10 00:29:18 2019-03-10 01:11:18 0:42:00 smithi master rhel 7.5 rados/multimon/{clusters/6.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} tasks/mon_clock_no_skews.yaml} 2
dead 3703140 2019-03-09 23:35:49 2019-03-10 00:29:19 2019-03-10 01:11:18 0:41:59 0:25:10 0:16:49 smithi master ubuntu 18.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
Failure Reason:

SSH connection to smithi014 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

dead 3703141 2019-03-09 23:35:49 2019-03-10 00:29:25 2019-03-10 01:09:25 0:40:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
dead 3703142 2019-03-09 23:35:50 2019-03-10 00:31:37 2019-03-10 01:09:37 0:38:00 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
dead 3703143 2019-03-09 23:35:51 2019-03-10 00:31:37 2019-03-10 01:09:37 0:38:00 smithi master ubuntu 16.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
dead 3703144 2019-03-09 23:35:52 2019-03-10 00:33:30 2019-03-10 01:09:30 0:36:00 smithi master centos 7.5 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-small-objects-many-deletes.yaml}
dead 3703145 2019-03-09 23:35:53 2019-03-10 00:33:30 2019-03-10 01:09:30 0:36:00 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/radosbench_4M_seq_read.yaml} 1
dead 3703146 2019-03-09 23:35:53 2019-03-10 00:33:30 2019-03-10 01:09:30 0:36:00 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest.yaml} workloads/misc.yaml} 1
dead 3703147 2019-03-09 23:35:54 2019-03-10 00:35:29 2019-03-10 01:09:29 0:34:00 smithi master ubuntu 16.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_api_tests.yaml} 2
pass 3703148 2019-03-09 23:35:55 2019-03-10 00:37:27 2019-03-10 01:01:27 0:24:00 0:15:50 0:08:10 smithi master rhel 7.5 rados/singleton/{all/resolve_stuck_peering.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 2
pass 3703149 2019-03-09 23:35:56 2019-03-10 00:37:27 2019-03-10 00:57:27 0:20:00 0:12:16 0:07:44 smithi master rhel 7.5 rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
pass 3703150 2019-03-09 23:35:57 2019-03-10 00:37:27 2019-03-10 01:09:27 0:32:00 0:21:05 0:10:55 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
pass 3703151 2019-03-09 23:35:57 2019-03-10 00:39:29 2019-03-10 00:55:29 0:16:00 0:08:10 0:07:50 smithi master ubuntu 18.04 rados/singleton/{all/test-crash.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
dead 3703152 2019-03-09 23:35:58 2019-03-10 00:39:29 2019-03-10 01:09:29 0:30:00 smithi master ubuntu 18.04 rados/objectstore/{backends/filestore-idempotent-aio-journal.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
fail 3703153 2019-03-09 23:35:59 2019-03-10 00:41:22 2019-03-10 00:59:22 0:18:00 0:06:04 0:11:56 smithi master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/radosbench_4M_write.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

pass 3703154 2019-03-09 23:36:00 2019-03-10 00:41:22 2019-03-10 01:09:22 0:28:00 0:12:53 0:15:07 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{rhel_latest.yaml} tasks/ssh_orchestrator.yaml} 2
dead 3703155 2019-03-09 23:36:01 2019-03-10 00:41:23 2019-03-10 01:09:22 0:27:59 smithi master rhel 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3703156 2019-03-09 23:36:01 2019-03-10 00:42:00 2019-03-10 01:09:59 0:27:59 smithi master rhel 7.5 rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 1
dead 3703157 2019-03-09 23:36:02 2019-03-10 00:43:29 2019-03-10 01:09:29 0:26:00 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
dead 3703158 2019-03-09 23:36:03 2019-03-10 00:43:29 2019-03-10 01:09:29 0:26:00 smithi master rhel 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} tasks/rados_cls_all.yaml}
dead 3703159 2019-03-09 23:36:04 2019-03-10 00:43:30 2019-03-10 01:09:29 0:25:59 smithi master ubuntu 16.04 rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3703160 2019-03-09 23:36:04 2019-03-10 00:43:29 2019-03-10 01:09:29 0:26:00 smithi master rhel 7.5 rados/singleton/{all/thrash-backfill-full.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}} 2
dead 3703161 2019-03-09 23:36:05 2019-03-10 00:45:31 2019-03-10 01:09:31 0:24:00 smithi master ubuntu 18.04 rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/sync.yaml workloads/rados_api_tests.yaml}
dead 3703162 2019-03-09 23:36:06 2019-03-10 00:45:31 2019-03-10 01:09:31 0:24:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}
dead 3703163 2019-03-09 23:36:07 2019-03-10 00:46:59 2019-03-10 01:10:58 0:23:59 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 3703164 2019-03-09 23:36:07 2019-03-10 00:47:15 2019-03-10 01:09:14 0:21:59 0:05:25 0:16:34 smithi master ubuntu 16.04 rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_16.04.yaml} workloads/sample_fio.yaml} 1
Failure Reason:

Command crashed: 'sudo ceph --cluster ceph osd crush tunables default'

dead 3703165 2019-03-09 23:36:08 2019-03-10 00:49:38 2019-03-10 01:09:37 0:19:59 smithi master ubuntu 16.04 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/fastclose.yaml msgr/simple.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/rbd_cls.yaml}
dead 3703166 2019-03-09 23:36:09 2019-03-10 00:53:18 2019-03-10 01:09:18 0:16:00 smithi master ubuntu 16.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} 2
dead 3703167 2019-03-09 23:36:10 2019-03-10 00:55:16 2019-03-10 01:11:16 0:16:00 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache.yaml} 2
dead 3703168 2019-03-09 23:36:11 2019-03-10 00:55:30 2019-03-10 01:09:29 0:13:59 smithi master ubuntu 18.04 rados/singleton/{all/thrash-eio.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
dead 3703169 2019-03-09 23:36:11 2019-03-10 00:57:33 2019-03-10 01:09:32 0:11:59 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/workunits.yaml} 2
dead 3703170 2019-03-09 23:36:12 2019-03-10 00:57:33 2019-03-10 01:09:32 0:11:59 smithi master ubuntu 16.04 rados/standalone/{supported-random-distro$/{ubuntu_16.04.yaml} workloads/mon-seesaw.yaml} 1
dead 3703171 2019-03-09 23:36:13 2019-03-10 00:57:33 2019-03-10 01:09:32 0:11:59 smithi master ubuntu 16.04 rados/objectstore/{backends/filestore-idempotent.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 1
dead 3703172 2019-03-09 23:36:14 2019-03-10 00:59:26 2019-03-10 01:09:25 0:09:59 smithi master ubuntu 16.04 rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}} 2
dead 3703173 2019-03-09 23:36:14 2019-03-10 00:59:26 2019-03-10 01:09:26 0:10:00 smithi master ubuntu 18.04 rados/multimon/{clusters/9.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/mon_clock_with_skews.yaml} 3
dead 3703174 2019-03-09 23:36:15 2019-03-10 00:59:27 2019-03-10 01:09:26 0:09:59 smithi master ubuntu 16.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml}
dead 3703175 2019-03-09 23:36:16 2019-03-10 01:01:34 2019-03-10 01:09:34 0:08:00 smithi master ubuntu 16.04 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
dead 3703176 2019-03-09 23:36:17 2019-03-10 01:01:34 2019-03-10 01:09:34 0:08:00 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml}
dead 3703177 2019-03-09 23:36:17 2019-03-10 01:01:35 2019-03-10 01:09:34 0:07:59 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/sample_radosbench.yaml} 1
dead 3703178 2019-03-09 23:36:18 2019-03-10 01:03:13 2019-03-10 01:11:12 0:07:59 smithi master centos 7.5 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_python.yaml}
dead 3703179 2019-03-09 23:36:19 2019-03-10 01:03:15 2019-03-10 01:11:14 0:07:59 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml}
fail 3703180 2019-03-09 23:36:20 2019-03-10 01:03:20 2019-03-10 01:11:19 0:07:59 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/health-warnings.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
Failure Reason:

psutil.NoSuchProcess process no longer exists (pid=7254)

dead 3703181 2019-03-09 23:36:20 2019-03-10 01:03:25 2019-03-10 01:09:24 0:05:59 smithi master rhel 7.5 rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}
dead 3703182 2019-03-09 23:36:21 2019-03-10 01:04:43 2019-03-10 01:10:42 0:05:59 smithi master rhel 7.5 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{rhel_latest.yaml} tasks/crash.yaml}
dead 3703183 2019-03-09 23:36:22 2019-03-10 01:05:12 2019-03-10 01:11:11 0:05:59 smithi master centos 7.5 rados/perf/{ceph.yaml objectstore/filestore-xfs.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{centos_latest.yaml} workloads/cosbench_64K_read_write.yaml}
dead 3703184 2019-03-09 23:36:23 2019-03-10 01:07:20 2019-03-10 01:09:19 0:01:59 smithi master rhel 7.5 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}