Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 4400358 2019-10-11 00:28:38 2019-10-11 00:28:54 2019-10-11 00:46:53 0:17:59 mira master rhel 7.7 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

fail 4400359 2019-10-11 00:28:39 2019-10-11 00:28:55 2019-10-11 01:44:55 1:16:00 1:04:55 0:11:05 mira master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2019-10-11T00:45:14.626668+0000 mon.a (mon.1) 35 : cluster [WRN] Health check update: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

dead 4400360 2019-10-11 00:28:40 2019-10-11 00:28:55 2019-10-11 00:46:54 0:17:59 mira master rhel 7.7 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} tasks/rados_workunit_loadgen_big.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

fail 4400361 2019-10-11 00:28:41 2019-10-11 00:28:54 2019-10-11 00:54:53 0:25:59 0:15:24 0:10:35 mira master ubuntu 18.04 rados/perf/{ceph.yaml objectstore/bluestore-basic-min-osd-mem-target.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/cosbench_64K_read_write.yaml} 1
Failure Reason:

Command failed on mira110 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest'

dead 4400362 2019-10-11 00:28:42 2019-10-11 00:28:54 2019-10-11 00:46:53 0:17:59 mira master rhel 7.7 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_7.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400363 2019-10-11 00:28:42 2019-10-11 00:28:55 2019-10-11 00:46:53 0:17:58 mira master rhel 7.7 rados/singleton-nomsgr/{all/version-number-sanity.yaml rados.yaml supported-random-distro$/{rhel_7.yaml}}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400364 2019-10-11 00:28:43 2019-10-11 00:28:55 2019-10-11 12:31:16 12:02:21 11:51:07 0:11:14 mira master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/fastclose.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
Failure Reason:

SSH connection to mira032 was lost: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 4400365 2019-10-11 00:28:44 2019-10-11 00:28:54 2019-10-11 01:04:53 0:35:59 0:10:08 0:25:51 mira master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/nautilus.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/few.yaml rados.yaml thrashers/none.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} 4
Failure Reason:

Command failed on mira083 with status 1: 'sudo yum -y install ceph-daemon'

dead 4400366 2019-10-11 00:28:45 2019-10-11 00:28:55 2019-10-11 12:31:16 12:02:21 mira master ubuntu 18.04 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_workunit_loadgen_mix.yaml} 2
dead 4400367 2019-10-11 00:28:46 2019-10-11 00:28:57 2019-10-11 12:31:18 12:02:21 mira master centos 7.6 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-async-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_7.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
fail 4400368 2019-10-11 00:28:47 2019-10-11 00:47:08 2019-10-11 01:57:08 1:10:00 0:59:48 0:10:12 mira master ubuntu 18.04 rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}} 1
Failure Reason:

"2019-10-11T01:01:30.375582+0000 mon.a (mon.0) 79 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

dead 4400369 2019-10-11 00:28:48 2019-10-11 00:47:08 2019-10-11 01:05:07 0:17:59 mira master rhel 7.7 rados/perf/{ceph.yaml objectstore/bluestore-bitmap.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_7.yaml} workloads/cosbench_64K_write.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

fail 4400370 2019-10-11 00:28:48 2019-10-11 00:47:08 2019-10-11 02:03:08 1:16:00 1:04:30 0:11:30 mira master ubuntu 18.04 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

Command failed on mira038 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

dead 4400371 2019-10-11 00:28:49 2019-10-11 00:47:08 2019-10-11 01:05:07 0:17:59 mira master rhel 7.7 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/osd-delay.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

fail 4400372 2019-10-11 00:28:50 2019-10-11 00:55:08 2019-10-11 01:41:08 0:46:00 0:33:23 0:12:37 mira master centos 7.6 rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_7.yaml}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on mira110 with status 6: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0dca79308d83afd02b99aa1aa8839d156215298c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

dead 4400373 2019-10-11 00:28:51 2019-10-11 01:05:10 2019-10-11 01:23:09 0:17:59 mira master rhel 7.7 rados/upgrade/mimic-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/mimic.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-nautilus.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} bluestore-bitmap.yaml supported-random-distro$/{rhel_7.yaml} thrashosds-health.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400374 2019-10-11 00:28:52 2019-10-11 01:05:10 2019-10-11 01:23:09 0:17:59 mira master rhel 7.7 rados/multimon/{clusters/3.yaml msgr-failures/few.yaml msgr/async-v2only.yaml no_pools.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} tasks/mon_clock_no_skews.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

fail 4400375 2019-10-11 00:28:53 2019-10-11 01:05:10 2019-10-11 01:37:09 0:31:59 0:21:27 0:10:32 mira master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/dashboard.yaml} 2
Failure Reason:

Test failure: test_perf_counters_mds_get (tasks.mgr.dashboard.test_perf_counters.PerfCountersControllerTest)

dead 4400376 2019-10-11 00:28:54 2019-10-11 01:23:24 2019-10-11 01:53:23 0:29:59 mira master rhel 7.7 rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{rhel_7.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400377 2019-10-11 00:28:55 2019-10-11 01:23:24 2019-10-11 01:53:23 0:29:59 mira master rhel 7.7 rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{rhel_7.yaml} thrashers/minsize_recovery.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400378 2019-10-11 00:28:56 2019-10-11 01:37:24 2019-10-11 12:47:35 11:10:11 mira master rhel 7.7 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/fastclose.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/none.yaml thrashosds-health.yaml workloads/cache.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400379 2019-10-11 00:28:57 2019-10-11 01:41:23 2019-10-11 12:59:34 11:18:11 mira master centos 7.6 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_7.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}
dead 4400380 2019-10-11 00:28:58 2019-10-11 01:45:10 2019-10-11 02:03:09 0:17:59 mira master rhel 7.7 rados/singleton/{all/deduptool.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_7.yaml}}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400381 2019-10-11 00:28:59 2019-10-11 01:53:37 2019-10-11 02:13:37 0:20:00 mira master rhel 7.7 rados/perf/{ceph.yaml objectstore/bluestore-comp.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_7.yaml} workloads/fio_4K_rand_read.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400382 2019-10-11 00:29:00 2019-10-11 01:53:37 2019-10-11 02:19:37 0:26:00 mira master rhel 7.7 rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/sync.yaml workloads/rados_5925.yaml}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400383 2019-10-11 00:29:01 2019-10-11 01:57:23 2019-10-11 12:59:34 11:02:11 mira master rhel 7.7 rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} tasks/rados_workunit_loadgen_mostlyread.yaml}
dead 4400384 2019-10-11 00:29:02 2019-10-11 02:03:24 2019-10-11 12:59:34 10:56:10 mira master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/hammer.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml}
dead 4400385 2019-10-11 00:29:03 2019-10-11 02:03:24 2019-10-11 12:59:34 10:56:10 mira master rhel 7.7 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-partial-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/dedup_tier.yaml}
dead 4400386 2019-10-11 00:29:03 2019-10-11 02:13:51 2019-10-11 12:48:01 10:34:10 mira master rhel 7.7 rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml supported-random-distro$/{rhel_7.yaml}}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400387 2019-10-11 00:29:04 2019-10-11 02:19:51 2019-10-11 12:48:02 10:28:11 mira master rhel 7.7 rados/singleton/{all/divergent_priors.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_7.yaml}}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

dead 4400388 2019-10-11 00:29:05 2019-10-11 12:31:32 2019-10-11 12:59:31 0:27:59 mira master rhel 7.7 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/osd-delay.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_7.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml}
dead 4400389 2019-10-11 00:29:06 2019-10-11 12:31:32 2019-10-11 12:59:31 0:27:59 mira master centos 7.6 rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-async-recovery.yaml} supported-random-distro$/{centos_7.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
dead 4400390 2019-10-11 00:29:07 2019-10-11 12:31:32 2019-10-11 12:59:31 0:27:59 mira master rhel 7.7 rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-async-partial-recovery.yaml} supported-random-distro$/{rhel_7.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml}
dead 4400391 2019-10-11 00:29:08 2019-10-11 12:47:36 2019-10-11 12:59:35 0:11:59 mira master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/fastclose.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml}
dead 4400392 2019-10-11 00:29:09 2019-10-11 12:48:03 2019-10-11 13:00:02 0:11:59 mira master centos 7.6 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_7.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml}
dead 4400393 2019-10-11 00:29:09 2019-10-11 12:48:03 2019-10-11 13:00:02 0:11:59 mira master rhel 7.7 rados/perf/{ceph.yaml objectstore/bluestore-stupid.yaml openstack.yaml settings/optimized.yaml supported-random-distro$/{rhel_7.yaml} workloads/fio_4M_rand_read.yaml}