Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 5216183 2020-07-12 02:30:41 2020-07-12 03:48:17 2020-07-12 04:18:17 0:30:00 0:19:36 0:10:24 smithi master rhel 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/force-sync-many workloads/pool-create-delete} 2
pass 5216185 2020-07-12 02:30:42 2020-07-12 03:48:18 2020-07-12 04:38:18 0:50:00 0:31:47 0:18:13 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/jewel-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/cache-snaps} 3
pass 5216187 2020-07-12 02:30:43 2020-07-12 03:48:17 2020-07-12 04:12:17 0:24:00 0:11:35 0:12:25 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2
pass 5216189 2020-07-12 02:30:44 2020-07-12 03:50:03 2020-07-12 04:30:03 0:40:00 0:34:48 0:05:12 smithi master rhel 8.1 rados/singleton/{all/lost-unfound msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest}} 1
pass 5216191 2020-07-12 02:30:45 2020-07-12 03:50:12 2020-07-12 05:16:14 1:26:02 1:18:20 0:07:42 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/radosbench} 2
pass 5216193 2020-07-12 02:30:45 2020-07-12 03:52:08 2020-07-12 04:40:08 0:48:00 0:38:16 0:09:44 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_api_tests} 2
pass 5216195 2020-07-12 02:30:46 2020-07-12 03:52:22 2020-07-12 04:36:22 0:44:00 0:36:13 0:07:47 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} tasks/rados_workunit_loadgen_mostlyread} 2
pass 5216197 2020-07-12 02:30:47 2020-07-12 03:53:49 2020-07-12 04:09:48 0:15:59 0:07:32 0:08:27 smithi master centos 8.1 rados/multimon/{clusters/3 msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews} 2
pass 5216199 2020-07-12 02:30:48 2020-07-12 03:54:02 2020-07-12 04:12:01 0:17:59 0:10:12 0:07:47 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/crash} 2
pass 5216201 2020-07-12 02:30:49 2020-07-12 03:54:05 2020-07-12 04:40:05 0:46:00 0:38:24 0:07:36 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216203 2020-07-12 02:30:50 2020-07-12 03:55:58 2020-07-12 05:09:59 1:14:01 1:02:38 0:11:23 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} tasks/dashboard} 2
pass 5216205 2020-07-12 02:30:50 2020-07-12 03:56:07 2020-07-12 04:16:07 0:20:00 0:12:30 0:07:30 smithi master ubuntu 18.04 rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} 1
pass 5216207 2020-07-12 02:30:51 2020-07-12 03:56:18 2020-07-12 04:16:18 0:20:00 0:13:21 0:06:39 smithi master centos 8.1 rados/rest/{mgr-restful supported-random-distro$/{centos_latest}} 1
pass 5216209 2020-07-12 02:30:52 2020-07-12 03:57:49 2020-07-12 04:17:49 0:20:00 0:14:55 0:05:05 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/admin_socket_output rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216211 2020-07-12 02:30:53 2020-07-12 03:58:02 2020-07-12 04:22:02 0:24:00 0:14:57 0:09:03 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_latest} workloads/crush} 1
pass 5216213 2020-07-12 02:30:54 2020-07-12 04:02:20 2020-07-12 10:04:30 6:02:10 5:40:23 0:21:47 smithi master ubuntu 18.04 rados/upgrade/mimic-x-singleton/{0-cluster/{openstack start} 1-install/mimic 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} 4
pass 5216215 2020-07-12 02:30:55 2020-07-12 04:04:08 2020-07-12 04:30:08 0:26:00 0:19:10 0:06:50 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} 1
pass 5216217 2020-07-12 02:30:56 2020-07-12 04:04:15 2020-07-12 04:22:15 0:18:00 0:11:38 0:06:22 smithi master ubuntu 18.04 rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} 2
pass 5216219 2020-07-12 02:30:56 2020-07-12 04:04:19 2020-07-12 04:18:19 0:14:00 0:07:29 0:06:31 smithi master ubuntu 18.04 rados/singleton/{all/max-pg-per-osd.from-mon msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216221 2020-07-12 02:30:57 2020-07-12 04:04:25 2020-07-12 04:30:25 0:26:00 0:17:44 0:08:16 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/redirect} 2
pass 5216223 2020-07-12 02:30:58 2020-07-12 04:05:52 2020-07-12 05:31:53 1:26:01 0:16:09 1:09:52 smithi master rhel 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5216225 2020-07-12 02:30:59 2020-07-12 04:06:05 2020-07-12 05:02:06 0:56:01 0:47:53 0:08:08 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_api_tests validater/valgrind} 2
pass 5216227 2020-07-12 02:31:00 2020-07-12 04:08:13 2020-07-12 04:56:13 0:48:00 0:11:49 0:36:11 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216229 2020-07-12 02:31:01 2020-07-12 04:10:05 2020-07-12 04:34:05 0:24:00 0:14:05 0:09:55 smithi master centos 7.6 rados/cephadm/smoke/{distro/centos_7 fixed-2 start} 2
pass 5216231 2020-07-12 02:31:02 2020-07-12 04:10:08 2020-07-12 04:42:08 0:32:00 0:24:52 0:07:08 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216234 2020-07-12 02:31:02 2020-07-12 04:12:06 2020-07-12 04:30:05 0:17:59 0:10:29 0:07:30 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} 1
pass 5216236 2020-07-12 02:31:03 2020-07-12 04:14:37 2020-07-12 04:46:37 0:32:00 0:24:30 0:07:30 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} 2
pass 5216238 2020-07-12 02:31:04 2020-07-12 04:16:16 2020-07-12 04:36:15 0:19:59 0:13:16 0:06:43 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} 2
pass 5216240 2020-07-12 02:31:05 2020-07-12 04:16:16 2020-07-12 04:40:16 0:24:00 0:13:12 0:10:48 smithi master centos 7.6 rados/cephadm/smoke-roleless/{distro/centos_7 start} 2
pass 5216242 2020-07-12 02:31:06 2020-07-12 04:16:16 2020-07-12 04:34:15 0:17:59 0:12:07 0:05:52 smithi master centos 8.1 rados/singleton/{all/max-pg-per-osd.from-primary msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} 1
pass 5216244 2020-07-12 02:31:07 2020-07-12 04:16:16 2020-07-12 04:40:16 0:24:00 0:16:09 0:07:51 smithi master rhel 8.1 rados/cephadm/upgrade/{1-start 2-start-upgrade 3-wait distro$/{rhel_latest} fixed-2} 2
pass 5216246 2020-07-12 02:31:08 2020-07-12 04:17:47 2020-07-12 04:47:48 0:30:01 0:17:06 0:12:55 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/redirect_set_object} 2
pass 5216248 2020-07-12 02:31:09 2020-07-12 04:17:50 2020-07-12 04:37:49 0:19:59 0:13:33 0:06:26 smithi master centos 8.1 rados/singleton-nomsgr/{all/balancer rados supported-random-distro$/{centos_latest}} 1
fail 5216250 2020-07-12 02:31:09 2020-07-12 04:18:17 2020-07-12 04:56:17 0:38:00 0:22:26 0:15:34 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/jewel backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/radosbench} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

pass 5216252 2020-07-12 02:31:10 2020-07-12 04:18:20 2020-07-12 04:38:19 0:19:59 0:12:44 0:07:15 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_adoption} 1
pass 5216254 2020-07-12 02:31:11 2020-07-12 04:20:18 2020-07-12 04:40:17 0:19:59 0:11:36 0:08:23 smithi master centos 8.1 rados/singleton/{all/max-pg-per-osd.from-replica msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest}} 1
pass 5216256 2020-07-12 02:31:12 2020-07-12 04:22:20 2020-07-12 04:44:20 0:22:00 0:08:39 0:13:21 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_5925} 2
pass 5216257 2020-07-12 02:31:13 2020-07-12 04:24:19 2020-07-12 04:46:19 0:22:00 0:12:06 0:09:54 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/readwrite} 2
pass 5216259 2020-07-12 02:31:14 2020-07-12 04:24:19 2020-07-12 04:44:19 0:20:00 0:11:58 0:08:02 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} 1
pass 5216261 2020-07-12 02:31:14 2020-07-12 04:26:21 2020-07-12 05:08:21 0:42:00 0:27:53 0:14:07 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_python} 2
pass 5216263 2020-07-12 02:31:15 2020-07-12 04:30:01 2020-07-12 04:48:00 0:17:59 0:10:51 0:07:08 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/set-chunk-promote-flush} 2
pass 5216265 2020-07-12 02:31:16 2020-07-12 04:30:06 2020-07-12 04:56:06 0:26:00 0:12:15 0:13:45 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/failover} 2
pass 5216267 2020-07-12 02:31:17 2020-07-12 04:30:26 2020-07-12 04:54:26 0:24:00 0:07:12 0:16:48 smithi master ubuntu 18.04 rados/multimon/{clusters/6 msgr-failures/few msgr/async no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} 2
pass 5216269 2020-07-12 02:31:18 2020-07-12 04:32:26 2020-07-12 04:52:25 0:19:59 0:11:34 0:08:25 smithi master centos 8.0 rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} 2
pass 5216271 2020-07-12 02:31:19 2020-07-12 04:34:22 2020-07-12 04:58:21 0:23:59 0:17:26 0:06:33 smithi master rhel 8.1 rados/singleton/{all/mon-auth-caps msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest}} 1
pass 5216273 2020-07-12 02:31:20 2020-07-12 04:34:22 2020-07-12 04:56:21 0:21:59 0:14:39 0:07:20 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} 2
pass 5216275 2020-07-12 02:31:20 2020-07-12 04:36:07 2020-07-12 04:58:06 0:21:59 0:10:48 0:11:11 smithi master centos 8.0 rados/cephadm/smoke-roleless/{distro/centos_8.0 start} 2
pass 5216277 2020-07-12 02:31:21 2020-07-12 04:36:07 2020-07-12 05:32:07 0:56:00 0:30:49 0:25:11 smithi master rhel 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
fail 5216279 2020-07-12 02:31:22 2020-07-12 04:36:15 2020-07-12 05:24:16 0:48:01 0:06:04 0:41:57 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
Failure Reason:

Command failed on smithi187 with status 1: 'sudo yum -y install ceph-mgr-diskprediction-local'

pass 5216281 2020-07-12 02:31:23 2020-07-12 04:36:19 2020-07-12 05:50:20 1:14:01 1:05:36 0:08:25 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_latest} tasks/dashboard} 2
pass 5216283 2020-07-12 02:31:24 2020-07-12 04:36:23 2020-07-12 04:54:23 0:18:00 0:12:20 0:05:40 smithi master centos 8.1 rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}} 1
pass 5216285 2020-07-12 02:31:25 2020-07-12 04:38:19 2020-07-12 05:14:19 0:36:00 0:26:06 0:09:54 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5216287 2020-07-12 02:31:25 2020-07-12 04:38:19 2020-07-12 05:04:19 0:26:00 0:17:39 0:08:21 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_cls_all validater/lockdep} 2
pass 5216289 2020-07-12 02:31:26 2020-07-12 04:38:21 2020-07-12 04:54:20 0:15:59 0:08:39 0:07:20 smithi master centos 8.1 rados/singleton-nomsgr/{all/cache-fs-trunc rados supported-random-distro$/{centos_latest}} 1
pass 5216291 2020-07-12 02:31:27 2020-07-12 04:38:24 2020-07-12 05:08:24 0:30:00 0:22:32 0:07:28 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} 1
pass 5216293 2020-07-12 02:31:28 2020-07-12 04:40:23 2020-07-12 05:16:23 0:36:00 0:27:58 0:08:02 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216295 2020-07-12 02:31:29 2020-07-12 04:40:23 2020-07-12 05:00:23 0:20:00 0:13:58 0:06:02 smithi master centos 8.1 rados/singleton/{all/mon-config-key-caps msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} 1
pass 5216297 2020-07-12 02:31:30 2020-07-12 04:42:27 2020-07-12 05:14:27 0:32:00 0:22:00 0:10:00 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} 2
pass 5216299 2020-07-12 02:31:31 2020-07-12 04:42:27 2020-07-12 05:22:27 0:40:00 0:19:22 0:20:38 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} 2
pass 5216301 2020-07-12 02:31:31 2020-07-12 04:42:37 2020-07-12 05:00:35 0:17:58 0:10:28 0:07:30 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} 1
pass 5216303 2020-07-12 02:31:32 2020-07-12 04:44:39 2020-07-12 05:28:39 0:44:00 0:35:32 0:08:28 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async start tasks/rados_api_tests} 2
pass 5216305 2020-07-12 02:31:33 2020-07-12 04:46:40 2020-07-12 05:32:40 0:46:00 0:37:35 0:08:25 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} 1
fail 5216307 2020-07-12 02:31:34 2020-07-12 04:48:08 2020-07-12 05:22:07 0:33:59 0:23:10 0:10:49 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

pass 5216309 2020-07-12 02:31:35 2020-07-12 04:48:07 2020-07-12 05:18:07 0:30:00 0:20:41 0:09:19 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/small-objects-localized} 2
pass 5216311 2020-07-12 02:31:36 2020-07-12 04:50:19 2020-07-12 05:14:18 0:23:59 0:11:29 0:12:30 smithi master centos 8.1 rados/cephadm/smoke/{distro/centos_latest fixed-2 start} 2
pass 5216313 2020-07-12 02:31:37 2020-07-12 04:50:25 2020-07-12 05:14:25 0:24:00 0:17:39 0:06:21 smithi master centos 8.1 rados/singleton/{all/mon-config-keys msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} 1
pass 5216315 2020-07-12 02:31:37 2020-07-12 04:52:08 2020-07-12 05:18:08 0:26:00 0:19:33 0:06:27 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} tasks/repair_test} 2
pass 5216317 2020-07-12 02:31:38 2020-07-12 04:54:19 2020-07-12 05:12:18 0:17:59 0:10:19 0:07:40 smithi master centos 8.1 rados/cephadm/smoke-roleless/{distro/centos_latest start} 2
pass 5216319 2020-07-12 02:31:39 2020-07-12 04:54:21 2020-07-12 05:30:21 0:36:00 0:29:54 0:06:06 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_api_tests} 2
fail 5216321 2020-07-12 02:31:40 2020-07-12 04:54:23 2020-07-12 05:22:23 0:28:00 0:20:19 0:07:41 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/small-objects} 2
Failure Reason:

"2020-07-12T05:13:58.307087+0000 mgr.x (mgr.4110) 286 : cluster [ERR] Unhandled exception from module 'balancer' while running on mgr.x: ('2.12',)" in cluster log

pass 5216323 2020-07-12 02:31:41 2020-07-12 04:54:29 2020-07-12 05:12:28 0:17:59 0:12:01 0:05:58 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/ceph-kvstore-tool rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216325 2020-07-12 02:31:42 2020-07-12 04:56:23 2020-07-12 05:12:23 0:16:00 0:08:46 0:07:14 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} 1
pass 5216327 2020-07-12 02:31:42 2020-07-12 04:56:24 2020-07-12 05:20:23 0:23:59 0:16:30 0:07:29 smithi master rhel 8.1 rados/singleton/{all/mon-config msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest}} 1
pass 5216329 2020-07-12 02:31:43 2020-07-12 04:56:24 2020-07-12 05:14:23 0:17:59 0:10:35 0:07:24 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} 1
pass 5216331 2020-07-12 02:31:44 2020-07-12 04:56:24 2020-07-12 05:18:23 0:21:59 0:16:31 0:05:28 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_latest} tasks/insights} 2
pass 5216333 2020-07-12 02:31:45 2020-07-12 04:58:23 2020-07-12 06:08:24 1:10:01 0:13:15 0:56:46 smithi master ubuntu 18.04 rados/multimon/{clusters/9 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 3
pass 5216335 2020-07-12 02:31:46 2020-07-12 04:58:23 2020-07-12 05:18:23 0:20:00 0:13:38 0:06:22 smithi master rhel 7.7 rados/cephadm/smoke/{distro/rhel_7 fixed-2 start} 2
pass 5216337 2020-07-12 02:31:47 2020-07-12 05:00:23 2020-07-12 05:38:23 0:38:00 0:30:26 0:07:34 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} 2
pass 5216339 2020-07-12 02:31:48 2020-07-12 05:00:24 2020-07-12 07:10:27 2:10:03 0:12:07 1:57:56 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216341 2020-07-12 02:31:48 2020-07-12 05:14:36 2020-07-12 05:34:35 0:19:59 0:12:42 0:07:17 smithi master rhel 7.7 rados/cephadm/smoke-roleless/{distro/rhel_7 start} 2
pass 5216343 2020-07-12 02:31:49 2020-07-12 05:14:36 2020-07-12 05:40:35 0:25:59 0:20:34 0:05:25 smithi master ubuntu 18.04 rados/singleton/{all/osd-backfill msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216345 2020-07-12 02:31:50 2020-07-12 05:14:36 2020-07-12 06:18:37 1:04:01 0:12:13 0:51:48 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5216347 2020-07-12 02:31:51 2020-07-12 05:16:32 2020-07-12 06:40:33 1:24:01 1:17:55 0:06:06 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_cls_all validater/valgrind} 2
pass 5216349 2020-07-12 02:31:52 2020-07-12 05:18:24 2020-07-12 05:58:24 0:40:00 0:30:06 0:09:54 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} 2
pass 5216351 2020-07-12 02:31:53 2020-07-12 05:18:25 2020-07-12 05:54:24 0:35:59 0:27:11 0:08:48 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async-v1only start tasks/rados_python} 2
pass 5216353 2020-07-12 02:31:54 2020-07-12 05:20:40 2020-07-12 06:04:40 0:44:00 0:37:39 0:06:21 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216355 2020-07-12 02:31:54 2020-07-12 05:22:24 2020-07-12 05:58:24 0:36:00 0:27:41 0:08:19 smithi master centos 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216357 2020-07-12 02:31:55 2020-07-12 05:22:25 2020-07-12 06:38:26 1:16:01 1:01:56 0:14:05 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} tasks/dashboard} 2
pass 5216359 2020-07-12 02:31:56 2020-07-12 05:22:25 2020-07-12 05:36:25 0:14:00 0:07:31 0:06:29 smithi master centos 8.1 rados/objectstore/{backends/filejournal supported-random-distro$/{centos_latest}} 1
fail 5216361 2020-07-12 02:31:57 2020-07-12 05:22:29 2020-07-12 06:04:29 0:42:00 0:23:23 0:18:37 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

pass 5216363 2020-07-12 02:31:58 2020-07-12 05:24:33 2020-07-12 05:54:33 0:30:00 0:19:33 0:10:27 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects} 2
pass 5216365 2020-07-12 02:31:59 2020-07-12 05:24:33 2020-07-12 05:44:33 0:20:00 0:10:56 0:09:04 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/ceph-post-file rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216367 2020-07-12 02:32:00 2020-07-12 05:28:39 2020-07-12 05:52:38 0:23:59 0:16:16 0:07:43 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
pass 5216369 2020-07-12 02:32:01 2020-07-12 05:28:39 2020-07-12 05:52:38 0:23:59 0:17:45 0:06:14 smithi master ubuntu 18.04 rados/singleton/{all/osd-recovery-incomplete msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216371 2020-07-12 02:32:01 2020-07-12 05:30:07 2020-07-12 05:58:07 0:28:00 0:21:06 0:06:54 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} 1
pass 5216373 2020-07-12 02:32:02 2020-07-12 05:30:25 2020-07-12 06:04:25 0:34:00 0:15:01 0:18:59 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} tasks/scrub_test} 2
pass 5216375 2020-07-12 02:32:03 2020-07-12 05:32:11 2020-07-12 06:12:11 0:40:00 0:31:41 0:08:19 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} 2
pass 5216377 2020-07-12 02:32:04 2020-07-12 05:32:42 2020-07-12 06:06:41 0:33:59 0:24:14 0:09:45 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
pass 5216379 2020-07-12 02:32:05 2020-07-12 05:34:40 2020-07-12 05:56:40 0:22:00 0:14:01 0:07:59 smithi master rhel 8.0 rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} 2
pass 5216381 2020-07-12 02:32:06 2020-07-12 05:36:42 2020-07-12 06:00:42 0:24:00 0:12:56 0:11:04 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} 2
pass 5216383 2020-07-12 02:32:07 2020-07-12 05:40:42 2020-07-12 06:10:42 0:30:00 0:13:19 0:16:41 smithi master rhel 8.0 rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} 2
pass 5216385 2020-07-12 02:32:08 2020-07-12 05:40:42 2020-07-12 06:04:42 0:24:00 0:17:30 0:06:30 smithi master rhel 8.1 rados/singleton/{all/osd-recovery msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest}} 1
pass 5216387 2020-07-12 02:32:09 2020-07-12 05:42:42 2020-07-12 06:42:42 1:00:00 0:49:58 0:10:02 smithi master centos 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/rados_mon_osdmap_prune} 2
pass 5216389 2020-07-12 02:32:10 2020-07-12 05:42:42 2020-07-12 06:14:41 0:31:59 0:24:04 0:07:55 smithi master centos 8.0 rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} 2
fail 5216391 2020-07-12 02:32:11 2020-07-12 05:44:49 2020-07-12 06:22:49 0:38:00 0:27:26 0:10:34 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{centos_latest} tasks/module_selftest} 2
Failure Reason:

"2020-07-12T06:12:57.383720+0000 mds.b (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi171:z (6915), after 302.595 seconds" in cluster log

pass 5216393 2020-07-12 02:32:12 2020-07-12 05:44:50 2020-07-12 06:06:49 0:21:59 0:14:22 0:07:37 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} 1
pass 5216395 2020-07-12 02:32:13 2020-07-12 05:46:15 2020-07-12 06:22:15 0:36:00 0:22:56 0:13:04 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} 2
pass 5216397 2020-07-12 02:32:13 2020-07-12 05:46:23 2020-07-12 06:02:22 0:15:59 0:07:45 0:08:14 smithi master ubuntu 18.04 rados/multimon/{clusters/21 msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} 3
pass 5216399 2020-07-12 02:32:14 2020-07-12 05:48:06 2020-07-12 06:08:06 0:20:00 0:13:33 0:06:27 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} 1
pass 5216401 2020-07-12 02:32:15 2020-07-12 05:48:22 2020-07-12 06:08:21 0:19:59 0:11:49 0:08:10 smithi master rhel 8.1 rados/singleton/{all/peer msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest}} 1
pass 5216403 2020-07-12 02:32:16 2020-07-12 05:48:29 2020-07-12 06:10:29 0:22:00 0:12:22 0:09:38 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_fio} 1
pass 5216405 2020-07-12 02:32:17 2020-07-12 05:48:38 2020-07-12 06:06:37 0:17:59 0:12:04 0:05:55 smithi master rhel 8.1 rados/singleton-nomsgr/{all/export-after-evict rados supported-random-distro$/{rhel_latest}} 1
pass 5216407 2020-07-12 02:32:18 2020-07-12 05:49:27 2020-07-12 06:31:27 0:42:00 0:14:54 0:27:06 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216409 2020-07-12 02:32:19 2020-07-12 05:50:22 2020-07-12 06:10:21 0:19:59 0:13:25 0:06:34 smithi master rhel 8.1 rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} 2
pass 5216411 2020-07-12 02:32:20 2020-07-12 05:52:32 2020-07-12 06:30:32 0:38:00 0:25:40 0:12:20 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} 2
pass 5216413 2020-07-12 02:32:20 2020-07-12 05:52:39 2020-07-12 06:34:39 0:42:00 0:26:57 0:15:03 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5216415 2020-07-12 02:32:21 2020-07-12 05:52:41 2020-07-12 06:16:40 0:23:59 0:12:20 0:11:39 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/lockdep} 2
fail 5216417 2020-07-12 02:32:22 2020-07-12 05:54:42 2020-07-12 06:40:42 0:46:00 0:23:05 0:22:55 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

fail 5216419 2020-07-12 02:32:23 2020-07-12 05:54:42 2020-07-12 06:30:42 0:36:00 0:27:40 0:08:20 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
Failure Reason:

"2020-07-12T06:17:07.608879+0000 mon.b (mon.1) 26 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

pass 5216421 2020-07-12 02:32:24 2020-07-12 05:56:33 2020-07-12 06:18:32 0:21:59 0:14:17 0:07:42 smithi master ubuntu 18.04 rados/singleton/{all/pg-autoscaler msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} 2
pass 5216423 2020-07-12 02:32:25 2020-07-12 05:57:16 2020-07-12 06:17:16 0:20:00 0:12:19 0:07:41 smithi master rhel 8.1 rados/cephadm/smoke-roleless/{distro/rhel_latest start} 2
pass 5216425 2020-07-12 02:32:26 2020-07-12 05:58:25 2020-07-12 06:42:25 0:44:00 0:30:24 0:13:36 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} tasks/rados_api_tests} 2
pass 5216427 2020-07-12 02:32:27 2020-07-12 05:58:26 2020-07-12 06:46:26 0:48:00 0:30:58 0:17:02 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 2
pass 5216429 2020-07-12 02:32:28 2020-07-12 06:00:39 2020-07-12 06:24:39 0:24:00 0:15:59 0:08:01 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/cache-agent-small} 2
pass 5216431 2020-07-12 02:32:28 2020-07-12 06:02:40 2020-07-12 07:14:41 1:12:01 1:05:44 0:06:17 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_latest} tasks/dashboard} 2
pass 5216433 2020-07-12 02:32:29 2020-07-12 06:04:44 2020-07-12 08:48:47 2:44:03 2:36:56 0:07:07 smithi master ubuntu 18.04 rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{ubuntu_latest}} 1
pass 5216435 2020-07-12 02:32:30 2020-07-12 06:04:44 2020-07-12 06:30:43 0:25:59 0:17:29 0:08:30 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
pass 5216437 2020-07-12 02:32:31 2020-07-12 06:06:40 2020-07-12 06:30:40 0:24:00 0:17:05 0:06:55 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} 1
pass 5216439 2020-07-12 02:32:32 2020-07-12 06:06:40 2020-07-12 06:22:39 0:15:59 0:10:15 0:05:44 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/sample_radosbench} 1
pass 5216441 2020-07-12 02:32:33 2020-07-12 06:06:50 2020-07-12 06:20:50 0:14:00 0:07:23 0:06:37 smithi master ubuntu 18.04 rados/singleton/{all/pg-removal-interruption msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216443 2020-07-12 02:32:34 2020-07-12 06:08:25 2020-07-12 07:10:25 1:02:00 0:33:25 0:28:35 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_api_tests} 2
pass 5216445 2020-07-12 02:32:34 2020-07-12 06:08:27 2020-07-12 06:22:26 0:13:59 0:07:57 0:06:02 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/full-tiering rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216447 2020-07-12 02:32:35 2020-07-12 06:10:32 2020-07-12 06:42:32 0:32:00 0:21:32 0:10:28 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} 2
pass 5216449 2020-07-12 02:32:36 2020-07-12 06:10:32 2020-07-12 06:34:32 0:24:00 0:13:01 0:10:59 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} 2
pass 5216451 2020-07-12 02:32:37 2020-07-12 06:10:35 2020-07-12 07:04:35 0:54:00 0:28:52 0:25:08 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_mon_workunits} 2
pass 5216453 2020-07-12 02:32:38 2020-07-12 06:10:51 2020-07-12 06:34:50 0:23:59 0:18:16 0:05:43 smithi master centos 8.1 rados/singleton/{all/radostool msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} 1
pass 5216455 2020-07-12 02:32:39 2020-07-12 06:12:29 2020-07-12 06:32:29 0:20:00 0:11:39 0:08:21 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} 2
pass 5216457 2020-07-12 02:32:40 2020-07-12 06:14:43 2020-07-12 06:52:43 0:38:00 0:28:50 0:09:10 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} 2
pass 5216459 2020-07-12 02:32:40 2020-07-12 06:14:43 2020-07-12 06:36:43 0:22:00 0:15:23 0:06:37 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} tasks/progress} 2
pass 5216461 2020-07-12 02:32:41 2020-07-12 06:16:28 2020-07-12 06:48:28 0:32:00 0:23:52 0:08:08 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 5216463 2020-07-12 02:32:42 2020-07-12 06:16:43 2020-07-12 06:52:43 0:36:00 0:11:59 0:24:01 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216464 2020-07-12 02:32:43 2020-07-12 06:17:17 2020-07-12 06:37:17 0:20:00 0:07:07 0:12:53 smithi master ubuntu 18.04 rados/multimon/{clusters/3 msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} 2
pass 5216466 2020-07-12 02:32:44 2020-07-12 06:18:43 2020-07-12 06:34:42 0:15:59 0:08:22 0:07:37 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} 1
pass 5216468 2020-07-12 02:32:45 2020-07-12 06:18:42 2020-07-12 07:00:43 0:42:01 0:28:31 0:13:30 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} 3
pass 5216470 2020-07-12 02:32:46 2020-07-12 06:18:43 2020-07-12 06:46:43 0:28:00 0:18:13 0:09:47 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} 1
pass 5216472 2020-07-12 02:32:47 2020-07-12 06:21:03 2020-07-12 06:55:03 0:34:00 0:18:53 0:15:07 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps-balanced} 2
pass 5216475 2020-07-12 02:32:48 2020-07-12 06:22:42 2020-07-12 06:48:42 0:26:00 0:18:08 0:07:52 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} 2
pass 5216477 2020-07-12 02:32:48 2020-07-12 06:24:57 2020-07-12 06:58:57 0:34:00 0:20:59 0:13:01 smithi master ubuntu 18.04 rados/singleton/{all/random-eio msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 2
pass 5216478 2020-07-12 02:32:49 2020-07-12 06:24:58 2020-07-12 07:08:58 0:44:00 0:29:17 0:14:43 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
pass 5216480 2020-07-12 02:32:50 2020-07-12 06:25:53 2020-07-12 07:01:53 0:36:00 0:10:45 0:25:15 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5216482 2020-07-12 02:32:51 2020-07-12 06:26:45 2020-07-12 07:24:45 0:58:00 0:47:36 0:10:24 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/valgrind} 2
pass 5216484 2020-07-12 02:32:52 2020-07-12 06:30:27 2020-07-12 06:54:26 0:23:59 0:17:09 0:06:50 smithi master rhel 8.1 rados/singleton-nomsgr/{all/health-warnings rados supported-random-distro$/{rhel_latest}} 1
pass 5216486 2020-07-12 02:32:53 2020-07-12 06:30:33 2020-07-12 07:12:33 0:42:00 0:31:01 0:10:59 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216488 2020-07-12 02:32:54 2020-07-12 06:30:41 2020-07-12 06:54:40 0:23:59 0:13:35 0:10:24 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} 2
pass 5216490 2020-07-12 02:32:55 2020-07-12 06:30:45 2020-07-12 07:02:45 0:32:00 0:24:46 0:07:14 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps} 2
pass 5216492 2020-07-12 02:32:56 2020-07-12 06:32:31 2020-07-12 07:06:32 0:34:01 0:27:33 0:06:28 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} 1
dead 5216494 2020-07-12 02:32:57 2020-07-12 06:32:42 2020-07-12 06:48:42 0:16:00 0:02:35 0:13:25 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} 2
Failure Reason:

Command failed with status 2: 'ansible-playbook -v --extra-vars \'{"ansible_ssh_user": "ubuntu"}\' -i /etc/ansible/hosts --limit smithi166.front.sepia.ceph.com,smithi037.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml'

fail 5216496 2020-07-12 02:32:58 2020-07-12 06:34:26 2020-07-12 06:56:26 0:22:00 0:14:13 0:07:47 smithi master ubuntu 18.04 rados/singleton/{all/rebuild-mondb msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

"2020-07-12T06:48:35.740532+0000 mon.a (mon.0) 78 : cluster [ERR] Health check failed: 4 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log

pass 5216498 2020-07-12 02:32:59 2020-07-12 06:34:31 2020-07-12 06:54:30 0:19:59 0:12:02 0:07:57 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} 2
pass 5216500 2020-07-12 02:32:59 2020-07-12 06:34:34 2020-07-12 07:22:34 0:48:00 0:41:42 0:06:18 smithi master rhel 8.1 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest}} 1
pass 5216502 2020-07-12 02:33:00 2020-07-12 06:34:41 2020-07-12 07:50:42 1:16:01 1:06:45 0:09:16 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_latest} tasks/dashboard} 2
pass 5216504 2020-07-12 02:33:01 2020-07-12 06:34:52 2020-07-12 08:52:55 2:18:03 2:10:34 0:07:29 smithi master ubuntu 18.04 rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} 1
pass 5216506 2020-07-12 02:33:02 2020-07-12 06:36:52 2020-07-12 06:56:51 0:19:59 0:12:14 0:07:45 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache} 2
pass 5216508 2020-07-12 02:33:03 2020-07-12 06:36:52 2020-07-12 07:00:51 0:23:59 0:15:09 0:08:50 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} 1
pass 5216510 2020-07-12 02:33:04 2020-07-12 06:37:18 2020-07-12 07:01:18 0:24:00 0:18:19 0:05:41 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} 1
pass 5216512 2020-07-12 02:33:05 2020-07-12 06:38:45 2020-07-12 07:04:44 0:25:59 0:19:51 0:06:08 smithi master rhel 8.1 rados/singleton/{all/recovery-preemption msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest}} 1
pass 5216514 2020-07-12 02:33:06 2020-07-12 06:38:45 2020-07-12 06:58:45 0:20:00 0:12:36 0:07:24 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} 2
pass 5216516 2020-07-12 02:33:07 2020-07-12 06:40:51 2020-07-12 07:04:51 0:24:00 0:14:59 0:09:01 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/dedup_tier} 2
pass 5216518 2020-07-12 02:33:08 2020-07-12 06:42:46 2020-07-12 07:04:46 0:22:00 0:16:30 0:05:30 smithi master rhel 8.1 rados/singleton-nomsgr/{all/large-omap-object-warnings rados supported-random-distro$/{rhel_latest}} 1
pass 5216520 2020-07-12 02:33:09 2020-07-12 06:42:46 2020-07-12 07:34:47 0:52:01 0:34:51 0:17:10 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest} thrashers/force-sync-many workloads/snaps-few-objects} 2
pass 5216522 2020-07-12 02:33:09 2020-07-12 06:44:46 2020-07-12 07:08:45 0:23:59 0:14:54 0:09:05 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{rhel_latest} tasks/prometheus} 2
pass 5216524 2020-07-12 02:33:10 2020-07-12 06:46:44 2020-07-12 07:04:43 0:17:59 0:11:44 0:06:15 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2
pass 5216526 2020-07-12 02:33:11 2020-07-12 06:46:45 2020-07-12 07:24:45 0:38:00 0:15:05 0:22:55 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216528 2020-07-12 02:33:12 2020-07-12 06:48:46 2020-07-12 08:26:48 1:38:02 1:27:06 0:10:56 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/radosbench} 3
pass 5216530 2020-07-12 02:33:13 2020-07-12 06:48:46 2020-07-12 07:06:45 0:17:59 0:10:50 0:07:09 smithi master ubuntu 18.04 rados/multimon/{clusters/6 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 2
pass 5216533 2020-07-12 02:33:14 2020-07-12 06:50:34 2020-07-12 07:20:34 0:30:00 0:21:55 0:08:05 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} tasks/rados_python} 2
pass 5216534 2020-07-12 02:33:15 2020-07-12 06:50:34 2020-07-12 07:06:34 0:16:00 0:08:46 0:07:14 smithi master ubuntu 18.04 rados/singleton/{all/resolve_stuck_peering msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 2
pass 5216536 2020-07-12 02:33:16 2020-07-12 06:50:52 2020-07-12 07:36:52 0:46:00 0:37:44 0:08:16 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2
pass 5216538 2020-07-12 02:33:17 2020-07-12 06:53:01 2020-07-12 07:29:01 0:36:00 0:28:42 0:07:18 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} 2
pass 5216540 2020-07-12 02:33:18 2020-07-12 06:53:01 2020-07-12 10:35:06 3:42:05 3:33:34 0:08:31 smithi master ubuntu 18.04 rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} 4
pass 5216542 2020-07-12 02:33:19 2020-07-12 06:54:43 2020-07-12 07:14:44 0:20:01 0:11:49 0:08:12 smithi master ubuntu 18.04 rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} 2
pass 5216544 2020-07-12 02:33:20 2020-07-12 06:54:46 2020-07-12 07:14:45 0:19:59 0:12:13 0:07:46 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_read} 1
pass 5216546 2020-07-12 02:33:21 2020-07-12 06:56:03 2020-07-12 07:42:03 0:46:00 0:30:42 0:15:18 smithi master rhel 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5216548 2020-07-12 02:33:21 2020-07-12 06:56:35 2020-07-12 07:22:34 0:25:59 0:18:26 0:07:33 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/lockdep} 2
pass 5216550 2020-07-12 02:33:22 2020-07-12 06:56:35 2020-07-12 07:36:35 0:40:00 0:30:50 0:09:10 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} 2
pass 5216552 2020-07-12 02:33:23 2020-07-12 06:56:52 2020-07-12 07:36:52 0:40:00 0:32:38 0:07:22 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216554 2020-07-12 02:33:24 2020-07-12 06:58:49 2020-07-12 07:20:48 0:21:59 0:17:31 0:04:28 smithi master rhel 8.1 rados/singleton/{all/test-crash msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest}} 1
pass 5216556 2020-07-12 02:33:25 2020-07-12 06:58:53 2020-07-12 07:22:53 0:24:00 0:14:06 0:09:54 smithi master centos 7.6 rados/cephadm/smoke/{distro/centos_7 fixed-2 start} 2
pass 5216558 2020-07-12 02:33:26 2020-07-12 07:08:57 2020-07-12 07:40:57 0:32:00 0:25:58 0:06:02 smithi master rhel 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} 2
pass 5216560 2020-07-12 02:33:27 2020-07-12 07:10:36 2020-07-12 07:46:36 0:36:00 0:29:15 0:06:45 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} 2
pass 5216562 2020-07-12 02:33:28 2020-07-12 07:10:36 2020-07-12 07:26:35 0:15:59 0:09:11 0:06:48 smithi master centos 8.1 rados/singleton-nomsgr/{all/lazy_omap_stats_output rados supported-random-distro$/{centos_latest}} 1
pass 5216564 2020-07-12 02:33:29 2020-07-12 07:12:52 2020-07-12 07:38:52 0:26:00 0:13:10 0:12:50 smithi master centos 7.6 rados/cephadm/smoke-roleless/{distro/centos_7 start} 2
pass 5216566 2020-07-12 02:33:30 2020-07-12 07:12:52 2020-07-12 08:06:53 0:54:01 0:46:06 0:07:55 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} 2
pass 5216568 2020-07-12 02:33:31 2020-07-12 07:14:59 2020-07-12 07:40:59 0:26:00 0:18:24 0:07:36 smithi master centos 8.1 rados/singleton/{all/test_envlibrados_for_rocksdb msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} 1
pass 5216570 2020-07-12 02:33:32 2020-07-12 07:14:59 2020-07-12 07:36:59 0:22:00 0:14:34 0:07:26 smithi master centos 8.0 rados/cephadm/upgrade/{1-start 2-start-upgrade 3-wait distro$/{centos_8.0} fixed-2} 2
pass 5216572 2020-07-12 02:33:33 2020-07-12 07:15:00 2020-07-12 08:25:01 1:10:01 1:02:12 0:07:49 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
pass 5216574 2020-07-12 02:33:34 2020-07-12 07:16:58 2020-07-12 07:34:57 0:17:59 0:10:56 0:07:03 smithi master ubuntu 18.04 rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} 1
pass 5216576 2020-07-12 02:33:34 2020-07-12 07:19:05 2020-07-12 07:39:05 0:20:00 0:12:25 0:07:35 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} 1
pass 5216578 2020-07-12 02:33:35 2020-07-12 07:20:52 2020-07-12 08:48:54 1:28:02 1:20:06 0:07:56 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/radosbench} 2
pass 5216580 2020-07-12 02:33:36 2020-07-12 07:20:52 2020-07-12 07:38:52 0:18:00 0:12:28 0:05:32 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_adoption} 1
pass 5216582 2020-07-12 02:33:37 2020-07-12 07:22:23 2020-07-12 08:30:24 1:08:01 0:21:21 0:46:40 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/rbd_cls} 3
pass 5216584 2020-07-12 02:33:38 2020-07-12 07:22:36 2020-07-12 07:52:36 0:30:00 0:20:25 0:09:35 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} 2
pass 5216586 2020-07-12 02:33:39 2020-07-12 07:22:54 2020-07-12 07:42:53 0:19:59 0:12:31 0:07:28 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} tasks/workunits} 2
pass 5216588 2020-07-12 02:33:40 2020-07-12 07:24:59 2020-07-12 07:56:59 0:32:00 0:16:46 0:15:14 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/many workloads/pool-create-delete} 2
pass 5216590 2020-07-12 02:33:41 2020-07-12 07:25:00 2020-07-12 08:25:00 1:00:00 0:53:10 0:06:50 smithi master rhel 8.1 rados/standalone/{supported-random-distro$/{rhel_latest} workloads/mon} 1
pass 5216592 2020-07-12 02:33:42 2020-07-12 07:25:06 2020-07-12 07:59:05 0:33:59 0:14:04 0:19:55 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216594 2020-07-12 02:33:43 2020-07-12 07:26:30 2020-07-12 08:42:31 1:16:01 1:01:16 0:14:45 smithi master centos 8.1 rados/singleton/{all/thrash-backfill-full msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest}} 2
pass 5216596 2020-07-12 02:33:44 2020-07-12 07:27:05 2020-07-12 08:07:05 0:40:00 0:26:19 0:13:41 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} 2
pass 5216598 2020-07-12 02:33:44 2020-07-12 07:29:03 2020-07-12 07:59:02 0:29:59 0:17:51 0:12:08 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/redirect} 2
pass 5216600 2020-07-12 02:33:45 2020-07-12 07:30:54 2020-07-12 08:12:54 0:42:00 0:07:47 0:34:13 smithi master ubuntu 18.04 rados/multimon/{clusters/9 msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} 3
pass 5216602 2020-07-12 02:33:46 2020-07-12 07:32:11 2020-07-12 07:54:11 0:22:00 0:13:09 0:08:51 smithi master centos 8.1 rados/singleton-nomsgr/{all/librados_hello_world rados supported-random-distro$/{centos_latest}} 1
pass 5216604 2020-07-12 02:33:47 2020-07-12 07:32:48 2020-07-12 07:54:48 0:22:00 0:11:36 0:10:24 smithi master centos 8.0 rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} 2
pass 5216606 2020-07-12 02:33:48 2020-07-12 07:35:04 2020-07-12 08:25:04 0:50:00 0:42:53 0:07:07 smithi master centos 8.1 rados/singleton/{all/thrash-eio msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} 2
pass 5216608 2020-07-12 02:33:49 2020-07-12 07:35:04 2020-07-12 07:53:03 0:17:59 0:10:47 0:07:12 smithi master centos 8.0 rados/cephadm/smoke-roleless/{distro/centos_8.0 start} 2
pass 5216610 2020-07-12 02:33:50 2020-07-12 07:36:52 2020-07-12 08:02:51 0:25:59 0:10:10 0:15:49 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5216612 2020-07-12 02:33:51 2020-07-12 07:36:54 2020-07-12 08:10:54 0:34:00 0:26:06 0:07:54 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind} 2
pass 5216614 2020-07-12 02:33:52 2020-07-12 07:37:00 2020-07-12 08:01:00 0:24:00 0:17:12 0:06:48 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/redirect_promote_tests} 2
pass 5216616 2020-07-12 02:33:53 2020-07-12 07:38:52 2020-07-12 07:56:52 0:18:00 0:10:41 0:07:19 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_read} 1
pass 5216618 2020-07-12 02:33:54 2020-07-12 07:38:53 2020-07-12 08:20:53 0:42:00 0:28:32 0:13:28 smithi master centos 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216620 2020-07-12 02:33:55 2020-07-12 07:38:57 2020-07-12 08:08:56 0:29:59 0:22:09 0:07:50 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} 1
pass 5216622 2020-07-12 02:33:56 2020-07-12 07:39:06 2020-07-12 09:47:09 2:08:03 1:38:57 0:29:06 smithi master centos 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-radosbench} 2
pass 5216624 2020-07-12 02:33:57 2020-07-12 07:40:48 2020-07-12 08:14:48 0:34:00 0:16:06 0:17:54 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} 2
pass 5216626 2020-07-12 02:33:58 2020-07-12 07:40:56 2020-07-12 08:16:56 0:36:00 0:24:10 0:11:50 smithi master centos 8.1 rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} 2
pass 5216628 2020-07-12 02:33:59 2020-07-12 07:41:00 2020-07-12 08:23:00 0:42:00 0:36:19 0:05:41 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async start tasks/rados_api_tests} 2
pass 5216631 2020-07-12 02:33:59 2020-07-12 07:42:57 2020-07-12 08:06:56 0:23:59 0:16:18 0:07:41 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/msgr rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216633 2020-07-12 02:34:00 2020-07-12 07:44:45 2020-07-12 08:10:45 0:26:00 0:11:05 0:14:55 smithi master centos 8.1 rados/cephadm/smoke/{distro/centos_latest fixed-2 start} 2
pass 5216635 2020-07-12 02:34:01 2020-07-12 07:46:54 2020-07-12 08:38:54 0:52:00 0:40:17 0:11:43 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/few rados thrashers/default thrashosds-health workloads/snaps-few-objects} 3
pass 5216637 2020-07-12 02:34:02 2020-07-12 07:46:54 2020-07-12 08:02:53 0:15:59 0:08:21 0:07:38 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} 2
pass 5216639 2020-07-12 02:34:03 2020-07-12 07:47:00 2020-07-12 08:14:59 0:27:59 0:11:24 0:16:35 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/set-chunk-promote-flush} 2
fail 5216641 2020-07-12 02:34:04 2020-07-12 07:49:05 2020-07-12 08:19:05 0:30:00 0:20:04 0:09:56 smithi master rhel 8.1 rados/singleton/{all/thrash_cache_writeback_proxy_none msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest}} 2
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base'

pass 5216643 2020-07-12 02:34:05 2020-07-12 07:51:00 2020-07-12 08:20:59 0:29:59 0:10:21 0:19:38 smithi master centos 8.1 rados/cephadm/smoke-roleless/{distro/centos_latest start} 2
pass 5216645 2020-07-12 02:34:06 2020-07-12 07:53:04 2020-07-12 08:15:03 0:21:59 0:10:15 0:11:44 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{centos_latest} tasks/crash} 2
pass 5216647 2020-07-12 02:34:07 2020-07-12 07:53:05 2020-07-12 08:15:05 0:22:00 0:13:33 0:08:27 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} 1
pass 5216649 2020-07-12 02:34:08 2020-07-12 07:54:39 2020-07-12 08:40:40 0:46:01 0:37:32 0:08:29 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216651 2020-07-12 02:34:09 2020-07-12 07:57:09 2020-07-12 08:43:10 0:46:01 0:36:46 0:09:15 smithi master rhel 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
pass 5216653 2020-07-12 02:34:10 2020-07-12 07:59:06 2020-07-12 08:19:05 0:19:59 0:11:35 0:08:24 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216655 2020-07-12 02:34:11 2020-07-12 07:59:06 2020-07-12 09:09:07 1:10:01 1:02:46 0:07:15 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
pass 5216657 2020-07-12 02:34:11 2020-07-12 07:59:07 2020-07-12 08:13:06 0:13:59 0:07:06 0:06:53 smithi master ubuntu 18.04 rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} 1
pass 5216659 2020-07-12 02:34:12 2020-07-12 08:11:03 2020-07-12 08:35:02 0:23:59 0:17:15 0:06:44 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} 1
pass 5216661 2020-07-12 02:34:13 2020-07-12 08:11:03 2020-07-12 08:31:02 0:19:59 0:08:58 0:11:01 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_5925} 2
pass 5216663 2020-07-12 02:34:14 2020-07-12 08:13:12 2020-07-12 08:39:12 0:26:00 0:18:05 0:07:55 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/set-chunks-read} 2
pass 5216665 2020-07-12 02:34:15 2020-07-12 08:13:12 2020-07-12 08:27:12 0:14:00 0:08:33 0:05:27 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} 1
pass 5216667 2020-07-12 02:34:16 2020-07-12 08:15:06 2020-07-12 09:15:07 1:00:01 0:11:24 0:48:37 smithi master rhel 8.1 rados/multimon/{clusters/21 msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} tasks/mon_clock_with_skews} 3
pass 5216669 2020-07-12 02:34:17 2020-07-12 08:15:06 2020-07-12 08:29:05 0:13:59 0:07:53 0:06:06 smithi master centos 8.1 rados/singleton/{all/watch-notify-same-primary msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} 1
pass 5216671 2020-07-12 02:34:18 2020-07-12 08:15:06 2020-07-12 08:47:06 0:32:00 0:25:14 0:06:46 smithi master centos 8.0 rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
pass 5216673 2020-07-12 02:34:19 2020-07-12 08:16:24 2020-07-12 08:48:24 0:32:00 0:26:26 0:05:34 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} 2
pass 5216675 2020-07-12 02:34:20 2020-07-12 08:16:50 2020-07-12 08:42:50 0:26:00 0:13:40 0:12:20 smithi master rhel 7.7 rados/cephadm/smoke/{distro/rhel_7 fixed-2 start} 2
pass 5216677 2020-07-12 02:34:21 2020-07-12 08:16:57 2020-07-12 08:58:57 0:42:00 0:26:15 0:15:45 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5216679 2020-07-12 02:34:21 2020-07-12 08:17:07 2020-07-12 08:49:07 0:32:00 0:25:57 0:06:03 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/lockdep} 2
pass 5216681 2020-07-12 02:34:22 2020-07-12 08:19:08 2020-07-12 08:49:08 0:30:00 0:21:15 0:08:45 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/multi-backfill-reject rados supported-random-distro$/{ubuntu_latest}} 2
fail 5216683 2020-07-12 02:34:23 2020-07-12 08:21:06 2020-07-12 11:13:10 2:52:04 2:44:46 0:07:18 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} 1
Failure Reason:

Command failed (workunit test osd/osd-rep-recov-eio.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ca563407a6946e29bd297949d09f1b669a9befa5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-rep-recov-eio.sh'

pass 5216685 2020-07-12 02:34:24 2020-07-12 08:21:07 2020-07-12 08:57:06 0:35:59 0:28:46 0:07:13 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216687 2020-07-12 02:34:25 2020-07-12 08:23:11 2020-07-12 08:37:11 0:14:00 0:06:59 0:07:01 smithi master ubuntu 18.04 rados/singleton/{all/admin-socket msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216689 2020-07-12 02:34:26 2020-07-12 08:23:11 2020-07-12 08:43:11 0:20:00 0:12:24 0:07:36 smithi master rhel 7.7 rados/cephadm/smoke-roleless/{distro/rhel_7 start} 2
pass 5216691 2020-07-12 02:34:27 2020-07-12 08:25:19 2020-07-12 08:45:18 0:19:59 0:11:08 0:08:51 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_write} 1
pass 5216693 2020-07-12 02:34:28 2020-07-12 08:25:19 2020-07-12 08:55:19 0:30:00 0:21:19 0:08:41 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects-localized} 2
pass 5216695 2020-07-12 02:34:29 2020-07-12 08:27:06 2020-07-12 08:55:05 0:27:59 0:19:50 0:08:09 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-balanced} 2
pass 5216697 2020-07-12 02:34:30 2020-07-12 08:27:13 2020-07-12 08:49:13 0:22:00 0:16:09 0:05:51 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
pass 5216699 2020-07-12 02:34:31 2020-07-12 08:29:23 2020-07-12 09:19:24 0:50:01 0:34:47 0:15:14 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_big} 2
fail 5216701 2020-07-12 02:34:32 2020-07-12 08:30:42 2020-07-12 09:08:42 0:38:00 0:21:58 0:16:02 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/hammer backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

pass 5216703 2020-07-12 02:34:33 2020-07-12 08:33:22 2020-07-12 09:11:22 0:38:00 0:26:18 0:11:42 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/small-objects} 2
pass 5216705 2020-07-12 02:34:33 2020-07-12 08:35:08 2020-07-12 08:57:08 0:22:00 0:16:05 0:05:55 smithi master rhel 8.1 rados/singleton/{all/deduptool msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest}} 1
pass 5216707 2020-07-12 02:34:34 2020-07-12 08:35:08 2020-07-12 08:57:08 0:22:00 0:14:12 0:07:48 smithi master rhel 8.0 rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} 2
pass 5216709 2020-07-12 02:34:35 2020-07-12 08:37:16 2020-07-12 09:25:16 0:48:00 0:11:25 0:36:35 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216711 2020-07-12 02:34:36 2020-07-12 08:37:16 2020-07-12 09:05:16 0:28:00 0:15:43 0:12:17 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{rhel_latest} tasks/failover} 2
pass 5216713 2020-07-12 02:34:37 2020-07-12 08:37:16 2020-07-12 09:01:16 0:24:00 0:13:37 0:10:23 smithi master rhel 8.0 rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} 2
pass 5216715 2020-07-12 02:34:38 2020-07-12 08:39:04 2020-07-12 09:03:04 0:24:00 0:17:45 0:06:15 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/osd_stale_reads rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216717 2020-07-12 02:34:39 2020-07-12 08:39:13 2020-07-12 09:13:13 0:34:00 0:28:07 0:05:53 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} 2
pass 5216719 2020-07-12 02:34:40 2020-07-12 08:40:58 2020-07-12 09:16:58 0:36:00 0:25:40 0:10:20 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/rados_api_tests} 2
pass 5216721 2020-07-12 02:34:41 2020-07-12 08:41:07 2020-07-12 08:59:07 0:18:00 0:09:03 0:08:57 smithi master centos 8.1 rados/singleton/{all/divergent_priors msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} 1
pass 5216723 2020-07-12 02:34:42 2020-07-12 08:42:48 2020-07-12 09:02:48 0:20:00 0:12:03 0:07:57 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} 1
pass 5216725 2020-07-12 02:34:42 2020-07-12 08:42:59 2020-07-12 09:24:59 0:42:00 0:34:00 0:08:00 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2
pass 5216727 2020-07-12 02:34:43 2020-07-12 08:43:12 2020-07-12 09:57:13 1:14:01 1:06:06 0:07:55 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{rhel_latest} tasks/dashboard} 2
pass 5216729 2020-07-12 02:34:44 2020-07-12 08:45:16 2020-07-12 09:19:15 0:33:59 0:27:53 0:06:06 smithi master centos 8.1 rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_latest}} 1
pass 5216731 2020-07-12 02:34:45 2020-07-12 08:45:16 2020-07-12 09:05:15 0:19:59 0:09:16 0:10:43 smithi master ubuntu 18.04 rados/multimon/{clusters/3 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 2
pass 5216733 2020-07-12 02:34:46 2020-07-12 08:45:16 2020-07-12 09:21:16 0:36:00 0:26:32 0:09:28 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} 2
pass 5216735 2020-07-12 02:34:47 2020-07-12 08:45:20 2020-07-12 09:07:20 0:22:00 0:13:42 0:08:18 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} 1
pass 5216737 2020-07-12 02:34:48 2020-07-12 08:47:14 2020-07-12 09:03:13 0:15:59 0:08:53 0:07:06 smithi master ubuntu 18.04 rados/singleton/{all/divergent_priors2 msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216739 2020-07-12 02:34:49 2020-07-12 08:48:43 2020-07-12 09:34:43 0:46:00 0:11:26 0:34:34 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5216741 2020-07-12 02:34:50 2020-07-12 08:48:55 2020-07-12 10:16:56 1:28:01 1:15:34 0:12:27 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} 2
pass 5216743 2020-07-12 02:34:51 2020-07-12 08:49:09 2020-07-12 09:13:09 0:24:00 0:13:01 0:10:59 smithi master rhel 8.1 rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} 2
pass 5216745 2020-07-12 02:34:52 2020-07-12 08:49:17 2020-07-12 09:21:17 0:32:00 0:22:07 0:09:53 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 5216747 2020-07-12 02:34:52 2020-07-12 08:51:00 2020-07-12 09:27:00 0:36:00 0:26:38 0:09:22 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/snaps-few-objects} 2
pass 5216749 2020-07-12 02:34:53 2020-07-12 08:53:16 2020-07-12 09:33:17 0:40:01 0:32:17 0:07:44 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216751 2020-07-12 02:34:54 2020-07-12 08:55:08 2020-07-12 09:27:08 0:32:00 0:24:49 0:07:11 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} 2
pass 5216753 2020-07-12 02:34:55 2020-07-12 08:55:12 2020-07-12 09:41:12 0:46:00 0:32:04 0:13:56 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/jewel-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/off distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} 3
pass 5216755 2020-07-12 02:34:56 2020-07-12 08:57:24 2020-07-12 09:17:24 0:20:00 0:12:27 0:07:33 smithi master rhel 8.1 rados/cephadm/smoke-roleless/{distro/rhel_latest start} 2
pass 5216757 2020-07-12 02:34:57 2020-07-12 08:57:24 2020-07-12 09:19:24 0:22:00 0:15:52 0:06:08 smithi master rhel 8.1 rados/singleton-nomsgr/{all/pool-access rados supported-random-distro$/{rhel_latest}} 1
pass 5216759 2020-07-12 02:34:58 2020-07-12 08:57:24 2020-07-12 09:23:24 0:26:00 0:19:18 0:06:42 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read} 2
pass 5216761 2020-07-12 02:34:59 2020-07-12 08:59:09 2020-07-12 09:17:09 0:18:00 0:10:41 0:07:19 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} 1
pass 5216763 2020-07-12 02:35:00 2020-07-12 08:59:09 2020-07-12 09:17:09 0:18:00 0:08:47 0:09:13 smithi master ubuntu 18.04 rados/singleton/{all/dump-stuck msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216765 2020-07-12 02:35:01 2020-07-12 08:59:10 2020-07-12 09:35:10 0:36:00 0:28:23 0:07:37 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} 2
pass 5216767 2020-07-12 02:35:01 2020-07-12 09:01:31 2020-07-12 09:31:31 0:30:00 0:14:28 0:15:32 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} 2
pass 5216769 2020-07-12 02:35:02 2020-07-12 09:01:31 2020-07-12 09:25:31 0:24:00 0:16:35 0:07:25 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} 1
pass 5216771 2020-07-12 02:35:03 2020-07-12 09:03:06 2020-07-12 09:41:06 0:38:00 0:15:13 0:22:47 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216773 2020-07-12 02:35:04 2020-07-12 09:03:07 2020-07-12 09:27:06 0:23:59 0:12:23 0:11:36 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/insights} 2
pass 5216775 2020-07-12 02:35:05 2020-07-12 09:05:34 2020-07-12 09:43:34 0:38:00 0:30:54 0:07:06 smithi master ubuntu 18.04 rados/singleton/{all/ec-lost-unfound msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216777 2020-07-12 02:35:06 2020-07-12 09:07:38 2020-07-12 10:31:40 1:24:02 1:18:31 0:05:31 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} 1
pass 5216779 2020-07-12 02:35:07 2020-07-12 09:09:10 2020-07-12 09:39:10 0:30:00 0:22:40 0:07:20 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} 2
pass 5216781 2020-07-12 02:35:08 2020-07-12 09:09:10 2020-07-12 09:29:10 0:20:00 0:12:38 0:07:22 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} 2
pass 5216783 2020-07-12 02:35:08 2020-07-12 09:11:17 2020-07-12 10:11:18 1:00:01 0:53:45 0:06:16 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/sync workloads/rados_mon_osdmap_prune} 2
pass 5216785 2020-07-12 02:35:09 2020-07-12 09:11:23 2020-07-12 09:29:23 0:18:00 0:11:40 0:06:20 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} 2
pass 5216787 2020-07-12 02:35:10 2020-07-12 09:13:13 2020-07-12 09:31:12 0:17:59 0:11:37 0:06:22 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} 1
pass 5216789 2020-07-12 02:35:11 2020-07-12 09:13:13 2020-07-12 09:29:13 0:16:00 0:07:17 0:08:43 smithi master ubuntu 18.04 rados/multimon/{clusters/6 msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} 2
pass 5216791 2020-07-12 02:35:12 2020-07-12 09:13:18 2020-07-12 09:45:18 0:32:00 0:22:58 0:09:02 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-agent-big} 2
pass 5216793 2020-07-12 02:35:13 2020-07-12 09:15:16 2020-07-12 09:49:16 0:34:00 0:27:54 0:06:06 smithi master centos 8.1 rados/singleton-nomsgr/{all/recovery-unfound-found rados supported-random-distro$/{centos_latest}} 1
pass 5216795 2020-07-12 02:35:14 2020-07-12 09:16:53 2020-07-12 09:34:52 0:17:59 0:11:08 0:06:51 smithi master ubuntu 18.04 rados/singleton/{all/erasure-code-nonregression msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216797 2020-07-12 02:35:14 2020-07-12 09:16:59 2020-07-12 10:00:59 0:44:00 0:38:17 0:05:43 smithi master centos 8.1 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} 1
pass 5216799 2020-07-12 02:35:15 2020-07-12 09:17:12 2020-07-12 10:31:11 1:13:59 1:01:56 0:12:03 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{centos_latest} tasks/dashboard} 2
pass 5216801 2020-07-12 02:35:16 2020-07-12 09:17:25 2020-07-12 14:03:32 4:46:07 4:34:28 0:11:39 smithi master centos 8.1 rados/objectstore/{backends/objectstore supported-random-distro$/{centos_latest}} 1
pass 5216803 2020-07-12 02:35:17 2020-07-12 09:19:35 2020-07-12 10:05:35 0:46:00 0:37:27 0:08:33 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async start tasks/rados_api_tests} 2
pass 5216805 2020-07-12 02:35:18 2020-07-12 09:19:35 2020-07-12 10:05:35 0:46:00 0:35:28 0:10:32 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} tasks/rados_workunit_loadgen_mostlyread} 2
fail 5216807 2020-07-12 02:35:19 2020-07-12 09:21:17 2020-07-12 10:03:17 0:42:00 0:23:02 0:18:58 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/jewel backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} 3
Failure Reason:

reached maximum tries (180) after waiting for 180 seconds

pass 5216809 2020-07-12 02:35:20 2020-07-12 09:21:18 2020-07-12 10:01:18 0:40:00 0:29:02 0:10:58 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5216811 2020-07-12 02:35:20 2020-07-12 09:21:18 2020-07-12 09:43:17 0:21:59 0:12:00 0:09:59 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} 2
pass 5216813 2020-07-12 02:35:21 2020-07-12 09:23:14 2020-07-12 09:39:13 0:15:59 0:08:28 0:07:31 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} 1
pass 5216815 2020-07-12 02:35:22 2020-07-12 09:23:26 2020-07-12 09:47:26 0:24:00 0:16:00 0:08:00 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/upmap msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-small} 2
pass 5216817 2020-07-12 02:35:23 2020-07-12 09:25:17 2020-07-12 09:55:16 0:29:59 0:23:07 0:06:52 smithi master centos 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5216819 2020-07-12 02:35:24 2020-07-12 09:25:17 2020-07-12 09:59:17 0:34:00 0:27:24 0:06:36 smithi master ubuntu 18.04 rados/singleton/{all/lost-unfound-delete msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
pass 5216821 2020-07-12 02:35:25 2020-07-12 09:25:32 2020-07-12 09:45:32 0:20:00 0:13:44 0:06:16 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} 2
pass 5216823 2020-07-12 02:35:26 2020-07-12 09:27:22 2020-07-12 09:59:22 0:32:00 0:24:11 0:07:49 smithi master centos 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-many-deletes} 2
pass 5216824 2020-07-12 02:35:27 2020-07-12 09:27:22 2020-07-12 10:01:22 0:34:00 0:22:51 0:11:09 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} 2
pass 5216826 2020-07-12 02:35:27 2020-07-12 09:27:22 2020-07-12 09:47:21 0:19:59 0:11:53 0:08:06 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} 2
pass 5216828 2020-07-12 02:35:28 2020-07-12 09:29:26 2020-07-12 09:49:26 0:20:00 0:11:43 0:08:17 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} 1
pass 5216830 2020-07-12 02:35:29 2020-07-12 09:29:27 2020-07-12 09:49:26 0:19:59 0:13:28 0:06:31 smithi master centos 8.1 rados/singleton-nomsgr/{all/version-number-sanity rados supported-random-distro$/{centos_latest}} 1
pass 5216832 2020-07-12 02:35:30 2020-07-12 09:29:27 2020-07-12 10:11:27 0:42:00 0:34:28 0:07:32 smithi master rhel 8.1 rados/singleton/{all/lost-unfound msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest}} 1
pass 5216834 2020-07-12 02:35:31 2020-07-12 09:31:26 2020-07-12 09:59:25 0:27:59 0:20:53 0:07:06 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} 2
pass 5216836 2020-07-12 02:35:32 2020-07-12 09:31:26 2020-07-12 09:51:25 0:19:59 0:11:17 0:08:42 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5216837 2020-07-12 02:35:33 2020-07-12 09:31:26 2020-07-12 09:53:25 0:21:59 0:15:36 0:06:23 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} 1
pass 5216838 2020-07-12 02:35:34 2020-07-12 09:31:31 2020-07-12 09:57:31 0:26:00 0:18:47 0:07:13 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/off msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps} 2
pass 5216839 2020-07-12 02:35:35 2020-07-12 09:31:33 2020-07-12 10:09:33 0:38:00 0:30:44 0:07:16 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_latest} tasks/module_selftest} 2
pass 5216840 2020-07-12 02:35:36 2020-07-12 09:33:30 2020-07-12 10:11:30 0:38:00 0:26:52 0:11:08 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} 2
pass 5216841 2020-07-12 02:35:37 2020-07-12 09:33:30 2020-07-12 10:13:30 0:40:00 0:29:38 0:10:22 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_workunits} 2
pass 5216842 2020-07-12 02:35:38 2020-07-12 09:35:01 2020-07-12 09:53:01 0:18:00 0:11:04 0:06:56 smithi master rhel 8.1 rados/singleton/{all/max-pg-per-osd.from-mon msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest}} 1