Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 5181578 2020-06-26 07:01:23 2020-06-27 09:22:14 2020-06-27 09:46:14 0:24:00 0:11:05 0:12:55 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2
pass 5181579 2020-06-26 07:01:24 2020-06-27 09:22:23 2020-06-27 10:02:23 0:40:00 0:30:58 0:09:02 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181580 2020-06-26 07:01:25 2020-06-27 09:22:23 2020-06-27 10:06:24 0:44:01 0:28:12 0:15:49 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
pass 5181581 2020-06-26 07:01:26 2020-06-27 09:22:36 2020-06-27 10:08:36 0:46:00 0:20:38 0:25:22 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} 2
pass 5181582 2020-06-26 07:01:27 2020-06-27 09:24:25 2020-06-27 09:40:25 0:16:00 0:11:02 0:04:58 smithi master centos 8.1 rados/singleton/{all/mon-config msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} 1
pass 5181583 2020-06-26 07:01:28 2020-06-27 09:24:25 2020-06-27 09:44:25 0:20:00 0:12:50 0:07:10 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} 1
pass 5181584 2020-06-26 07:01:29 2020-06-27 09:25:59 2020-06-27 10:01:59 0:36:00 0:25:21 0:10:39 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} 2
pass 5181585 2020-06-26 07:01:30 2020-06-27 09:26:06 2020-06-27 09:54:06 0:28:00 0:09:03 0:18:57 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} 1
pass 5181586 2020-06-26 07:01:31 2020-06-27 09:26:08 2020-06-27 10:10:08 0:44:00 0:37:31 0:06:29 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_api_tests} 2
pass 5181587 2020-06-26 07:01:32 2020-06-27 09:26:08 2020-06-27 10:38:09 1:12:01 1:05:35 0:06:26 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} 2
pass 5181588 2020-06-26 07:01:32 2020-06-27 09:26:17 2020-06-27 09:56:17 0:30:00 0:10:35 0:19:25 smithi master centos 8.1 rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} 1
pass 5181589 2020-06-26 07:01:33 2020-06-27 09:26:23 2020-06-27 09:52:23 0:26:00 0:15:55 0:10:05 smithi master rhel 8.1 rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} 1
pass 5181590 2020-06-26 07:01:34 2020-06-27 09:28:17 2020-06-27 10:04:17 0:36:00 0:13:33 0:22:27 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/admin_socket_output rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181591 2020-06-26 07:01:35 2020-06-27 09:28:17 2020-06-27 10:02:17 0:34:00 0:17:02 0:16:58 smithi master rhel 8.1 rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} 1
fail 5181592 2020-06-26 07:01:36 2020-06-27 09:30:47 2020-06-27 14:14:54 4:44:07 3:44:02 1:00:05 smithi master ubuntu 18.04 rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} 4
Failure Reason:

"2020-06-27T10:39:59.829127+0000 mon.b (mon.0) 300 : cluster [ERR] Health check failed: 4 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log

pass 5181593 2020-06-26 07:01:37 2020-06-27 09:30:47 2020-06-27 09:58:47 0:28:00 0:17:43 0:10:17 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} 1
fail 5181594 2020-06-26 07:01:38 2020-06-27 09:32:35 2020-06-27 11:06:36 1:34:01 1:27:35 0:06:26 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_python} 2
Failure Reason:

"2020-06-27T09:53:28.245186+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.4316.0:23 8.9 8.9 (undecoded) ondisk+read+ignore_overlay+known_if_redirected e64) initiated 2020-06-27T09:52:58.216204+0000 currently delayed" in cluster log

pass 5181595 2020-06-26 07:01:39 2020-06-27 09:32:35 2020-06-27 10:00:34 0:27:59 0:19:11 0:08:48 smithi master ubuntu 18.04 rados/singleton/{all/osd-backfill msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181596 2020-06-26 07:01:40 2020-06-27 09:32:35 2020-06-27 10:18:35 0:46:00 0:28:52 0:17:08 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} 2
pass 5181597 2020-06-26 07:01:40 2020-06-27 09:34:23 2020-06-27 09:52:23 0:18:00 0:06:33 0:11:27 smithi master centos 8.1 rados/multimon/{clusters/21 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} 3
pass 5181598 2020-06-26 07:01:41 2020-06-27 09:36:24 2020-06-27 09:58:23 0:21:59 0:10:55 0:11:04 smithi master ubuntu 18.04 rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} 2
pass 5181599 2020-06-26 07:01:42 2020-06-27 09:36:24 2020-06-27 10:46:25 1:10:01 0:10:53 0:59:08 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181600 2020-06-26 07:01:43 2020-06-27 09:36:25 2020-06-27 09:56:24 0:19:59 0:12:36 0:07:23 smithi master centos 8.0 rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} 2
pass 5181601 2020-06-26 07:01:44 2020-06-27 09:36:32 2020-06-27 10:06:32 0:30:00 0:20:50 0:09:10 smithi master ubuntu 18.04 rados/singleton/{all/osd-recovery-incomplete msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181602 2020-06-26 07:01:45 2020-06-27 09:38:40 2020-06-27 10:10:39 0:31:59 0:23:48 0:08:11 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} 1
pass 5181603 2020-06-26 07:01:46 2020-06-27 09:38:40 2020-06-27 10:04:39 0:25:59 0:10:55 0:15:04 smithi master centos 8.0 rados/cephadm/smoke-roleless/{distro/centos_8.0 start} 2
pass 5181604 2020-06-26 07:01:46 2020-06-27 09:38:40 2020-06-27 10:10:39 0:31:59 0:22:20 0:09:39 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} 2
pass 5181605 2020-06-26 07:01:47 2020-06-27 09:38:40 2020-06-27 10:14:40 0:36:00 0:26:46 0:09:14 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5181606 2020-06-26 07:01:48 2020-06-27 09:38:50 2020-06-27 10:14:50 0:36:00 0:24:07 0:11:53 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_api_tests validater/lockdep} 2
pass 5181607 2020-06-26 07:01:49 2020-06-27 09:40:23 2020-06-27 10:06:23 0:26:00 0:15:22 0:10:38 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/progress} 2
pass 5181608 2020-06-26 07:01:50 2020-06-27 09:40:23 2020-06-27 10:28:24 0:48:01 0:36:18 0:11:43 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} 3
pass 5181609 2020-06-26 07:01:51 2020-06-27 09:40:24 2020-06-27 10:54:25 1:14:01 1:02:38 0:11:23 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} 2
pass 5181610 2020-06-26 07:01:52 2020-06-27 09:40:26 2020-06-27 10:14:26 0:34:00 0:17:36 0:16:24 smithi master ubuntu 18.04 rados/cephadm/upgrade/{1-start 2-start-upgrade 3-wait distro$/{ubuntu_latest} fixed-2} 2
pass 5181611 2020-06-26 07:01:52 2020-06-27 09:42:39 2020-06-27 10:04:39 0:22:00 0:16:10 0:05:50 smithi master rhel 8.1 rados/singleton-nomsgr/{all/balancer rados supported-random-distro$/{rhel_8}} 1
pass 5181612 2020-06-26 07:01:53 2020-06-27 09:42:40 2020-06-27 10:16:40 0:34:00 0:26:28 0:07:32 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181613 2020-06-26 07:01:54 2020-06-27 09:42:39 2020-06-27 10:06:39 0:24:00 0:12:39 0:11:21 smithi master ubuntu 18.04 rados/singleton/{all/osd-recovery msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181614 2020-06-26 07:01:55 2020-06-27 09:44:41 2020-06-27 10:00:41 0:16:00 0:10:10 0:05:50 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_adoption} 1
pass 5181615 2020-06-26 07:01:56 2020-06-27 09:44:42 2020-06-27 10:22:42 0:38:00 0:29:20 0:08:40 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} 2
pass 5181616 2020-06-26 07:01:57 2020-06-27 09:44:41 2020-06-27 10:24:41 0:40:00 0:28:10 0:11:50 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 2
pass 5181617 2020-06-26 07:01:58 2020-06-27 09:46:25 2020-06-27 10:24:25 0:38:00 0:21:03 0:16:57 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} 2
pass 5181618 2020-06-26 07:01:59 2020-06-27 09:46:25 2020-06-27 10:24:25 0:38:00 0:28:59 0:09:01 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_python} 2
pass 5181619 2020-06-26 07:02:00 2020-06-27 09:46:25 2020-06-27 10:06:25 0:20:00 0:12:05 0:07:55 smithi master rhel 8.1 rados/singleton/{all/peer msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} 1
pass 5181620 2020-06-26 07:02:00 2020-06-27 09:46:37 2020-06-27 10:10:37 0:24:00 0:11:45 0:12:15 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/sample_fio} 1
pass 5181621 2020-06-26 07:02:01 2020-06-27 09:48:36 2020-06-27 10:18:36 0:30:00 0:21:22 0:08:38 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} 1
pass 5181622 2020-06-26 07:02:02 2020-06-27 09:48:36 2020-06-27 10:06:36 0:18:00 0:08:53 0:09:07 smithi master centos 8.1 rados/multimon/{clusters/3 msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_recovery} 2
pass 5181623 2020-06-26 07:02:03 2020-06-27 09:50:34 2020-06-27 10:36:34 0:46:00 0:39:17 0:06:43 smithi master rhel 8.1 rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} 1
pass 5181624 2020-06-26 07:02:04 2020-06-27 09:50:34 2020-06-27 10:18:34 0:28:00 0:16:33 0:11:27 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} 2
pass 5181625 2020-06-26 07:02:05 2020-06-27 09:50:34 2020-06-27 10:28:34 0:38:00 0:26:18 0:11:42 smithi master ubuntu 18.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
pass 5181626 2020-06-26 07:02:06 2020-06-27 09:52:27 2020-06-27 10:56:27 1:04:00 0:15:12 0:48:48 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181627 2020-06-26 07:02:07 2020-06-27 09:52:27 2020-06-27 11:12:28 1:20:01 1:01:25 0:18:36 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
pass 5181628 2020-06-26 07:02:08 2020-06-27 09:52:27 2020-06-27 10:12:26 0:19:59 0:11:17 0:08:42 smithi master ubuntu 18.04 rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} 1
pass 5181629 2020-06-26 07:02:09 2020-06-27 09:52:27 2020-06-27 10:08:27 0:16:00 0:07:31 0:08:29 smithi master centos 8.1 rados/singleton-nomsgr/{all/cache-fs-trunc rados supported-random-distro$/{centos_8}} 1
pass 5181630 2020-06-26 07:02:10 2020-06-27 09:52:27 2020-06-27 10:16:26 0:23:59 0:17:12 0:06:47 smithi master rhel 8.1 rados/singleton/{all/pg-autoscaler msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} 2
pass 5181631 2020-06-26 07:02:11 2020-06-27 09:52:39 2020-06-27 10:20:39 0:28:00 0:10:03 0:17:57 smithi master centos 8.1 rados/cephadm/smoke/{distro/centos_latest fixed-2 start} 2
pass 5181632 2020-06-26 07:02:11 2020-06-27 09:54:24 2020-06-27 10:10:23 0:15:59 0:09:21 0:06:38 smithi master centos 8.1 rados/cephadm/smoke-roleless/{distro/centos_latest start} 2
pass 5181633 2020-06-26 07:02:12 2020-06-27 09:54:24 2020-06-27 10:16:24 0:22:00 0:10:20 0:11:40 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/prometheus} 2
pass 5181634 2020-06-26 07:02:13 2020-06-27 09:54:24 2020-06-27 10:18:23 0:23:59 0:09:10 0:14:49 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
fail 5181635 2020-06-26 07:02:14 2020-06-27 09:54:39 2020-06-27 11:00:40 1:06:01 0:53:06 0:12:55 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

"2020-06-27T10:56:24.825488+0000 mon.a (mon.0) 309 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 5181636 2020-06-26 07:02:15 2020-06-27 09:56:34 2020-06-27 10:40:35 0:44:01 0:37:19 0:06:42 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} 1
pass 5181637 2020-06-26 07:02:16 2020-06-27 09:56:35 2020-06-27 10:36:35 0:40:00 0:33:30 0:06:30 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} 2
pass 5181638 2020-06-26 07:02:17 2020-06-27 09:56:38 2020-06-27 10:16:37 0:19:59 0:09:20 0:10:39 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_radosbench} 1
pass 5181639 2020-06-26 07:02:18 2020-06-27 09:58:41 2020-06-27 10:12:40 0:13:59 0:06:26 0:07:33 smithi master centos 8.1 rados/singleton/{all/pg-removal-interruption msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} 1
pass 5181640 2020-06-26 07:02:18 2020-06-27 09:58:48 2020-06-27 10:38:48 0:40:00 0:28:09 0:11:51 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} 2
pass 5181641 2020-06-26 07:02:19 2020-06-27 10:00:27 2020-06-27 11:20:29 1:20:02 1:08:40 0:11:22 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} 3
pass 5181642 2020-06-26 07:02:20 2020-06-27 10:00:36 2020-06-27 10:46:36 0:46:00 0:34:40 0:11:20 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async start tasks/rados_api_tests} 2
pass 5181643 2020-06-26 07:02:21 2020-06-27 10:00:42 2020-06-27 10:44:42 0:44:00 0:31:00 0:13:00 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181644 2020-06-26 07:02:22 2020-06-27 10:02:16 2020-06-27 10:18:15 0:15:59 0:07:23 0:08:36 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_striper} 2
pass 5181645 2020-06-26 07:02:23 2020-06-27 10:02:16 2020-06-27 10:16:16 0:14:00 0:06:41 0:07:19 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} 1
pass 5181646 2020-06-26 07:02:24 2020-06-27 10:02:18 2020-06-27 10:26:18 0:24:00 0:16:15 0:07:45 smithi master centos 8.1 rados/singleton/{all/radostool msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} 1
pass 5181647 2020-06-26 07:02:24 2020-06-27 10:02:25 2020-06-27 10:20:24 0:17:59 0:10:34 0:07:25 smithi master centos 8.1 rados/singleton-nomsgr/{all/ceph-kvstore-tool rados supported-random-distro$/{centos_8}} 1
pass 5181648 2020-06-26 07:02:25 2020-06-27 10:04:34 2020-06-27 10:46:34 0:42:00 0:26:44 0:15:16 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} 2
pass 5181649 2020-06-26 07:02:26 2020-06-27 10:04:35 2020-06-27 10:48:35 0:44:00 0:32:49 0:11:11 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big} 2
pass 5181650 2020-06-26 07:02:27 2020-06-27 10:04:34 2020-06-27 10:24:34 0:20:00 0:14:21 0:05:39 smithi master rhel 8.0 rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} 2
fail 5181651 2020-06-26 07:02:28 2020-06-27 10:04:40 2020-06-27 10:20:40 0:16:00 0:06:51 0:09:09 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} 1
Failure Reason:

Command failed on smithi003 with status 1: 'cd /home/ubuntu/cephtest/cos && chmod +x *.sh'

pass 5181652 2020-06-26 07:02:29 2020-06-27 10:04:41 2020-06-27 10:24:40 0:19:59 0:13:15 0:06:44 smithi master rhel 8.0 rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} 2
pass 5181653 2020-06-26 07:02:30 2020-06-27 10:06:26 2020-06-27 10:54:26 0:48:00 0:32:04 0:15:56 smithi master rhel 8.1 rados/singleton/{all/random-eio msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} 2
pass 5181654 2020-06-26 07:02:30 2020-06-27 10:06:26 2020-06-27 10:40:26 0:34:00 0:11:08 0:22:52 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181655 2020-06-26 07:02:31 2020-06-27 10:06:26 2020-06-27 10:26:26 0:20:00 0:11:01 0:08:59 smithi master rhel 8.1 rados/multimon/{clusters/6 msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} 2
pass 5181656 2020-06-26 07:02:32 2020-06-27 10:06:26 2020-06-27 10:40:26 0:34:00 0:17:55 0:16:05 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} 2
pass 5181657 2020-06-26 07:02:33 2020-06-27 10:06:33 2020-06-27 10:44:33 0:38:00 0:26:00 0:12:00 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async-v1only start tasks/rados_python} 2
pass 5181658 2020-06-26 07:02:34 2020-06-27 10:06:34 2020-06-27 11:22:35 1:16:01 1:04:57 0:11:04 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} tasks/dashboard} 2
pass 5181659 2020-06-26 07:02:35 2020-06-27 10:06:37 2020-06-27 10:24:36 0:17:59 0:06:59 0:11:00 smithi master ubuntu 18.04 rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} 1
fail 5181660 2020-06-26 07:02:36 2020-06-27 10:06:40 2020-06-27 10:28:40 0:22:00 0:13:25 0:08:35 smithi master centos 8.1 rados/singleton/{all/rebuild-mondb msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} 1
Failure Reason:

"2020-06-27T10:19:45.976409+0000 mon.a (mon.0) 54 : cluster [ERR] Health check failed: 4 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log

pass 5181661 2020-06-26 07:02:36 2020-06-27 10:08:43 2020-06-27 10:30:43 0:22:00 0:12:12 0:09:48 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
pass 5181662 2020-06-26 07:02:37 2020-06-27 10:08:44 2020-06-27 10:30:43 0:21:59 0:15:21 0:06:38 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/workunits} 2
pass 5181663 2020-06-26 07:02:38 2020-06-27 10:10:26 2020-06-27 10:28:25 0:17:59 0:09:22 0:08:37 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/ceph-post-file rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181664 2020-06-26 07:02:39 2020-06-27 10:10:26 2020-06-27 10:50:26 0:40:00 0:29:18 0:10:42 smithi master rhel 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5181665 2020-06-26 07:02:40 2020-06-27 10:10:33 2020-06-27 10:30:32 0:19:59 0:11:46 0:08:13 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/lockdep} 2
pass 5181666 2020-06-26 07:02:41 2020-06-27 10:10:38 2020-06-27 10:38:38 0:28:00 0:18:52 0:09:08 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} 2
pass 5181667 2020-06-26 07:02:42 2020-06-27 10:10:41 2020-06-27 10:38:40 0:27:59 0:19:41 0:08:18 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 5181668 2020-06-26 07:02:43 2020-06-27 10:10:41 2020-06-27 10:50:41 0:40:00 0:32:11 0:07:49 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} 2
pass 5181669 2020-06-26 07:02:43 2020-06-27 10:12:25 2020-06-27 10:54:25 0:42:00 0:33:37 0:08:23 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} 2
pass 5181670 2020-06-26 07:02:44 2020-06-27 10:12:27 2020-06-27 10:40:27 0:28:00 0:13:14 0:14:46 smithi master rhel 8.1 rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} 2
fail 5181671 2020-06-26 07:02:45 2020-06-27 10:12:42 2020-06-27 10:28:41 0:15:59 0:07:07 0:08:52 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/cosbench_64K_write} 1
Failure Reason:

Command failed on smithi028 with status 1: 'cd /home/ubuntu/cephtest/cos && chmod +x *.sh'

pass 5181672 2020-06-26 07:02:46 2020-06-27 10:14:40 2020-06-27 10:58:40 0:44:00 0:22:30 0:21:30 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} 3
pass 5181673 2020-06-26 07:02:47 2020-06-27 10:14:40 2020-06-27 10:52:40 0:38:00 0:30:40 0:07:20 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181674 2020-06-26 07:02:48 2020-06-27 10:14:40 2020-06-27 10:40:39 0:25:59 0:20:46 0:05:13 smithi master rhel 8.1 rados/singleton/{all/recovery-preemption msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} 1
pass 5181675 2020-06-26 07:02:49 2020-06-27 10:14:41 2020-06-27 10:32:41 0:18:00 0:12:19 0:05:41 smithi master rhel 8.1 rados/cephadm/smoke-roleless/{distro/rhel_latest start} 2
pass 5181676 2020-06-26 07:02:49 2020-06-27 10:14:52 2020-06-27 10:52:52 0:38:00 0:28:11 0:09:49 smithi master centos 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} 2
pass 5181677 2020-06-26 07:02:50 2020-06-27 10:16:33 2020-06-27 11:02:33 0:46:00 0:23:19 0:22:41 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} 2
pass 5181678 2020-06-26 07:02:51 2020-06-27 10:16:33 2020-06-27 10:50:33 0:34:00 0:24:11 0:09:49 smithi master centos 8.0 rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async-v1only start tasks/rados_python} 2
pass 5181679 2020-06-26 07:02:52 2020-06-27 10:16:34 2020-06-27 10:38:33 0:21:59 0:13:23 0:08:36 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} 1
pass 5181680 2020-06-26 07:02:53 2020-06-27 10:16:34 2020-06-27 10:32:33 0:15:59 0:07:43 0:08:16 smithi master centos 8.1 rados/singleton/{all/resolve_stuck_peering msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} 2
pass 5181681 2020-06-26 07:02:54 2020-06-27 10:16:39 2020-06-27 10:36:38 0:19:59 0:06:59 0:13:00 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/export-after-evict rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181682 2020-06-26 07:02:55 2020-06-27 10:16:41 2020-06-27 11:20:42 1:04:01 0:15:33 0:48:28 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181683 2020-06-26 07:02:55 2020-06-27 10:18:33 2020-06-27 10:40:33 0:22:00 0:12:38 0:09:22 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} 1
pass 5181684 2020-06-26 07:02:56 2020-06-27 10:18:33 2020-06-27 10:58:33 0:40:00 0:06:56 0:33:04 smithi master ubuntu 18.04 rados/multimon/{clusters/9 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} 3
pass 5181685 2020-06-26 07:02:57 2020-06-27 10:18:35 2020-06-27 10:42:35 0:24:00 0:11:49 0:12:11 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_read} 1
pass 5181686 2020-06-26 07:02:58 2020-06-27 10:18:36 2020-06-27 10:46:36 0:28:00 0:12:48 0:15:12 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} 2
pass 5181687 2020-06-26 07:02:59 2020-06-27 10:18:37 2020-06-27 10:42:37 0:24:00 0:12:02 0:11:58 smithi master ubuntu 18.04 rados/singleton/{all/test-crash msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181688 2020-06-26 07:03:00 2020-06-27 10:18:39 2020-06-27 10:56:39 0:38:00 0:27:14 0:10:46 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} 2
pass 5181689 2020-06-26 07:03:01 2020-06-27 10:20:42 2020-06-27 10:44:42 0:24:00 0:11:34 0:12:26 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} 2
pass 5181690 2020-06-26 07:03:01 2020-06-27 10:20:42 2020-06-27 10:50:42 0:30:00 0:09:37 0:20:23 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} 2
fail 5181691 2020-06-26 07:03:02 2020-06-27 10:20:43 2020-06-27 11:04:43 0:44:00 0:30:51 0:13:09 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} 2
Failure Reason:

"2020-06-27T10:39:46.355714+0000 mon.b (mon.0) 185 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 5181692 2020-06-26 07:03:03 2020-06-27 10:22:42 2020-06-27 11:08:42 0:46:00 0:35:34 0:10:26 smithi master ubuntu 18.04 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181693 2020-06-26 07:03:04 2020-06-27 10:22:42 2020-06-27 11:30:43 1:08:01 1:00:34 0:07:27 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/dashboard} 2
pass 5181694 2020-06-26 07:03:05 2020-06-27 10:22:43 2020-06-27 12:56:46 2:34:03 2:26:32 0:07:31 smithi master centos 8.1 rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} 1
pass 5181695 2020-06-26 07:03:05 2020-06-27 10:24:42 2020-06-27 10:50:41 0:25:59 0:19:49 0:06:10 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
pass 5181696 2020-06-26 07:03:06 2020-06-27 10:24:42 2020-06-27 10:42:41 0:17:59 0:10:34 0:07:25 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5181697 2020-06-26 07:03:07 2020-06-27 10:24:42 2020-06-27 11:18:43 0:54:01 0:48:10 0:05:51 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/rados_api_tests validater/valgrind} 2
pass 5181698 2020-06-26 07:03:08 2020-06-27 10:24:42 2020-06-27 10:50:42 0:26:00 0:18:07 0:07:53 smithi master centos 8.1 rados/singleton/{all/test_envlibrados_for_rocksdb msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} 1
pass 5181699 2020-06-26 07:03:09 2020-06-27 10:24:42 2020-06-27 10:46:42 0:22:00 0:15:38 0:06:22 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} 1
pass 5181700 2020-06-26 07:03:10 2020-06-27 10:24:42 2020-06-27 10:54:42 0:30:00 0:19:18 0:10:42 smithi master rhel 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/sync workloads/pool-create-delete} 2
pass 5181701 2020-06-26 07:03:10 2020-06-27 10:24:43 2020-06-27 11:02:43 0:38:00 0:28:43 0:09:17 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} 2
pass 5181702 2020-06-26 07:03:11 2020-06-27 10:26:35 2020-06-27 10:44:34 0:17:59 0:11:35 0:06:24 smithi master rhel 8.1 rados/singleton-nomsgr/{all/full-tiering rados supported-random-distro$/{rhel_8}} 1
pass 5181703 2020-06-26 07:03:12 2020-06-27 10:26:35 2020-06-27 11:12:36 0:46:01 0:26:36 0:19:25 smithi master ubuntu 18.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181704 2020-06-26 07:03:13 2020-06-27 10:28:41 2020-06-27 10:48:41 0:20:00 0:11:47 0:08:13 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} 1
dead 5181705 2020-06-26 07:03:14 2020-06-27 10:28:42 2020-06-27 22:31:14 12:02:32 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v2only start tasks/rados_api_tests} 2
pass 5181706 2020-06-26 07:03:15 2020-06-27 10:28:42 2020-06-27 11:24:42 0:56:00 0:39:52 0:16:08 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} 3
pass 5181707 2020-06-26 07:03:15 2020-06-27 10:28:42 2020-06-27 12:00:43 1:32:01 1:15:22 0:16:39 smithi master ubuntu 18.04 rados/singleton/{all/thrash-backfill-full msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} 2
pass 5181708 2020-06-26 07:03:16 2020-06-27 10:28:42 2020-06-27 12:14:43 1:46:01 1:36:52 0:09:09 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-radosbench} 2
pass 5181709 2020-06-26 07:03:17 2020-06-27 10:28:43 2020-06-27 10:52:43 0:24:00 0:13:24 0:10:36 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} 2
pass 5181710 2020-06-26 07:03:18 2020-06-27 10:30:53 2020-06-27 10:58:52 0:27:59 0:11:00 0:16:59 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache} 2
pass 5181711 2020-06-26 07:03:19 2020-06-27 10:30:53 2020-06-27 11:04:52 0:33:59 0:26:28 0:07:31 smithi master rhel 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} 2
pass 5181712 2020-06-26 07:03:20 2020-06-27 10:30:53 2020-06-27 11:22:53 0:52:00 0:15:03 0:36:57 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181713 2020-06-26 07:03:21 2020-06-27 10:32:52 2020-06-27 10:56:52 0:24:00 0:11:48 0:12:12 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} 2
pass 5181714 2020-06-26 07:03:21 2020-06-27 10:32:52 2020-06-27 11:24:53 0:52:01 0:41:36 0:10:25 smithi master rhel 8.1 rados/singleton/{all/thrash-eio msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} 2
pass 5181715 2020-06-26 07:03:22 2020-06-27 10:32:52 2020-06-27 11:06:52 0:34:00 0:22:56 0:11:04 smithi master centos 8.1 rados/multimon/{clusters/21 msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/mon_recovery} 3
pass 5181716 2020-06-26 07:03:23 2020-06-27 10:34:41 2020-06-27 10:52:40 0:17:59 0:07:04 0:10:55 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} 1
pass 5181717 2020-06-26 07:03:24 2020-06-27 10:34:41 2020-06-27 10:56:41 0:22:00 0:11:49 0:10:11 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/health-warnings rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181718 2020-06-26 07:03:25 2020-06-27 10:36:37 2020-06-27 10:54:36 0:17:59 0:09:47 0:08:12 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_read} 1
pass 5181719 2020-06-26 07:03:26 2020-06-27 10:36:37 2020-06-27 11:20:37 0:44:00 0:30:32 0:13:28 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} 2
pass 5181720 2020-06-26 07:03:27 2020-06-27 10:36:37 2020-06-27 11:06:37 0:30:00 0:15:12 0:14:48 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/dedup_tier} 2
pass 5181721 2020-06-26 07:03:28 2020-06-27 10:36:37 2020-06-27 10:54:36 0:17:59 0:11:18 0:06:41 smithi master centos 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/failover} 2
pass 5181722 2020-06-26 07:03:28 2020-06-27 10:36:39 2020-06-27 11:12:39 0:36:00 0:28:59 0:07:01 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
pass 5181723 2020-06-26 07:03:29 2020-06-27 10:38:26 2020-06-27 11:06:26 0:28:00 0:21:55 0:06:05 smithi master centos 8.1 rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} 2
pass 5181724 2020-06-26 07:03:30 2020-06-27 10:38:26 2020-06-27 11:16:26 0:38:00 0:31:05 0:06:55 smithi master rhel 8.1 rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} 1
pass 5181725 2020-06-26 07:03:31 2020-06-27 10:38:34 2020-06-27 11:08:34 0:30:00 0:22:53 0:07:07 smithi master centos 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
fail 5181726 2020-06-26 07:03:32 2020-06-27 10:38:39 2020-06-27 11:02:39 0:24:00 0:16:36 0:07:24 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados tasks/rados_cls_all validater/lockdep} 2
Failure Reason:

Command failed (workunit test cls/test_cls_lock.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8415e75682a2aa2453fff8bf9efee6cc55dfca3f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh'

pass 5181727 2020-06-26 07:03:33 2020-06-27 10:38:42 2020-06-27 11:02:41 0:23:59 0:12:37 0:11:22 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} 2
pass 5181728 2020-06-26 07:03:33 2020-06-27 10:38:49 2020-06-27 11:54:50 1:16:01 1:01:08 0:14:53 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
pass 5181729 2020-06-26 07:03:34 2020-06-27 10:40:43 2020-06-27 13:08:46 2:28:03 2:22:43 0:05:20 smithi master rhel 8.1 rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{rhel_8}} 1
pass 5181730 2020-06-26 07:03:35 2020-06-27 10:40:43 2020-06-27 10:58:43 0:18:00 0:08:48 0:09:12 smithi master centos 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_5925} 2
pass 5181731 2020-06-26 07:03:36 2020-06-27 10:40:43 2020-06-27 11:24:43 0:44:00 0:23:07 0:20:53 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} 2
pass 5181732 2020-06-26 07:03:37 2020-06-27 10:40:43 2020-06-27 11:16:43 0:36:00 0:27:14 0:08:46 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181733 2020-06-26 07:03:38 2020-06-27 10:40:43 2020-06-27 11:20:43 0:40:00 0:27:12 0:12:48 smithi master ubuntu 18.04 rados/singleton/{all/thrash_cache_writeback_proxy_none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 2
pass 5181734 2020-06-26 07:03:38 2020-06-27 10:40:43 2020-06-27 11:14:43 0:34:00 0:11:29 0:22:31 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2
pass 5181735 2020-06-26 07:03:39 2020-06-27 10:42:36 2020-06-27 11:00:35 0:17:59 0:09:56 0:08:03 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} 1
pass 5181736 2020-06-26 07:03:40 2020-06-27 10:42:36 2020-06-27 11:38:36 0:56:00 0:21:18 0:34:42 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_7.6} msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} 3
pass 5181737 2020-06-26 07:03:41 2020-06-27 10:42:38 2020-06-27 11:02:37 0:19:59 0:10:33 0:09:26 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/large-omap-object-warnings rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181738 2020-06-26 07:03:42 2020-06-27 10:42:43 2020-06-27 11:18:43 0:36:00 0:19:18 0:16:42 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} 2
pass 5181739 2020-06-26 07:03:43 2020-06-27 10:44:51 2020-06-27 11:00:50 0:15:59 0:10:46 0:05:13 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} 1
pass 5181740 2020-06-26 07:03:44 2020-06-27 10:44:51 2020-06-27 11:02:51 0:18:00 0:09:46 0:08:14 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181741 2020-06-26 07:03:45 2020-06-27 10:44:51 2020-06-27 11:00:50 0:15:59 0:07:10 0:08:49 smithi master ubuntu 18.04 rados/singleton/{all/watch-notify-same-primary msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
fail 5181742 2020-06-26 07:03:46 2020-06-27 10:44:51 2020-06-27 11:22:51 0:38:00 0:30:56 0:07:04 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/rados_api_tests} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi143 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8415e75682a2aa2453fff8bf9efee6cc55dfca3f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

pass 5181743 2020-06-26 07:03:47 2020-06-27 10:46:43 2020-06-27 11:30:43 0:44:00 0:37:02 0:06:58 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2
fail 5181744 2020-06-26 07:03:47 2020-06-27 10:46:43 2020-06-27 21:12:58 10:26:15 10:16:46 0:09:29 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/readwrite} 2
Failure Reason:

"2020-06-27T11:04:40.191518+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4510.0:518 3.b 3.f7ade0ab (undecoded) ondisk+write+known_if_redirected e22) initiated 2020-06-27T11:04:09.738766+0000 currently delayed" in cluster log

pass 5181745 2020-06-26 07:03:48 2020-06-27 10:46:43 2020-06-27 11:10:42 0:23:59 0:06:42 0:17:17 smithi master ubuntu 18.04 rados/multimon/{clusters/3 msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} 2
pass 5181746 2020-06-26 07:03:49 2020-06-27 10:46:43 2020-06-27 11:08:42 0:21:59 0:11:22 0:10:37 smithi master centos 8.1 rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} 2
pass 5181747 2020-06-26 07:03:50 2020-06-27 10:46:43 2020-06-27 11:02:43 0:16:00 0:06:47 0:09:13 smithi master ubuntu 18.04 rados/singleton/{all/admin-socket msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181748 2020-06-26 07:03:51 2020-06-27 10:48:52 2020-06-27 11:12:52 0:24:00 0:14:28 0:09:32 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/fio_4M_rand_write} 1
pass 5181749 2020-06-26 07:03:52 2020-06-27 10:48:52 2020-06-27 11:10:52 0:22:00 0:11:05 0:10:55 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/insights} 2
pass 5181750 2020-06-26 07:03:52 2020-06-27 10:50:44 2020-06-27 11:38:44 0:48:00 0:42:25 0:05:35 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} 2
pass 5181751 2020-06-26 07:03:53 2020-06-27 10:50:44 2020-06-27 11:10:43 0:19:59 0:11:50 0:08:09 smithi master centos 8.0 rados/cephadm/smoke/{distro/centos_8.0 fixed-2 start} 2
pass 5181752 2020-06-26 07:03:54 2020-06-27 10:50:44 2020-06-27 11:28:44 0:38:00 0:31:19 0:06:41 smithi master rhel 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
pass 5181753 2020-06-26 07:03:55 2020-06-27 10:50:44 2020-06-27 11:08:43 0:17:59 0:08:05 0:09:54 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/lazy_omap_stats_output rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181754 2020-06-26 07:03:56 2020-06-27 10:50:44 2020-06-27 11:16:44 0:26:00 0:10:26 0:15:34 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
fail 5181755 2020-06-26 07:03:57 2020-06-27 10:50:44 2020-06-27 11:52:45 1:02:01 0:54:33 0:07:28 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

"2020-06-27T11:48:28.037236+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 5181756 2020-06-26 07:03:58 2020-06-27 10:52:43 2020-06-27 11:10:42 0:17:59 0:11:11 0:06:48 smithi master centos 8.0 rados/cephadm/smoke-roleless/{distro/centos_8.0 start} 2
pass 5181757 2020-06-26 07:03:58 2020-06-27 10:52:43 2020-06-27 11:12:42 0:19:59 0:10:45 0:09:14 smithi master ubuntu 18.04 rados/singleton/{all/deduptool msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181758 2020-06-26 07:03:59 2020-06-27 10:52:43 2020-06-27 11:32:43 0:40:00 0:29:33 0:10:27 smithi master ubuntu 18.04 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_api_tests} 2
pass 5181759 2020-06-26 07:04:00 2020-06-27 10:52:44 2020-06-27 11:20:44 0:28:00 0:20:13 0:07:47 smithi master rhel 8.0 rados/cephadm/upgrade/{1-start 2-start-upgrade 3-wait distro$/{rhel_8.0} fixed-2} 2
pass 5181760 2020-06-26 07:04:01 2020-06-27 10:52:53 2020-06-27 11:38:53 0:46:00 0:39:41 0:06:19 smithi master rhel 8.1 rados/singleton-bluestore/{all/cephtool msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} 1
pass 5181761 2020-06-26 07:04:02 2020-06-27 10:54:42 2020-06-27 11:34:42 0:40:00 0:32:52 0:07:08 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
fail 5181762 2020-06-26 07:04:03 2020-06-27 10:54:43 2020-06-27 11:30:43 0:36:00 0:26:34 0:09:26 smithi master centos 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/dashboard} 2
Failure Reason:

"2020-06-27T11:08:48.217483+0000 osd.2 (osd.2) 3 : cluster [WRN] slow request osd_op(client.4174.0:2 2.1 2.3e6a8311 (undecoded) ondisk+read+known_if_redirected e18) initiated 2020-06-27T11:08:17.359206+0000 currently delayed" in cluster log

pass 5181763 2020-06-26 07:04:03 2020-06-27 10:54:43 2020-06-27 11:12:42 0:17:59 0:09:31 0:08:28 smithi master ubuntu 18.04 rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} 1
pass 5181764 2020-06-26 07:04:04 2020-06-27 10:54:43 2020-06-27 12:50:45 1:56:02 1:49:06 0:06:56 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} 2
pass 5181765 2020-06-26 07:04:05 2020-06-27 10:54:42 2020-06-27 11:08:42 0:14:00 0:07:47 0:06:13 smithi master centos 8.1 rados/singleton/{all/divergent_priors msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} 1
pass 5181766 2020-06-26 07:04:06 2020-06-27 10:54:44 2020-06-27 11:10:43 0:15:59 0:10:20 0:05:39 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_adoption} 1
pass 5181767 2020-06-26 07:04:07 2020-06-27 10:56:44 2020-06-27 11:52:45 0:56:01 0:47:12 0:08:49 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} 1
pass 5181768 2020-06-26 07:04:08 2020-06-27 10:56:44 2020-06-27 11:16:44 0:20:00 0:11:29 0:08:31 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} 1
pass 5181769 2020-06-26 07:04:09 2020-06-27 10:56:44 2020-06-27 11:26:44 0:30:00 0:19:02 0:10:58 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} 2
pass 5181770 2020-06-26 07:04:09 2020-06-27 10:56:53 2020-06-27 11:38:53 0:42:00 0:25:09 0:16:51 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181771 2020-06-26 07:04:10 2020-06-27 10:58:51 2020-06-27 11:50:52 0:52:01 0:34:26 0:17:35 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} 3
pass 5181772 2020-06-26 07:04:11 2020-06-27 10:58:51 2020-06-27 11:24:51 0:26:00 0:15:08 0:10:52 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} 2
pass 5181773 2020-06-26 07:04:12 2020-06-27 10:58:51 2020-06-27 11:34:51 0:36:00 0:25:05 0:10:55 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} 2
pass 5181774 2020-06-26 07:04:13 2020-06-27 10:58:53 2020-06-27 11:20:53 0:22:00 0:11:14 0:10:46 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/librados_hello_world rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181775 2020-06-26 07:04:14 2020-06-27 11:00:53 2020-06-27 11:24:53 0:24:00 0:16:57 0:07:03 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect} 2
pass 5181776 2020-06-26 07:04:15 2020-06-27 11:00:53 2020-06-27 11:12:52 0:11:59 0:06:03 0:05:56 smithi master centos 8.1 rados/multimon/{clusters/6 msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} 2
pass 5181777 2020-06-26 07:04:15 2020-06-27 11:00:53 2020-06-27 11:18:53 0:18:00 0:08:49 0:09:11 smithi master ubuntu 18.04 rados/singleton/{all/divergent_priors2 msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181778 2020-06-26 07:04:16 2020-06-27 11:00:53 2020-06-27 11:32:53 0:32:00 0:21:39 0:10:21 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm} 1
pass 5181779 2020-06-26 07:04:17 2020-06-27 11:02:51 2020-06-27 11:46:51 0:44:00 0:27:10 0:16:50 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/module_selftest} 2
pass 5181780 2020-06-26 07:04:18 2020-06-27 11:02:51 2020-06-27 11:32:51 0:30:00 0:18:04 0:11:56 smithi master centos 8.1 rados/cephadm/smoke/{distro/centos_latest fixed-2 start} 2
pass 5181781 2020-06-26 07:04:19 2020-06-27 11:02:51 2020-06-27 11:22:50 0:19:59 0:11:53 0:08:06 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} 2
pass 5181782 2020-06-26 07:04:20 2020-06-27 11:02:51 2020-06-27 11:22:50 0:19:59 0:09:41 0:10:18 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} 1
pass 5181783 2020-06-26 07:04:20 2020-06-27 11:02:51 2020-06-27 11:22:50 0:19:59 0:12:30 0:07:29 smithi master rhel 8.1 rados/singleton/{all/dump-stuck msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} 1
pass 5181784 2020-06-26 07:04:21 2020-06-27 11:02:51 2020-06-27 11:22:51 0:20:00 0:09:14 0:10:46 smithi master centos 8.1 rados/cephadm/smoke-roleless/{distro/centos_latest start} 2
pass 5181785 2020-06-26 07:04:22 2020-06-27 11:02:52 2020-06-27 11:58:53 0:56:01 0:30:01 0:26:00 smithi master rhel 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 5181786 2020-06-26 07:04:23 2020-06-27 11:05:00 2020-06-27 11:23:00 0:18:00 0:10:02 0:07:58 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/lockdep} 2
pass 5181787 2020-06-26 07:04:24 2020-06-27 11:05:00 2020-06-27 11:31:00 0:26:00 0:15:49 0:10:11 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/msgr rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181788 2020-06-26 07:04:25 2020-06-27 11:06:43 2020-06-27 12:22:44 1:16:01 1:04:30 0:11:31 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_osdmap_prune} 2
pass 5181789 2020-06-26 07:04:26 2020-06-27 11:06:44 2020-06-27 11:56:44 0:50:00 0:35:21 0:14:39 smithi master ubuntu 18.04 rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root msgr/async start tasks/rados_api_tests} 2
pass 5181790 2020-06-26 07:04:27 2020-06-27 11:06:43 2020-06-27 11:40:43 0:34:00 0:25:55 0:08:05 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181791 2020-06-26 07:04:27 2020-06-27 11:06:54 2020-06-27 11:40:54 0:34:00 0:26:37 0:07:23 smithi master centos 8.1 rados/singleton/{all/ec-lost-unfound msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} 1
fail 5181792 2020-06-26 07:04:28 2020-06-27 11:08:51 2020-06-27 11:56:52 0:48:01 0:40:32 0:07:29 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 5181793 2020-06-26 07:04:29 2020-06-27 11:08:51 2020-06-27 11:30:51 0:22:00 0:14:31 0:07:29 smithi master rhel 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/scrub_test} 2
pass 5181794 2020-06-26 07:04:30 2020-06-27 11:08:52 2020-06-27 11:46:52 0:38:00 0:24:44 0:13:16 smithi master ubuntu 18.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
pass 5181795 2020-06-26 07:04:31 2020-06-27 11:08:52 2020-06-27 11:32:51 0:23:59 0:10:10 0:13:49 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181796 2020-06-26 07:04:31 2020-06-27 11:08:51 2020-06-27 11:20:51 0:12:00 0:06:42 0:05:18 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm_repos} 1
pass 5181797 2020-06-26 07:04:32 2020-06-27 11:10:59 2020-06-27 12:23:00 1:12:01 1:04:33 0:07:28 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/dashboard} 2
pass 5181798 2020-06-26 07:04:33 2020-06-27 11:10:59 2020-06-27 11:32:59 0:22:00 0:16:14 0:05:46 smithi master rhel 8.1 rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} 1
pass 5181799 2020-06-26 07:04:34 2020-06-27 11:10:59 2020-06-27 11:34:59 0:24:00 0:18:03 0:05:57 smithi master centos 8.1 rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} 1
pass 5181800 2020-06-26 07:04:35 2020-06-27 11:10:59 2020-06-27 11:46:59 0:36:00 0:28:44 0:07:16 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} 2
pass 5181801 2020-06-26 07:04:36 2020-06-27 11:10:59 2020-06-27 13:11:02 2:00:03 1:48:36 0:11:27 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_7.6} msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} 3
pass 5181802 2020-06-26 07:04:37 2020-06-27 11:12:36 2020-06-27 11:34:36 0:22:00 0:11:11 0:10:49 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} 1
pass 5181803 2020-06-26 07:04:38 2020-06-27 11:12:36 2020-06-27 11:34:36 0:22:00 0:13:51 0:08:09 smithi master rhel 8.1 rados/singleton/{all/erasure-code-nonregression msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} 1
pass 5181804 2020-06-26 07:04:38 2020-06-27 11:12:37 2020-06-27 11:36:37 0:24:00 0:14:32 0:09:28 smithi master rhel 8.0 rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 start} 2
pass 5181805 2020-06-26 07:04:39 2020-06-27 11:12:40 2020-06-27 11:38:40 0:26:00 0:10:12 0:15:48 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunk-promote-flush} 2
pass 5181806 2020-06-26 07:04:40 2020-06-27 11:12:43 2020-06-27 11:44:43 0:32:00 0:19:04 0:12:56 smithi master rhel 8.1 rados/multimon/{clusters/9 msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_recovery} 3
pass 5181807 2020-06-26 07:04:41 2020-06-27 11:12:44 2020-06-27 11:40:43 0:27:59 0:14:11 0:13:48 smithi master rhel 8.0 rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} 2
pass 5181808 2020-06-26 07:04:42 2020-06-27 11:12:53 2020-06-27 11:52:53 0:40:00 0:26:52 0:13:08 smithi master rhel 8.1 rados/singleton-nomsgr/{all/multi-backfill-reject rados supported-random-distro$/{rhel_8}} 2
pass 5181809 2020-06-26 07:04:43 2020-06-27 11:12:54 2020-06-27 11:50:54 0:38:00 0:30:11 0:07:49 smithi master centos 8.1 rados/singleton/{all/lost-unfound-delete msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} 1
pass 5181810 2020-06-26 07:04:43 2020-06-27 11:15:00 2020-06-27 11:41:00 0:26:00 0:13:17 0:12:43 smithi master ubuntu 18.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/progress} 2
fail 5181811 2020-06-26 07:04:44 2020-06-27 11:16:44 2020-06-27 13:04:46 1:48:02 1:36:50 0:11:12 smithi master ubuntu 18.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} 1
Failure Reason:

Command failed (workunit test osd/osd-backfill-stats.sh) on smithi200 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8415e75682a2aa2453fff8bf9efee6cc55dfca3f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-stats.sh'

pass 5181812 2020-06-26 07:04:45 2020-06-27 11:16:44 2020-06-27 11:50:44 0:34:00 0:24:13 0:09:47 smithi master centos 8.0 rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged msgr/async start tasks/rados_python} 2
pass 5181813 2020-06-26 07:04:46 2020-06-27 11:16:45 2020-06-27 11:50:45 0:34:00 0:22:33 0:11:27 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/set-chunks-read} 2
pass 5181814 2020-06-26 07:04:47 2020-06-27 11:16:45 2020-06-27 11:42:45 0:26:00 0:12:30 0:13:30 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} 1
pass 5181815 2020-06-26 07:04:48 2020-06-27 11:18:37 2020-06-27 11:38:36 0:19:59 0:10:24 0:09:35 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} 1
fail 5181816 2020-06-26 07:04:48 2020-06-27 11:18:44 2020-06-27 11:52:44 0:34:00 0:26:29 0:07:31 smithi master rhel 8.1 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
Failure Reason:

Command failed on smithi029 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json'

dead 5181817 2020-06-26 07:04:49 2020-06-27 11:18:44 2020-06-27 11:28:43 0:09:59 0:01:35 0:08:24 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

{'Failure object was': {'smithi008.front.sepia.ceph.com': {'msg': 'Failed to download packages: Cannot download Packages/boost-random-1.66.0-6.el8.x86_64.rpm: All mirrors were tried', 'results': [], 'invocation': {'module_args': {'name': ['dnf-utils', 'sysstat', 'libedit', 'boost-thread', 'xfsprogs', 'gdisk', 'parted', 'libgcrypt', 'fuse-libs', 'openssl', 'libuuid', 'podman', 'podman-docker', 'attr', 'ant', 'lsof', 'gettext', 'bc', 'xfsdump', 'blktrace', 'usbredir', 'libev-devel', 'valgrind', 'nfs-utils', 'ncurses-devel', 'gcc', 'git', 'python3-nose', 'python3-virtualenv', 'genisoimage', 'qemu-img', 'qemu-kvm-core', 'qemu-kvm-block-rbd', 'libacl-devel', 'dbench', 'autoconf', 'gdb', 'iozone'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '/')"}

pass 5181818 2020-06-26 07:04:50 2020-06-27 11:18:54 2020-06-27 12:00:54 0:42:00 0:34:59 0:07:01 smithi master rhel 8.1 rados/singleton/{all/lost-unfound msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} 1
pass 5181819 2020-06-26 07:04:51 2020-06-27 11:20:46 2020-06-27 11:56:46 0:36:00 0:24:54 0:11:06 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} 2
pass 5181820 2020-06-26 07:04:52 2020-06-27 11:20:47 2020-06-27 11:42:46 0:21:59 0:14:00 0:07:59 smithi master rhel 8.1 rados/cephadm/smoke/{distro/rhel_latest fixed-2 start} 2
pass 5181821 2020-06-26 07:04:53 2020-06-27 11:20:47 2020-06-27 12:02:46 0:41:59 0:32:27 0:09:32 smithi master rhel 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_workunits} 2
pass 5181822 2020-06-26 07:04:54 2020-06-27 11:20:47 2020-06-27 11:56:46 0:35:59 0:29:31 0:06:28 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181823 2020-06-26 07:04:55 2020-06-27 11:20:47 2020-06-27 12:06:47 0:46:00 0:22:11 0:23:49 smithi master ubuntu 18.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181824 2020-06-26 07:04:55 2020-06-27 11:20:52 2020-06-27 11:52:52 0:32:00 0:22:47 0:09:13 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} 2
pass 5181825 2020-06-26 07:04:56 2020-06-27 11:20:54 2020-06-27 11:40:54 0:20:00 0:12:23 0:07:37 smithi master rhel 8.1 rados/cephadm/smoke-roleless/{distro/rhel_latest start} 2
pass 5181826 2020-06-26 07:04:57 2020-06-27 11:22:52 2020-06-27 11:48:52 0:26:00 0:16:50 0:09:10 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/osd_stale_reads rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181827 2020-06-26 07:04:58 2020-06-27 11:22:52 2020-06-27 11:36:51 0:13:59 0:06:45 0:07:14 smithi master centos 8.1 rados/singleton/{all/max-pg-per-osd.from-mon msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} 1
pass 5181828 2020-06-26 07:04:59 2020-06-27 11:22:52 2020-06-27 11:56:52 0:34:00 0:26:20 0:07:40 smithi master rhel 8.1 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} 2
pass 5181829 2020-06-26 07:05:00 2020-06-27 11:22:52 2020-06-27 12:02:52 0:40:00 0:35:12 0:04:48 smithi master centos 8.1 rados/singleton-bluestore/{all/cephtool msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} 1
pass 5181830 2020-06-26 07:05:01 2020-06-27 11:22:52 2020-06-27 12:36:53 1:14:01 1:01:41 0:12:20 smithi master ubuntu 18.04 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/dashboard} 2
pass 5181831 2020-06-26 07:05:01 2020-06-27 11:22:53 2020-06-27 12:04:53 0:42:00 0:29:32 0:12:28 smithi master rhel 8.1 rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} 1
pass 5181832 2020-06-26 07:05:02 2020-06-27 11:22:55 2020-06-27 12:00:55 0:38:00 0:32:19 0:05:41 smithi master centos 8.1 rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root msgr/async-v1only start tasks/rados_api_tests} 2
pass 5181833 2020-06-26 07:05:03 2020-06-27 11:23:01 2020-06-27 11:53:01 0:30:00 0:18:11 0:11:49 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{centos_7.6} msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} 3
pass 5181834 2020-06-26 07:05:04 2020-06-27 11:24:59 2020-06-27 11:44:58 0:19:59 0:09:40 0:10:19 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-comp openstack settings/optimized ubuntu_latest workloads/radosbench_4M_write} 1
pass 5181835 2020-06-26 07:05:05 2020-06-27 11:24:59 2020-06-27 11:48:58 0:23:59 0:11:22 0:12:37 smithi master rhel 8.1 rados/multimon/{clusters/21 msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} 3
pass 5181836 2020-06-26 07:05:06 2020-06-27 11:24:59 2020-06-27 11:46:58 0:21:59 0:12:02 0:09:57 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_adoption} 1
pass 5181837 2020-06-26 07:05:07 2020-06-27 11:24:59 2020-06-27 11:44:59 0:20:00 0:10:57 0:09:03 smithi master centos 8.1 rados/singleton/{all/max-pg-per-osd.from-primary msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} 1
pass 5181838 2020-06-26 07:05:07 2020-06-27 11:24:59 2020-06-27 11:58:59 0:34:00 0:24:27 0:09:33 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-localized} 2
pass 5181839 2020-06-26 07:05:08 2020-06-27 11:27:02 2020-06-27 12:03:02 0:36:00 0:26:20 0:09:40 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
pass 5181840 2020-06-26 07:05:09 2020-06-27 11:29:01 2020-06-27 11:51:01 0:22:00 0:14:31 0:07:29 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/prometheus} 2
fail 5181841 2020-06-26 07:05:10 2020-06-27 11:29:01 2020-06-27 11:59:01 0:30:00 0:12:17 0:17:43 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 start} 2
Failure Reason:

'/home/ubuntu/cephtest/archive/syslog/misc.log:2020-06-27T11:55:28.401501+00:00 smithi031 bash[7031]: cephadm 2020-06-27T11:55:27.149203+0000 mgr.x (mgr.34109) 141 : cephadm [ERR] cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploy daemon prometheus.a ... ' in syslog

pass 5181842 2020-06-26 07:05:11 2020-06-27 11:30:59 2020-06-27 11:52:59 0:22:00 0:15:52 0:06:08 smithi master rhel 8.1 rados/singleton/{all/max-pg-per-osd.from-replica msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} 1
pass 5181843 2020-06-26 07:05:12 2020-06-27 11:30:59 2020-06-27 11:52:59 0:22:00 0:14:46 0:07:14 smithi master rhel 8.1 rados/singleton-nomsgr/{all/pool-access rados supported-random-distro$/{rhel_8}} 1
pass 5181844 2020-06-26 07:05:12 2020-06-27 11:30:59 2020-06-27 12:02:59 0:32:00 0:20:20 0:11:40 smithi master ubuntu 18.04 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} 2
pass 5181845 2020-06-26 07:05:13 2020-06-27 11:30:59 2020-06-27 11:54:59 0:24:00 0:11:17 0:12:43 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} 2
pass 5181846 2020-06-26 07:05:14 2020-06-27 11:31:01 2020-06-27 12:01:01 0:30:00 0:21:31 0:08:29 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects} 2
pass 5181847 2020-06-26 07:05:15 2020-06-27 11:33:00 2020-06-27 12:17:00 0:44:00 0:27:33 0:16:27 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
fail 5181848 2020-06-26 07:05:16 2020-06-27 11:33:00 2020-06-27 12:21:00 0:48:00 0:40:22 0:07:38 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/lockdep} 2
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

pass 5181849 2020-06-26 07:05:17 2020-06-27 11:33:00 2020-06-27 12:07:00 0:34:00 0:23:03 0:10:57 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-low-osd-mem-target openstack settings/optimized ubuntu_latest workloads/radosbench_omap_write} 1
pass 5181850 2020-06-26 07:05:17 2020-06-27 11:33:00 2020-06-27 12:09:00 0:36:00 0:28:43 0:07:17 smithi master rhel 8.0 rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged msgr/async-v2only start tasks/rados_python} 2
pass 5181851 2020-06-26 07:05:18 2020-06-27 11:33:00 2020-06-27 12:17:00 0:44:00 0:33:13 0:10:47 smithi master rhel 8.1 rados/monthrash/{ceph clusters/9-mons msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/sync workloads/snaps-few-objects} 2
pass 5181852 2020-06-26 07:05:19 2020-06-27 11:34:53 2020-06-27 12:14:53 0:40:00 0:32:37 0:07:23 smithi master rhel 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181853 2020-06-26 07:05:20 2020-06-27 11:34:53 2020-06-27 12:12:53 0:38:00 0:14:55 0:23:05 smithi master rhel 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181854 2020-06-26 07:05:21 2020-06-27 11:34:53 2020-06-27 11:56:53 0:22:00 0:11:34 0:10:26 smithi master ubuntu 18.04 rados/singleton/{all/mon-auth-caps msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181855 2020-06-26 07:05:21 2020-06-27 11:34:54 2020-06-27 13:02:55 1:28:01 1:19:43 0:08:18 smithi master centos 8.1 rados/standalone/{supported-random-distro$/{centos_8} workloads/scrub} 1
pass 5181856 2020-06-26 07:05:22 2020-06-27 11:35:00 2020-06-27 11:57:00 0:22:00 0:14:51 0:07:09 smithi master centos 8.1 rados/cephadm/workunits/{distro/centos_latest task/test_cephadm} 1
pass 5181857 2020-06-26 07:05:23 2020-06-27 11:36:54 2020-06-27 12:10:53 0:33:59 0:26:21 0:07:38 smithi master rhel 8.1 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} 2
pass 5181858 2020-06-26 07:05:24 2020-06-27 11:36:54 2020-06-27 12:14:54 0:38:00 0:26:58 0:11:02 smithi master ubuntu 18.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 2
fail 5181859 2020-06-26 07:05:25 2020-06-27 11:38:54 2020-06-27 12:02:53 0:23:59 0:13:00 0:10:59 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 start} 2
Failure Reason:

'/home/ubuntu/cephtest/archive/syslog/misc.log:2020-06-27T11:58:45.905003+00:00 smithi080 bash[11909]: cephadm 2020-06-27T11:58:45.340174+0000 mgr.x (mgr.34109) 119 : cephadm [ERR] cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploy daemon prometheus.a ... ' in syslog

pass 5181860 2020-06-26 07:05:26 2020-06-27 11:38:54 2020-06-27 12:02:53 0:23:59 0:17:02 0:06:57 smithi master rhel 8.1 rados/singleton/{all/mon-config-key-caps msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} 1
pass 5181861 2020-06-26 07:05:26 2020-06-27 11:38:54 2020-06-27 12:14:54 0:36:00 0:27:41 0:08:19 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/recovery-unfound-found rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181862 2020-06-26 07:05:27 2020-06-27 11:38:54 2020-06-27 12:28:54 0:50:00 0:38:41 0:11:19 smithi master centos 7.6 rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{centos_7.6} msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} 3
pass 5181863 2020-06-26 07:05:28 2020-06-27 11:38:55 2020-06-27 12:56:56 1:18:01 1:04:15 0:13:46 smithi master rhel 8.1 rados/dashboard/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} 2
pass 5181864 2020-06-26 07:05:29 2020-06-27 11:38:55 2020-06-27 16:37:02 4:58:07 4:44:08 0:13:59 smithi master rhel 8.1 rados/objectstore/{backends/objectstore supported-random-distro$/{rhel_8}} 1
fail 5181865 2020-06-26 07:05:30 2020-06-27 11:41:00 2020-06-27 12:05:00 0:24:00 0:11:48 0:12:12 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} 2
Failure Reason:

'/home/ubuntu/cephtest/archive/syslog/misc.log:2020-06-27T11:57:25.488213+00:00 smithi073 bash[10606]: debug 2020-06-27T11:57:25.484+0000 7f3f4d437700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploy daemon alertmanager.smithi073 ... ' in syslog

pass 5181866 2020-06-26 07:05:31 2020-06-27 11:41:00 2020-06-27 12:03:00 0:22:00 0:11:43 0:10:17 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-stupid openstack settings/optimized ubuntu_latest workloads/sample_fio} 1
pass 5181867 2020-06-26 07:05:32 2020-06-27 11:41:00 2020-06-27 11:59:00 0:18:00 0:10:42 0:07:18 smithi master rhel 8.1 rados/multimon/{clusters/3 msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} 2
pass 5181868 2020-06-26 07:05:32 2020-06-27 11:41:00 2020-06-27 12:13:00 0:32:00 0:25:12 0:06:48 smithi master centos 8.1 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} 2
pass 5181869 2020-06-26 07:05:33 2020-06-27 11:41:01 2020-06-27 12:07:01 0:26:00 0:14:58 0:11:02 smithi master rhel 8.1 rados/mgr/{clusters/{2-node-mgr} debug/mgr objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/workunits} 2
pass 5181870 2020-06-26 07:05:34 2020-06-27 11:43:02 2020-06-27 12:11:02 0:28:00 0:16:04 0:11:56 smithi master centos 8.1 rados/basic/{ceph clusters/{fixed-2 openstack} msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_python} 2
dead 5181871 2020-06-26 07:05:35 2020-06-27 11:43:02 2020-06-27 23:45:31 12:02:29 smithi master rhel 8.1 rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root msgr/async start tasks/rados_api_tests} 2
pass 5181872 2020-06-26 07:05:36 2020-06-27 11:45:01 2020-06-27 12:11:01 0:26:00 0:20:16 0:05:44 smithi master rhel 8.1 rados/singleton/{all/mon-config-keys msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} 1
pass 5181873 2020-06-26 07:05:37 2020-06-27 11:45:01 2020-06-27 12:01:01 0:16:00 0:06:46 0:09:14 smithi master ubuntu 18.04 rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_cephadm_repos} 1
pass 5181874 2020-06-26 07:05:38 2020-06-27 11:45:01 2020-06-27 12:13:01 0:28:00 0:14:40 0:13:20 smithi master ubuntu 18.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 5181875 2020-06-26 07:05:39 2020-06-27 11:47:08 2020-06-27 12:21:08 0:34:00 0:24:52 0:09:08 smithi master centos 8.1 rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind} 2
pass 5181876 2020-06-26 07:05:39 2020-06-27 11:47:09 2020-06-27 12:27:09 0:40:00 0:27:26 0:12:34 smithi master ubuntu 18.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/crush-compat msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects} 2
pass 5181877 2020-06-26 07:05:40 2020-06-27 11:47:09 2020-06-27 12:05:08 0:17:59 0:10:56 0:07:03 smithi master centos 8.1 rados/singleton/{all/mon-config msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} 1
pass 5181878 2020-06-26 07:05:41 2020-06-27 11:47:08 2020-06-27 12:13:08 0:26:00 0:12:37 0:13:23 smithi master ubuntu 18.04 rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 start} 2
pass 5181879 2020-06-26 07:05:42 2020-06-27 11:49:09 2020-06-27 12:09:08 0:19:59 0:11:12 0:08:47 smithi master ubuntu 18.04 rados/singleton-nomsgr/{all/version-number-sanity rados supported-random-distro$/{ubuntu_latest}} 1
pass 5181880 2020-06-26 07:05:43 2020-06-27 11:49:09 2020-06-27 12:19:09 0:30:00 0:20:09 0:09:51 smithi master centos 8.1 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 5181881 2020-06-26 07:05:44 2020-06-27 11:51:02 2020-06-27 12:11:02 0:20:00 0:11:02 0:08:58 smithi master centos 8.1 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 5181882 2020-06-26 07:05:45 2020-06-27 11:51:02 2020-06-27 12:11:02 0:20:00 0:10:09 0:09:51 smithi master ubuntu 18.04 rados/perf/{ceph objectstore/bluestore-basic-min-osd-mem-target openstack settings/optimized ubuntu_latest workloads/sample_radosbench} 1
pass 5181883 2020-06-26 07:05:45 2020-06-27 11:51:02 2020-06-27 12:13:02 0:22:00 0:14:54 0:07:06 smithi master centos 8.1 rados/monthrash/{ceph clusters/3-mons msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete} 2
pass 5181884 2020-06-26 07:05:46 2020-06-27 11:51:02 2020-06-27 12:27:02 0:36:00 0:27:56 0:08:04 smithi master centos 8.1 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 5181885 2020-06-26 07:05:47 2020-06-27 11:51:03 2020-06-27 12:15:02 0:23:59 0:11:13 0:12:46 smithi master ubuntu 18.04 rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} 2