Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7251140 2023-04-25 14:17:32 2023-04-25 14:19:45 2023-04-25 14:49:58 0:30:13 0:19:05 0:11:08 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} 1
fail 7251141 2023-04-25 14:17:33 2023-04-25 14:19:55 2023-04-25 14:39:30 0:19:35 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/radosbench} 3
Failure Reason:

Cannot connect to remote host smithi103

pass 7251142 2023-04-25 14:17:34 2023-04-25 14:50:46 1339 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} 2
pass 7251143 2023-04-25 14:17:35 2023-04-25 14:21:46 2023-04-25 20:14:50 5:53:04 4:54:51 0:58:13 smithi main ubuntu 20.04 rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} 1
pass 7251144 2023-04-25 14:17:36 2023-04-25 14:25:47 2023-04-25 14:54:56 0:29:09 0:23:16 0:05:53 smithi main rhel 8.4 rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_recovery} 3
pass 7251145 2023-04-25 14:17:36 2023-04-25 14:25:47 2023-04-25 14:48:21 0:22:34 0:12:10 0:10:24 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} 1
pass 7251146 2023-04-25 14:17:37 2023-04-25 14:26:38 2023-04-25 16:34:41 2:08:03 1:37:29 0:30:34 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_pids_limit} 1
pass 7251147 2023-04-25 14:17:38 2023-04-25 14:26:38 2023-04-25 16:36:29 2:09:51 1:41:08 0:28:43 smithi main centos 8.stream rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} 1
pass 7251148 2023-04-25 14:17:39 2023-04-25 14:27:19 2023-04-25 14:51:10 0:23:51 0:16:45 0:07:06 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7251149 2023-04-25 14:17:40 2023-04-25 14:27:29 2023-04-25 15:50:28 1:22:59 1:08:42 0:14:17 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} 2
fail 7251150 2023-04-25 14:17:40 2023-04-25 15:20:29 2023-04-25 15:47:39 0:27:10 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

Command failed on smithi123 with status 1: 'sudo yum install -y kernel'

pass 7251151 2023-04-25 14:17:41 2023-04-25 15:20:39 2023-04-25 16:05:30 0:44:51 0:31:41 0:13:10 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/connectivity} 2
dead 7251152 2023-04-25 14:17:42 2023-04-25 15:23:30 2023-04-25 16:18:27 0:54:57 0:26:02 0:28:55 smithi main centos 8.stream rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
Failure Reason:

{'smithi179.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

dead 7251153 2023-04-25 14:17:43 2023-04-25 15:23:30 2023-04-25 16:19:35 0:56:05 0:26:17 0:29:48 smithi main centos 8.stream rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_osdmap_prune} 2
Failure Reason:

{'smithi188.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}, 'smithi006.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

dead 7251154 2023-04-25 14:17:44 2023-04-25 15:24:51 2023-04-25 16:20:22 0:55:31 0:25:53 0:29:38 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_api_tests} 2
Failure Reason:

{'smithi096.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}, 'smithi114.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

dead 7251155 2023-04-25 14:17:45 2023-04-25 15:25:11 2023-04-25 16:26:31 1:01:20 0:30:27 0:30:53 smithi main centos 8.stream rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/prometheus} 2
Failure Reason:

{'smithi100.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}, 'smithi037.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

dead 7251156 2023-04-25 14:17:46 2023-04-25 15:27:22 2023-04-25 16:25:02 0:57:40 0:26:00 0:31:40 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} 2
Failure Reason:

{'smithi183.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}, 'smithi063.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

dead 7251157 2023-04-25 14:17:46 2023-04-25 15:29:42 2023-04-25 16:27:11 0:57:29 0:25:51 0:31:38 smithi main centos 8.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
Failure Reason:

{'smithi087.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}, 'smithi090.front.sepia.ceph.com': {'reason': "Could not find or access '/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/yum/baseurls_not_mirrorlists.yml' on the Ansible Controller."}}

pass 7251158 2023-04-25 14:17:47 2023-04-25 15:31:03 2023-04-25 16:07:44 0:36:41 0:18:37 0:18:04 smithi main ubuntu 20.04 rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7251159 2023-04-25 14:17:48 2023-04-25 15:37:54 2023-04-25 16:00:03 0:22:09 0:09:14 0:12:55 smithi main ubuntu 20.04 rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251160 2023-04-25 14:17:49 2023-04-25 15:40:05 2023-04-25 16:30:43 0:50:38 0:36:30 0:14:08 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/rbd_cls} 3
pass 7251161 2023-04-25 14:17:50 2023-04-25 15:42:06 2023-04-25 16:13:54 0:31:48 0:20:14 0:11:34 smithi main rhel 8.4 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251162 2023-04-25 14:17:50 2023-04-25 15:46:27 2023-04-25 18:03:45 2:17:18 1:48:01 0:29:17 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} 1
dead 7251163 2023-04-25 14:17:51 2023-04-25 15:46:37 2023-04-25 16:50:35 1:03:58 0:33:59 0:29:59 smithi main centos 8.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
Failure Reason:

{'smithi129.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'CentOS-PowerTools': Yum repo downloading error: Downloading error(s): repodata/e830a7a4e881ef24680d161802ae07874dd447031dd12d47f2d3d4a911245522-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/38c2c78c3f89e4d347f07263805d33d8ae17df60ebab2e8a259218c57bcae2fb-comps-PowerTools.x86_64.xml - Cannot download, all mirrors were already tried without success", 'rc': 1, 'results': []}}

pass 7251164 2023-04-25 14:17:52 2023-04-25 15:47:18 2023-04-25 16:15:23 0:28:05 0:17:52 0:10:13 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect} 2
pass 7251165 2023-04-25 14:17:53 2023-04-25 15:47:28 2023-04-25 16:19:17 0:31:49 0:21:48 0:10:01 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} 2
fail 7251166 2023-04-25 14:17:54 2023-04-25 15:47:39 2023-04-25 16:03:42 0:16:03 0:06:24 0:09:39 smithi main ubuntu 20.04 rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/master} 1
Failure Reason:

Command failed on smithi123 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull'

pass 7251167 2023-04-25 14:17:55 2023-04-25 15:47:49 2023-04-25 16:15:27 0:27:38 0:21:20 0:06:18 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7251168 2023-04-25 14:17:55 2023-04-25 15:47:59 2023-04-25 16:40:50 0:52:51 0:46:32 0:06:19 smithi main rhel 8.4 rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251169 2023-04-25 14:17:56 2023-04-25 15:48:00 2023-04-25 17:49:50 2:01:50 1:32:32 0:29:18 smithi main centos 8.stream rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} 2
pass 7251170 2023-04-25 14:17:57 2023-04-25 15:48:20 2023-04-25 18:16:57 2:28:37 1:57:07 0:31:30 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7251171 2023-04-25 14:17:58 2023-04-25 15:49:41 2023-04-25 16:28:26 0:38:45 0:30:40 0:08:05 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7251172 2023-04-25 14:17:59 2023-04-25 15:50:31 2023-04-25 16:38:34 0:48:03 0:33:41 0:14:22 smithi main ubuntu 20.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 2
pass 7251173 2023-04-25 14:17:59 2023-04-25 15:52:52 2023-04-25 17:52:01 1:59:09 1:30:30 0:28:39 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} 1
pass 7251174 2023-04-25 14:18:00 2023-04-25 15:52:52 2023-04-25 16:17:32 0:24:40 0:14:16 0:10:24 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} 2
pass 7251175 2023-04-25 14:18:01 2023-04-25 15:53:22 2023-04-25 16:23:08 0:29:46 0:18:49 0:10:57 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7251176 2023-04-25 14:18:02 2023-04-25 15:55:43 2023-04-25 16:19:11 0:23:28 0:11:12 0:12:16 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} 1
pass 7251177 2023-04-25 14:18:03 2023-04-25 15:57:24 2023-04-25 18:23:38 2:26:14 1:54:20 0:31:54 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251178 2023-04-25 14:18:03 2023-04-25 16:00:15 2023-04-25 17:55:24 1:55:09 1:25:34 0:29:35 smithi main centos 8.stream rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251179 2023-04-25 14:18:04 2023-04-25 16:00:15 2023-04-25 16:31:02 0:30:47 0:22:22 0:08:25 smithi main rhel 8.4 rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} 1
pass 7251180 2023-04-25 14:18:05 2023-04-25 16:01:35 2023-04-25 18:11:20 2:09:45 1:38:04 0:31:41 smithi main centos 8.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251181 2023-04-25 14:18:06 2023-04-25 16:04:16 2023-04-25 17:04:50 1:00:34 0:50:18 0:10:16 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} 3
pass 7251182 2023-04-25 14:18:06 2023-04-25 16:04:57 2023-04-25 16:47:27 0:42:30 0:31:22 0:11:08 smithi main ubuntu 20.04 rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251183 2023-04-25 14:18:07 2023-04-25 16:05:37 2023-04-25 16:41:19 0:35:42 0:23:16 0:12:26 smithi main ubuntu 20.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} 2
pass 7251184 2023-04-25 14:18:08 2023-04-25 16:07:28 2023-04-25 17:20:51 1:13:23 0:43:13 0:30:10 smithi main centos 8.stream rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7251185 2023-04-25 14:18:09 2023-04-25 16:07:48 2023-04-25 16:32:16 0:24:28 0:13:06 0:11:22 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_set_object} 2
fail 7251186 2023-04-25 14:18:10 2023-04-25 16:08:09 2023-04-25 17:47:34 1:39:25 1:09:32 0:29:53 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_cls_all} 2
Failure Reason:

"2023-04-25T17:43:25.762765+0000 mon.a (mon.0) 473 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7251187 2023-04-25 14:18:10 2023-04-25 16:08:19 2023-04-25 16:40:56 0:32:37 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

Command failed on smithi169 with status 1: 'sudo yum install -y kernel'

fail 7251188 2023-04-25 14:18:11 2023-04-25 16:15:31 2023-04-25 16:40:48 0:25:17 smithi main centos 8.stream rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} 3
Failure Reason:

Command failed on smithi133 with status 1: 'sudo yum install -y kernel'

pass 7251189 2023-04-25 14:18:12 2023-04-25 16:15:31 2023-04-25 16:36:51 0:21:20 0:14:40 0:06:40 smithi main rhel 8.4 rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} 1
pass 7251190 2023-04-25 14:18:13 2023-04-25 16:15:51 2023-04-25 16:47:46 0:31:55 0:19:32 0:12:23 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7251191 2023-04-25 14:18:14 2023-04-25 16:17:42 2023-04-25 19:09:59 2:52:17 2:21:34 0:30:43 smithi main centos 8.stream rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/dashboard} 2
pass 7251192 2023-04-25 14:18:14 2023-04-25 16:18:33 2023-04-25 18:14:17 1:55:44 1:25:05 0:30:39 smithi main centos 8.stream rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} 1
pass 7251193 2023-04-25 14:18:15 2023-04-25 16:19:13 2023-04-25 16:45:19 0:26:06 0:15:24 0:10:42 smithi main ubuntu 20.04 rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} 1
pass 7251194 2023-04-25 14:18:16 2023-04-25 16:19:23 2023-04-25 16:47:31 0:28:08 0:18:19 0:09:49 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251195 2023-04-25 14:18:17 2023-04-25 16:19:24 2023-04-25 18:14:46 1:55:22 1:26:02 0:29:20 smithi main centos 8.stream rados/standalone/{supported-random-distro$/{centos_8} workloads/crush} 1
fail 7251196 2023-04-25 14:18:18 2023-04-25 16:19:44 2023-04-25 16:41:10 0:21:26 smithi main ubuntu 18.04 rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04} 4
Failure Reason:

Cannot connect to remote host smithi144

pass 7251197 2023-04-25 14:18:18 2023-04-25 16:23:15 2023-04-25 18:30:04 2:06:49 1:37:33 0:29:16 smithi main centos 8.stream rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} 1
fail 7251198 2023-04-25 14:18:19 2023-04-25 16:23:15 2023-04-25 18:52:12 2:28:57 1:58:30 0:30:27 smithi main centos 8.stream rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi063 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

pass 7251199 2023-04-25 14:18:20 2023-04-25 16:25:06 2023-04-25 18:52:07 2:27:01 1:55:14 0:31:47 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251200 2023-04-25 14:18:21 2023-04-25 16:26:37 2023-04-25 18:40:19 2:13:42 1:44:41 0:29:01 smithi main centos 8.stream rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} 2
pass 7251201 2023-04-25 14:18:22 2023-04-25 16:27:17 2023-04-25 16:47:42 0:20:25 0:09:36 0:10:49 smithi main ubuntu 20.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/workunits} 2
pass 7251202 2023-04-25 14:18:22 2023-04-25 16:27:48 2023-04-25 18:41:14 2:13:26 1:43:05 0:30:21 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} 2
pass 7251203 2023-04-25 14:18:23 2023-04-25 16:27:48 2023-04-25 17:58:21 1:30:33 0:58:54 0:31:39 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7251204 2023-04-25 14:18:24 2023-04-25 16:28:28 2023-04-25 16:58:13 0:29:45 0:22:22 0:07:23 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read} 2
pass 7251205 2023-04-25 14:18:25 2023-04-25 16:28:29 2023-04-25 16:55:44 0:27:15 0:19:30 0:07:45 smithi main rhel 8.4 rados/cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_3.0} 2-node-mgr orchestrator_cli} 2
pass 7251206 2023-04-25 14:18:25 2023-04-25 16:30:50 2023-04-25 16:55:57 0:25:07 0:11:37 0:13:30 smithi main ubuntu 20.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251207 2023-04-25 14:18:26 2023-04-25 16:32:20 2023-04-25 17:08:26 0:36:06 0:28:12 0:07:54 smithi main rhel 8.4 rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} 1
pass 7251208 2023-04-25 14:18:27 2023-04-25 16:32:20 2023-04-25 17:12:14 0:39:54 0:29:38 0:10:16 smithi main rhel 8.4 rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{rhel_8} workloads/ceph_iscsi} 3
pass 7251209 2023-04-25 14:18:28 2023-04-25 16:36:31 2023-04-25 18:34:07 1:57:36 1:27:55 0:29:41 smithi main centos 8.stream rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251210 2023-04-25 14:18:29 2023-04-25 16:36:52 2023-04-25 18:53:45 2:16:53 1:40:49 0:36:04 smithi main centos 8.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 7251211 2023-04-25 14:18:29 2023-04-25 16:40:53 2023-04-25 17:55:59 1:15:06 0:45:05 0:30:01 smithi main centos 8.stream rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} 2
pass 7251212 2023-04-25 14:18:30 2023-04-25 16:40:53 2023-04-25 17:00:41 0:19:48 0:08:39 0:11:09 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} 1
pass 7251213 2023-04-25 14:18:31 2023-04-25 16:40:54 2023-04-25 18:02:16 1:21:22 0:52:02 0:29:20 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
fail 7251214 2023-04-25 14:18:32 2023-04-25 16:41:04 2023-04-25 16:58:49 0:17:45 smithi main ubuntu 18.04 rados/cephadm/smoke-singlehost/{0-distro$/{ubuntu_18.04} 1-start 2-services/basic 3-final} 1
Failure Reason:

Cannot connect to remote host smithi141

pass 7251215 2023-04-25 14:18:33 2023-04-25 16:41:04 2023-04-25 17:17:01 0:35:57 0:27:54 0:08:03 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} 2
pass 7251216 2023-04-25 14:18:33 2023-04-25 16:41:05 2023-04-25 17:17:11 0:36:06 0:27:46 0:08:20 smithi main rhel 8.4 rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} 2
pass 7251217 2023-04-25 14:18:34 2023-04-25 16:41:15 2023-04-25 19:03:20 2:22:05 1:52:30 0:29:35 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7251218 2023-04-25 14:18:35 2023-04-25 16:41:16 2023-04-25 17:21:04 0:39:48 0:31:53 0:07:55 smithi main rhel 8.4 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} 2
pass 7251219 2023-04-25 14:18:36 2023-04-25 16:41:26 2023-04-25 18:12:40 1:31:14 0:55:46 0:35:28 smithi main centos 8.stream rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} 2
pass 7251220 2023-04-25 14:18:37 2023-04-25 16:47:27 2023-04-25 17:22:12 0:34:45 0:23:32 0:11:13 smithi main ubuntu 20.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 7251221 2023-04-25 14:18:37 2023-04-25 16:47:48 2023-04-25 18:41:03 1:53:15 1:24:52 0:28:23 smithi main centos 8.stream rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} 1
pass 7251222 2023-04-25 14:18:38 2023-04-25 16:47:48 2023-04-25 17:32:58 0:45:10 0:39:19 0:05:51 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
pass 7251223 2023-04-25 14:18:39 2023-04-25 16:47:49 2023-04-25 17:21:48 0:33:59 0:24:05 0:09:54 smithi main rhel 8.4 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_python} 2
pass 7251224 2023-04-25 14:18:40 2023-04-25 16:50:39 2023-04-25 18:51:52 2:01:13 1:32:07 0:29:06 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} 1
pass 7251225 2023-04-25 14:18:41 2023-04-25 16:50:40 2023-04-25 17:29:54 0:39:14 0:24:37 0:14:37 smithi main ubuntu 20.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251226 2023-04-25 14:18:41 2023-04-25 16:55:51 2023-04-25 17:40:26 0:44:35 0:34:27 0:10:08 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} 3
pass 7251227 2023-04-25 14:18:42 2023-04-25 16:56:01 2023-04-25 17:31:51 0:35:50 0:28:22 0:07:28 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-localized} 2
pass 7251228 2023-04-25 14:18:43 2023-04-25 16:56:02 2023-04-25 17:25:43 0:29:41 0:15:35 0:14:06 smithi main ubuntu 20.04 rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251229 2023-04-25 14:18:44 2023-04-25 17:24:23 1094 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7251230 2023-04-25 14:18:45 2023-04-25 16:58:53 2023-04-25 18:58:56 2:00:03 1:30:09 0:29:54 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} 1
Failure Reason:

SELinux denials found on ubuntu@smithi174.front.sepia.ceph.com: ['type=AVC msg=audit(1682448972.018:18672): avc: denied { ioctl } for pid=154347 comm="iptables" path="/var/lib/containers/storage/overlay/0c5773dc52c2020155b9cdee6325d44a3abe236af2eac421ffadc291f9e30f70/merged" dev="overlay" ino=3412083 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1682448971.910:18670): avc: denied { ioctl } for pid=154332 comm="iptables" path="/var/lib/containers/storage/overlay/0c5773dc52c2020155b9cdee6325d44a3abe236af2eac421ffadc291f9e30f70/merged" dev="overlay" ino=3412083 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']

pass 7251231 2023-04-25 14:18:45 2023-04-25 16:59:43 2023-04-25 17:19:12 0:19:29 0:13:14 0:06:15 smithi main rhel 8.4 rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} 1
pass 7251232 2023-04-25 14:18:46 2023-04-25 16:59:44 2023-04-25 17:25:32 0:25:48 0:16:06 0:09:42 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7251233 2023-04-25 14:18:47 2023-04-25 17:02:24 2023-04-25 18:58:54 1:56:30 1:24:04 0:32:26 smithi main centos 8.stream rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} 2
fail 7251234 2023-04-25 14:18:48 2023-04-25 17:04:55 2023-04-25 17:25:04 0:20:09 0:10:59 0:09:10 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} 2
Failure Reason:

Command failed on smithi131 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 989e59d2-e38d-11ed-9b00-001a4aab830c -- ceph mon dump -f json'

pass 7251235 2023-04-25 14:18:49 2023-04-25 17:08:06 2023-04-25 19:20:50 2:12:44 1:41:36 0:31:08 smithi main centos 8.stream rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} 1
pass 7251236 2023-04-25 14:18:50 2023-04-25 17:08:36 2023-04-25 17:46:09 0:37:33 0:26:28 0:11:05 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects} 2
pass 7251237 2023-04-25 14:18:51 2023-04-25 17:12:17 2023-04-25 17:58:30 0:46:13 0:34:38 0:11:35 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/cache-snaps} 3
pass 7251238 2023-04-25 14:18:51 2023-04-25 17:14:58 2023-04-25 17:35:41 0:20:43 0:09:00 0:11:43 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} 1
dead 7251239 2023-04-25 14:18:52 2023-04-25 17:16:09 2023-04-26 05:24:57 12:08:48 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

hit max job timeout

pass 7251240 2023-04-25 14:18:53 2023-04-25 17:16:09 2023-04-25 17:43:45 0:27:36 0:21:40 0:05:56 smithi main rhel 8.4 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/crash} 2
pass 7251241 2023-04-25 14:18:54 2023-04-25 17:16:39 2023-04-25 18:00:37 0:43:58 0:36:53 0:07:05 smithi main rhel 8.4 rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} 2
pass 7251242 2023-04-25 14:18:55 2023-04-25 17:17:10 2023-04-25 17:39:03 0:21:53 0:12:39 0:09:14 smithi main ubuntu 20.04 rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} 1
pass 7251243 2023-04-25 14:18:56 2023-04-25 17:17:20 2023-04-25 20:00:53 2:43:33 2:12:30 0:31:03 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} 2
pass 7251244 2023-04-25 14:18:56 2023-04-25 17:19:21 2023-04-25 17:47:37 0:28:16 0:20:08 0:08:08 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7251245 2023-04-25 14:18:57 2023-04-25 17:20:31 2023-04-25 18:27:47 1:07:16 0:37:46 0:29:30 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} 1
pass 7251246 2023-04-25 14:18:58 2023-04-25 17:20:52 2023-04-25 17:44:08 0:23:16 0:12:10 0:11:06 smithi main ubuntu 20.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251247 2023-04-25 14:18:59 2023-04-25 17:21:52 2023-04-25 17:39:51 0:17:59 0:08:07 0:09:52 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251248 2023-04-25 14:19:00 2023-04-25 17:21:53 2023-04-25 19:53:51 2:31:58 2:02:59 0:28:59 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7251249 2023-04-25 14:19:00 2023-04-25 17:22:13 2023-04-25 17:41:19 0:19:06 0:13:38 0:05:28 smithi main rhel 8.4 rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} 2
pass 7251250 2023-04-25 14:19:01 2023-04-25 17:22:14 2023-04-25 19:36:17 2:14:03 1:43:56 0:30:07 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} 2
fail 7251251 2023-04-25 14:19:02 2023-04-25 17:22:24 2023-04-25 17:42:33 0:20:09 smithi main ubuntu 18.04 rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

Cannot connect to remote host smithi171

pass 7251252 2023-04-25 14:19:03 2023-04-25 17:24:25 2023-04-25 18:07:36 0:43:11 0:32:19 0:10:52 smithi main ubuntu 20.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 7251253 2023-04-25 14:19:04 2023-04-25 17:25:25 2023-04-25 17:50:50 0:25:25 0:18:33 0:06:52 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} 2
pass 7251254 2023-04-25 14:19:04 2023-04-25 17:25:35 2023-04-25 17:53:46 0:28:11 0:17:02 0:11:09 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7251255 2023-04-25 14:19:05 2023-04-25 17:25:46 2023-04-25 19:05:50 1:40:04 1:09:02 0:31:02 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} 2
pass 7251256 2023-04-25 14:19:06 2023-04-25 17:25:46 2023-04-25 18:23:53 0:58:07 0:48:01 0:10:06 smithi main rhel 8.4 rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} 1
pass 7251257 2023-04-25 14:19:07 2023-04-25 17:29:07 2023-04-25 19:32:41 2:03:34 1:32:24 0:31:10 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_pids_limit} 1
pass 7251258 2023-04-25 14:19:08 2023-04-25 17:29:07 2023-04-25 17:47:55 0:18:48 0:08:43 0:10:05 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251259 2023-04-25 14:19:08 2023-04-25 17:29:58 2023-04-25 17:51:57 0:21:59 0:10:46 0:11:13 smithi main ubuntu 20.04 rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251260 2023-04-25 14:19:09 2023-04-25 17:29:58 2023-04-25 19:57:10 2:27:12 1:55:02 0:32:10 smithi main centos 8.stream rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} 2
pass 7251261 2023-04-25 14:19:10 2023-04-25 17:31:59 2023-04-25 19:08:38 1:36:39 1:24:58 0:11:41 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/radosbench} 3
pass 7251262 2023-04-25 14:19:11 2023-04-25 17:32:59 2023-04-25 18:10:31 0:37:32 0:24:28 0:13:04 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} 2
pass 7251263 2023-04-25 14:19:12 2023-04-25 17:35:50 2023-04-25 18:04:55 0:29:05 0:18:58 0:10:07 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7251264 2023-04-25 14:19:12 2023-04-25 17:40:01 2023-04-25 19:53:53 2:13:52 1:44:05 0:29:47 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251265 2023-04-25 14:19:13 2023-04-25 17:40:32 2023-04-25 18:47:24 1:06:52 0:54:00 0:12:52 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} 2
pass 7251266 2023-04-25 14:19:14 2023-04-25 17:41:22 2023-04-25 18:01:12 0:19:50 0:08:55 0:10:55 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} 1
pass 7251267 2023-04-25 14:19:15 2023-04-25 17:41:22 2023-04-25 20:06:12 2:24:50 1:53:58 0:30:52 smithi main centos 8.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251268 2023-04-25 14:19:16 2023-04-25 17:42:43 2023-04-25 18:17:53 0:35:10 0:23:06 0:12:04 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7251269 2023-04-25 14:19:16 2023-04-25 17:43:54 2023-04-25 19:41:18 1:57:24 1:29:10 0:28:14 smithi main centos 8.stream rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} 1
pass 7251270 2023-04-25 14:19:17 2023-04-25 17:44:14 2023-04-25 18:03:21 0:19:07 0:13:04 0:06:03 smithi main rhel 8.4 rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251271 2023-04-25 14:19:18 2023-04-25 17:44:14 2023-04-25 19:53:40 2:09:26 1:39:20 0:30:06 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} 1
pass 7251272 2023-04-25 14:19:19 2023-04-25 17:44:15 2023-04-25 20:05:57 2:21:42 1:49:20 0:32:22 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} 2
pass 7251273 2023-04-25 14:19:20 2023-04-25 17:46:15 2023-04-25 18:19:02 0:32:47 0:21:29 0:11:18 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} 2
pass 7251274 2023-04-25 14:19:20 2023-04-25 17:47:36 2023-04-25 18:05:44 0:18:08 0:08:12 0:09:56 smithi main ubuntu 20.04 rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} 2
pass 7251275 2023-04-25 14:19:21 2023-04-25 17:47:46 2023-04-25 18:33:44 0:45:58 0:34:54 0:11:04 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/rbd_cls} 3
pass 7251276 2023-04-25 14:19:22 2023-04-25 17:49:57 2023-04-25 20:08:10 2:18:13 1:47:46 0:30:27 smithi main centos 8.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} 2
pass 7251277 2023-04-25 14:19:23 2023-04-25 17:50:57 2023-04-25 19:17:03 1:26:06 0:55:53 0:30:13 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7251278 2023-04-25 14:19:23 2023-04-25 17:51:58 2023-04-25 18:18:25 0:26:27 0:15:41 0:10:46 smithi main ubuntu 20.04 rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251279 2023-04-25 14:19:24 2023-04-25 17:52:08 2023-04-25 19:09:25 1:17:17 0:44:43 0:32:34 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7251280 2023-04-25 14:19:25 2023-04-25 17:53:49 2023-04-25 18:15:25 0:21:36 0:10:06 0:11:30 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251281 2023-04-25 14:19:26 2023-04-25 17:55:30 2023-04-25 20:15:14 2:19:44 1:50:29 0:29:15 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
fail 7251282 2023-04-25 14:19:27 2023-04-25 17:56:00 2023-04-25 20:03:15 2:07:15 1:34:29 0:32:46 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} 2
Failure Reason:

"2023-04-25T19:59:59.283785+0000 mon.a (mon.0) 471 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7251283 2023-04-25 14:19:27 2023-04-25 17:58:31 2023-04-25 20:05:56 2:07:25 1:37:14 0:30:11 smithi main centos 8.stream rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/failover} 2
pass 7251284 2023-04-25 14:19:28 2023-04-25 17:58:31 2023-04-25 18:47:01 0:48:30 0:35:35 0:12:55 smithi main ubuntu 20.04 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} 2
pass 7251285 2023-04-25 14:19:29 2023-04-25 18:00:42 2023-04-25 18:30:33 0:29:51 0:23:11 0:06:40 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} 2
pass 7251286 2023-04-25 14:19:30 2023-04-25 18:01:12 2023-04-25 20:13:12 2:12:00 1:40:40 0:31:20 smithi main centos 8.stream rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251287 2023-04-25 14:19:30 2023-04-25 18:03:53 2023-04-25 18:58:05 0:54:12 0:42:02 0:12:10 smithi main ubuntu 20.04 rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7251288 2023-04-25 14:19:31 2023-04-25 18:05:04 2023-04-25 19:34:59 1:29:55 1:00:03 0:29:52 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_striper} 2
pass 7251290 2023-04-25 14:19:32 2023-04-25 18:06:11 2023-04-25 20:24:48 2:18:37 1:47:14 0:31:23 smithi main centos 8.stream rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} 1
pass 7251292 2023-04-25 14:19:33 2023-04-25 18:10:32 2023-04-25 20:12:34 2:02:02 1:32:22 0:29:40 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} 1
pass 7251294 2023-04-25 14:19:34 2023-04-25 18:12:43 2023-04-25 20:09:59 1:57:16 1:28:17 0:28:59 smithi main centos 8.stream rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} 1
pass 7251296 2023-04-25 14:19:35 2023-04-25 18:14:54 2023-04-25 19:35:08 1:20:14 1:07:29 0:12:45 smithi main ubuntu 20.04 rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 2
pass 7251298 2023-04-25 14:19:35 2023-04-25 18:17:55 2023-04-25 20:33:57 2:16:02 1:46:43 0:29:19 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251300 2023-04-25 14:19:36 2023-04-25 18:19:06 2023-04-25 18:51:20 0:32:14 0:20:09 0:12:05 smithi main rhel 8.4 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 7251302 2023-04-25 14:19:37 2023-04-25 18:30:09 2023-04-25 18:51:56 0:21:47 0:15:04 0:06:43 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7251304 2023-04-25 14:19:38 2023-04-25 18:33:50 2023-04-25 20:27:26 1:53:36 1:24:50 0:28:46 smithi main centos 8.stream rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} 1
pass 7251306 2023-04-25 14:19:39 2023-04-25 18:38:41 2023-04-25 19:13:04 0:34:23 0:23:15 0:11:08 smithi main ubuntu 20.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} 2
pass 7251308 2023-04-25 14:19:39 2023-04-25 18:40:22 2023-04-25 19:28:43 0:48:21 0:41:11 0:07:10 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} 2
pass 7251310 2023-04-25 14:19:40 2023-04-25 18:42:54 2023-04-25 19:02:41 0:19:47 0:08:58 0:10:49 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} 1
fail 7251313 2023-04-25 14:19:41 2023-04-25 19:08:51 682 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} 2
Failure Reason:

Command failed on smithi189 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fa6fe76-e39c-11ed-9b00-001a4aab830c -- ceph mon dump -f json'

pass 7251315 2023-04-25 14:19:42 2023-04-25 21:11:40 6567 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} 5
fail 7251317 2023-04-25 14:19:43 2023-04-25 18:53:48 2023-04-25 19:09:11 0:15:23 0:06:15 0:09:08 smithi main ubuntu 18.04 rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.6.2} 1
Failure Reason:

Command failed on smithi135 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull'

pass 7251319 2023-04-25 14:19:43 2023-04-25 21:51:46 8876 smithi main centos 8.stream rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} 2
pass 7251321 2023-04-25 14:19:44 2023-04-25 18:57:20 2023-04-25 19:53:05 0:55:45 0:48:17 0:07:28 smithi main rhel 8.4 rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} 2
pass 7251322 2023-04-25 14:19:45 2023-04-25 18:58:11 2023-04-25 20:03:38 1:05:27 0:56:49 0:08:38 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} 3
pass 7251324 2023-04-25 14:19:46 2023-04-25 18:58:21 2023-04-25 19:26:28 0:28:07 0:20:28 0:07:39 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7251326 2023-04-25 14:19:47 2023-04-25 18:59:12 2023-04-25 20:55:44 1:56:32 1:28:04 0:28:28 smithi main centos 8.stream rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
fail 7251328 2023-04-25 14:19:48 2023-04-25 19:02:44 2023-04-25 19:21:11 0:18:27 0:07:31 0:10:56 smithi main rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5b35a461e1c1b935b2b3fc7c43d68a58c1a41547

pass 7251330 2023-04-25 14:19:48 2023-04-25 19:05:55 2023-04-25 20:12:57 1:07:02 0:53:55 0:13:07 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} 2
pass 7251332 2023-04-25 14:19:49 2023-04-25 19:08:46 2023-04-25 19:46:54 0:38:08 0:28:07 0:10:01 smithi main ubuntu 20.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251334 2023-04-25 14:19:50 2023-04-25 19:09:27 2023-04-25 21:27:55 2:18:28 1:47:58 0:30:30 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251336 2023-04-25 14:19:51 2023-04-25 19:13:08 2023-04-25 19:44:58 0:31:50 0:24:02 0:07:48 smithi main rhel 8.4 rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} 2
pass 7251338 2023-04-25 14:19:51 2023-04-25 19:16:39 2023-04-25 20:30:25 1:13:46 0:44:22 0:29:24 smithi main centos 8.stream rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} 2
pass 7251340 2023-04-25 14:19:52 2023-04-25 19:17:40 2023-04-25 19:44:56 0:27:16 0:16:10 0:11:06 smithi main rhel 8.4 rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} 1
pass 7251342 2023-04-25 14:19:53 2023-04-25 19:21:22 2023-04-25 19:48:39 0:27:17 0:18:20 0:08:57 smithi main rhel 8.4 rados/standalone/{supported-random-distro$/{rhel_8} workloads/mgr} 1
pass 7251344 2023-04-25 14:19:54 2023-04-25 19:22:22 2023-04-25 21:50:15 2:27:53 1:54:35 0:33:18 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
pass 7251346 2023-04-25 14:19:55 2023-04-25 19:27:04 2023-04-25 19:59:18 0:32:14 0:14:23 0:17:51 smithi main ubuntu 20.04 rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 3
pass 7251348 2023-04-25 14:19:55 2023-04-25 19:35:16 2023-04-25 20:00:09 0:24:53 0:19:05 0:05:48 smithi main rhel 8.4 rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251350 2023-04-25 14:19:56 2023-04-25 19:36:27 2023-04-25 21:38:43 2:02:16 1:27:16 0:35:00 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big} 2
pass 7251352 2023-04-25 14:19:57 2023-04-25 19:44:59 2023-04-25 20:28:09 0:43:10 0:32:07 0:11:03 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} 2
pass 7251354 2023-04-25 14:19:58 2023-04-25 19:47:00 2023-04-25 20:14:02 0:27:02 0:13:57 0:13:05 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} 2
pass 7251356 2023-04-25 14:19:59 2023-04-25 19:53:12 2023-04-25 22:05:47 2:12:35 1:42:04 0:30:31 smithi main centos 8.stream rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7251358 2023-04-25 14:19:59 2023-04-25 19:53:53 2023-04-25 20:29:36 0:35:43 0:29:28 0:06:15 smithi main rhel 8.4 rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} 2
pass 7251360 2023-04-25 14:20:00 2023-04-25 19:54:14 2023-04-25 22:00:38 2:06:24 1:32:19 0:34:05 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} 1
pass 7251362 2023-04-25 14:20:01 2023-04-25 19:57:15 2023-04-25 22:12:10 2:14:55 1:45:16 0:29:39 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} 2
pass 7251364 2023-04-25 14:20:02 2023-04-25 19:59:26 2023-04-25 20:19:45 0:20:19 0:08:52 0:11:27 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} 1
pass 7251366 2023-04-25 14:20:03 2023-04-25 20:00:17 2023-04-25 20:24:20 0:24:03 0:16:33 0:07:30 smithi main rhel 8.4 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} tasks/insights} 2
pass 7251368 2023-04-25 14:20:04 2023-04-25 20:03:18 2023-04-25 22:05:01 2:01:43 1:32:43 0:29:00 smithi main centos 8.stream rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} 2
pass 7251370 2023-04-25 14:20:04 2023-04-25 20:05:49 2023-04-25 20:28:42 0:22:53 0:12:07 0:10:46 smithi main ubuntu 20.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251372 2023-04-25 14:20:05 2023-04-25 20:06:20 2023-04-25 20:35:49 0:29:29 0:17:22 0:12:07 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7251374 2023-04-25 14:20:06 2023-04-25 20:08:11 2023-04-25 20:52:38 0:44:27 0:32:52 0:11:35 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} 3
pass 7251376 2023-04-25 14:20:07 2023-04-25 20:10:02 2023-04-25 20:41:25 0:31:23 0:17:34 0:13:49 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251378 2023-04-25 14:20:07 2023-04-25 20:13:04 2023-04-25 20:43:13 0:30:09 0:20:34 0:09:35 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} 2
fail 7251380 2023-04-25 14:20:08 2023-04-25 20:13:15 2023-04-25 20:42:54 0:29:39 0:16:30 0:13:09 smithi main ubuntu 18.04 rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

Command failed on smithi023 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f2a9b6c8-e3a7-11ed-9b00-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nceph orch daemon add osd $HOST:$DEV\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\''

pass 7251382 2023-04-25 14:20:09 2023-04-25 20:15:16 2023-04-25 20:35:04 0:19:48 0:08:55 0:10:53 smithi main ubuntu 20.04 rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251384 2023-04-25 14:20:10 2023-04-25 20:18:27 2023-04-25 22:21:34 2:03:07 1:33:59 0:29:08 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} 1
pass 7251386 2023-04-25 14:20:11 2023-04-25 20:22:38 2023-04-25 21:05:54 0:43:16 0:34:01 0:09:15 smithi main rhel 8.4 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 7251388 2023-04-25 14:20:12 2023-04-25 20:24:59 2023-04-25 22:52:44 2:27:45 2:20:48 0:06:57 smithi main rhel 8.4 rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} 1
pass 7251390 2023-04-25 14:20:13 2023-04-25 20:25:40 2023-04-25 22:36:56 2:11:16 1:42:05 0:29:11 smithi main centos 8.stream rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
pass 7251392 2023-04-25 14:20:13 2023-04-25 20:28:11 2023-04-25 20:53:18 0:25:07 0:18:38 0:06:29 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} 2
pass 7251394 2023-04-25 14:20:14 2023-04-25 20:28:12 2023-04-25 22:48:45 2:20:33 1:49:40 0:30:53 smithi main centos 8.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 2
pass 7251396 2023-04-25 14:20:15 2023-04-25 20:28:53 2023-04-25 21:00:00 0:31:07 0:19:03 0:12:04 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7251398 2023-04-25 14:20:16 2023-04-25 20:32:54 2023-04-25 22:55:02 2:22:08 1:50:59 0:31:09 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251400 2023-04-25 14:20:17 2023-04-25 20:34:05 2023-04-25 21:05:11 0:31:06 0:20:30 0:10:36 smithi main ubuntu 20.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} 2
pass 7251402 2023-04-25 14:20:17 2023-04-25 20:35:56 2023-04-25 21:09:15 0:33:19 0:26:06 0:07:13 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} 2
pass 7251404 2023-04-25 14:20:18 2023-04-25 20:36:07 2023-04-25 21:43:01 1:06:54 0:37:51 0:29:03 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} 1
pass 7251406 2023-04-25 14:20:19 2023-04-25 20:39:18 2023-04-25 21:22:05 0:42:47 0:29:13 0:13:34 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} 2
pass 7251408 2023-04-25 14:20:20 2023-04-25 20:42:29 2023-04-25 21:03:56 0:21:27 0:08:16 0:13:11 smithi main ubuntu 20.04 rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
fail 7251410 2023-04-25 14:20:20 2023-04-25 20:43:10 2023-04-25 21:01:55 0:18:45 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/cache-snaps} 3
Failure Reason:

Cannot connect to remote host smithi049

pass 7251412 2023-04-25 14:20:21 2023-04-25 20:51:12 2023-04-25 22:19:54 1:28:42 0:58:06 0:30:36 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7251414 2023-04-25 14:20:22 2023-04-25 20:52:43 2023-04-25 22:06:25 1:13:42 0:44:08 0:29:34 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7251416 2023-04-25 14:20:23 2023-04-25 20:55:14 2023-04-25 21:36:53 0:41:39 0:35:26 0:06:13 smithi main rhel 8.4 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251418 2023-04-25 14:20:24 2023-04-25 20:56:05 2023-04-25 21:30:50 0:34:45 0:21:58 0:12:47 smithi main ubuntu 20.04 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} 2
pass 7251420 2023-04-25 14:20:24 2023-04-25 20:56:56 2023-04-25 23:19:18 2:22:22 1:52:53 0:29:29 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
pass 7251422 2023-04-25 14:20:25 2023-04-25 21:32:21 1402 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} 1
pass 7251424 2023-04-25 14:20:26 2023-04-25 21:00:08 2023-04-25 23:13:02 2:12:54 1:41:24 0:31:30 smithi main centos 8.stream rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} 1
fail 7251426 2023-04-25 14:20:27 2023-04-25 21:01:59 2023-04-25 23:19:39 2:17:40 1:43:04 0:34:36 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} 2
Failure Reason:

"2023-04-25T23:15:08.382251+0000 osd.0 (osd.0) 8 : cluster [ERR] osd.0 found snap mapper error on pg 3.1 oid 3:92a79226:::smithi023144718-273:16f snaps missing in mapper, should be: 15b,15d,163,164,16d,16f was r -2...repaired" in cluster log

pass 7251428 2023-04-25 14:20:28 2023-04-25 21:05:20 2023-04-25 21:53:34 0:48:14 0:41:00 0:07:14 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7251430 2023-04-25 14:20:28 2023-04-25 21:06:01 2023-04-25 23:10:46 2:04:45 1:31:53 0:32:52 smithi main centos 8.stream rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251432 2023-04-25 14:20:29 2023-04-25 21:11:42 2023-04-25 21:39:00 0:27:18 0:20:27 0:06:51 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/connectivity start} 2
pass 7251434 2023-04-25 14:20:30 2023-04-25 21:11:43 2023-04-25 21:43:00 0:31:17 0:08:49 0:22:28 smithi main ubuntu 20.04 rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} 3
pass 7251436 2023-04-25 14:20:31 2023-04-25 21:22:35 2023-04-25 23:25:23 2:02:48 1:31:18 0:31:30 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_iscsi_pids_limit} 1
pass 7251438 2023-04-25 14:20:31 2023-04-25 21:24:56 2023-04-25 21:59:07 0:34:11 0:18:28 0:15:43 smithi main ubuntu 20.04 rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7251440 2023-04-25 14:20:32 2023-04-25 21:29:18 2023-04-25 21:56:05 0:26:47 0:18:57 0:07:50 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7251442 2023-04-25 14:20:33 2023-04-25 21:30:59 2023-04-25 23:26:24 1:55:25 1:24:51 0:30:34 smithi main centos 8.stream rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} 1
pass 7251444 2023-04-25 14:20:34 2023-04-25 21:31:40 2023-04-25 23:37:56 2:06:16 1:36:46 0:29:30 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} 2
pass 7251446 2023-04-25 14:20:35 2023-04-25 21:37:01 2023-04-25 22:07:27 0:30:26 0:20:28 0:09:58 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps} 2
pass 7251448 2023-04-25 14:20:36 2023-04-25 21:38:42 2023-04-25 22:14:52 0:36:10 0:25:48 0:10:22 smithi main ubuntu 20.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/module_selftest} 2
pass 7251450 2023-04-25 14:20:36 2023-04-25 21:40:33 2023-04-25 23:45:17 2:04:44 1:31:46 0:32:58 smithi main centos 8.stream rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251452 2023-04-25 14:20:37 2023-04-25 21:45:15 2023-04-26 00:06:48 2:21:33 1:51:45 0:29:48 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251454 2023-04-25 14:20:38 2023-04-25 21:50:17 2023-04-25 22:13:08 0:22:51 0:15:41 0:07:10 smithi main rhel 8.4 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} 2
pass 7251456 2023-04-25 14:20:39 2023-04-25 21:52:48 2023-04-25 22:12:04 0:19:16 0:12:41 0:06:35 smithi main rhel 8.4 rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251458 2023-04-25 14:20:40 2023-04-25 21:56:09 2023-04-25 23:39:33 1:43:24 1:14:19 0:29:05 smithi main centos 8.stream rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} 2
fail 7251460 2023-04-25 14:20:40 2023-04-25 21:56:30 2023-04-25 22:18:02 0:21:32 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/radosbench} 3
Failure Reason:

Cannot connect to remote host smithi179

pass 7251462 2023-04-25 14:20:41 2023-04-25 22:05:02 2023-04-26 00:18:05 2:13:03 1:43:48 0:29:15 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_nfs} 1
pass 7251464 2023-04-25 14:20:42 2023-04-25 22:05:03 2023-04-26 00:25:19 2:20:16 1:48:11 0:32:05 smithi main centos 8.stream rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} 1
pass 7251466 2023-04-25 14:20:43 2023-04-25 22:06:13 2023-04-25 22:30:31 0:24:18 0:15:54 0:08:24 smithi main rhel 8.4 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 7251468 2023-04-25 14:20:44 2023-04-25 22:07:34 2023-04-25 22:32:46 0:25:12 0:18:20 0:06:52 smithi main rhel 8.4 rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} 1
pass 7251470 2023-04-25 14:20:44 2023-04-25 22:08:25 2023-04-25 22:40:09 0:31:44 0:21:30 0:10:14 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/classic start} 2
pass 7251472 2023-04-25 14:20:45 2023-04-25 22:08:56 2023-04-25 22:41:24 0:32:28 0:22:12 0:10:16 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache} 2
pass 7251474 2023-04-25 14:20:46 2023-04-25 22:12:17 2023-04-25 22:40:35 0:28:18 0:19:58 0:08:20 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7251476 2023-04-25 14:20:47 2023-04-25 22:13:58 2023-04-26 00:47:23 2:33:25 2:04:32 0:28:53 smithi main centos 8.stream rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251478 2023-04-25 14:20:48 2023-04-25 22:14:49 2023-04-26 00:54:15 2:39:26 2:10:31 0:28:55 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
pass 7251481 2023-04-25 14:20:49 2023-04-26 02:33:18 13455 smithi main centos 8.stream rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} 1
pass 7251483 2023-04-25 14:20:50 2023-04-25 22:20:32 2023-04-25 23:05:56 0:45:24 0:33:26 0:11:58 smithi main ubuntu 20.04 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} 2
pass 7251485 2023-04-25 14:20:51 2023-04-25 22:23:03 2023-04-25 22:45:37 0:22:34 0:10:50 0:11:44 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} 1
pass 7251487 2023-04-25 14:20:51 2023-04-25 22:25:34 2023-04-25 23:06:26 0:40:52 0:27:56 0:12:56 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7251489 2023-04-25 14:20:52 2023-04-25 22:30:36 2023-04-26 00:40:26 2:09:50 1:39:52 0:29:58 smithi main centos 8.stream rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} 2
pass 7251491 2023-04-25 14:20:53 2023-04-25 22:31:16 2023-04-25 22:52:44 0:21:28 0:09:49 0:11:39 smithi main ubuntu 20.04 rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251493 2023-04-25 14:20:54 2023-04-25 22:32:47 2023-04-26 00:38:04 2:05:17 1:34:32 0:30:45 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli} 1
pass 7251495 2023-04-25 14:20:55 2023-04-25 22:40:09 2023-04-25 23:22:14 0:42:05 0:31:44 0:10:21 smithi main ubuntu 20.04 rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
fail 7251496 2023-04-25 14:20:55 2023-04-25 22:40:09 2023-04-25 22:58:53 0:18:44 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

Cannot connect to remote host smithi172

pass 7251499 2023-04-25 14:20:56 2023-04-25 22:41:41 2023-04-26 01:05:46 2:24:05 1:52:28 0:31:37 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} 2
dead 7251501 2023-04-25 14:20:57 2023-04-26 10:58:26 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

hit max job timeout

pass 7251503 2023-04-25 14:20:58 2023-04-26 01:08:57 6303 smithi main centos 8.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251504 2023-04-25 14:20:59 2023-04-25 22:52:54 2023-04-26 00:52:07 1:59:13 1:30:40 0:28:33 smithi main centos 8.stream rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} 1
pass 7251506 2023-04-25 14:20:59 2023-04-25 22:52:55 2023-04-26 00:08:27 1:15:32 0:42:52 0:32:40 smithi main centos 8.stream rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7251508 2023-04-25 14:21:00 2023-04-25 22:58:17 2023-04-25 23:32:49 0:34:32 0:24:37 0:09:55 smithi main ubuntu 20.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 2
pass 7251510 2023-04-25 14:21:01 2023-04-25 22:59:28 2023-04-25 23:43:53 0:44:25 0:34:55 0:09:30 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/rbd_cls} 3
pass 7251512 2023-04-25 14:21:02 2023-04-25 22:59:48 2023-04-25 23:21:42 0:21:54 0:15:19 0:06:35 smithi main rhel 8.4 rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} 1
pass 7251514 2023-04-25 14:21:03 2023-04-25 23:06:00 2023-04-25 23:40:59 0:34:59 0:23:09 0:11:50 smithi main ubuntu 20.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/connectivity start} 2
pass 7251516 2023-04-25 14:21:04 2023-04-25 23:07:21 2023-04-26 01:02:36 1:55:15 1:23:42 0:31:33 smithi main centos 8.stream rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} 2
pass 7251518 2023-04-25 14:21:04 2023-04-25 23:10:52 2023-04-26 01:30:30 2:19:38 1:46:43 0:32:55 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli_mon} 5
pass 7251520 2023-04-25 14:21:05 2023-04-25 23:19:45 2023-04-26 00:02:55 0:43:10 0:35:42 0:07:28 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/rados_api_tests} 2
pass 7251522 2023-04-25 14:21:06 2023-04-25 23:23:16 2023-04-25 23:57:04 0:33:48 0:19:16 0:14:32 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7251524 2023-04-25 14:21:07 2023-04-25 23:26:17 2023-04-26 02:22:58 2:56:41 2:21:56 0:34:45 smithi main centos 8.stream rados/dashboard/{centos_8.stream_container_tools clusters/{2-node-mgr} debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} 2
pass 7251526 2023-04-25 14:21:08 2023-04-25 23:32:59 2023-04-26 00:08:26 0:35:27 0:28:27 0:07:00 smithi main rhel 8.4 rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} 1
fail 7251528 2023-04-25 14:21:08 2023-04-25 23:33:20 2023-04-26 03:03:44 3:30:24 3:14:56 0:15:28 smithi main ubuntu 18.04 rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_18.04} 4
Failure Reason:

"2023-04-26T02:32:30.868869+0000 osd.10 (osd.10) 27 : cluster [ERR] 5.9 deep-scrub : stat mismatch, got 4/4 objects, 0/0 clones, 0/0 dirty, 0/0 omap, 0/0 pinned, 4/4 hit_set_archive, 0/0 whiteouts, 7635/0 bytes, 0/0 manifest objects, 7635/7635 hit_set_archive bytes." in cluster log

pass 7251530 2023-04-25 14:21:09 2023-04-25 23:41:02 2023-04-26 00:41:28 1:00:26 0:50:58 0:09:28 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/snaps-few-objects} 3
pass 7251532 2023-04-25 14:21:10 2023-04-25 23:44:03 2023-04-26 00:13:15 0:29:12 0:22:40 0:06:32 smithi main rhel 8.4 rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} 1
fail 7251534 2023-04-25 14:21:11 2023-04-25 23:45:24 2023-04-26 06:03:33 6:18:09 5:46:39 0:31:30 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

Command failed (workunit test cls/test_cls_rbd.sh) on smithi157 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh'

pass 7251536 2023-04-25 14:21:12 2023-04-25 23:46:15 2023-04-26 00:13:56 0:27:41 0:12:03 0:15:38 smithi main ubuntu 20.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
fail 7251538 2023-04-25 14:21:13 2023-04-25 23:56:27 2023-04-26 02:30:00 2:33:33 2:03:52 0:29:41 smithi main centos 8.stream rados/cephadm/dashboard/{0-distro/centos_8.stream_container_tools task/test_e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi116 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5b35a461e1c1b935b2b3fc7c43d68a58c1a41547 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

pass 7251540 2023-04-25 14:21:13 2023-04-25 23:56:38 2023-04-26 02:01:16 2:04:38 1:32:55 0:31:43 smithi main centos 8.stream rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/progress} 2
pass 7251542 2023-04-25 14:21:14 2023-04-25 23:58:59 2023-04-26 00:40:29 0:41:30 0:34:54 0:06:36 smithi main rhel 8.4 rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/force-sync-many workloads/rados_api_tests} 2
pass 7251544 2023-04-25 14:21:15 2023-04-26 00:01:10 2023-04-26 02:16:32 2:15:22 1:43:34 0:31:48 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251546 2023-04-25 14:21:16 2023-04-26 00:06:58 2023-04-26 00:28:31 0:21:33 0:08:48 0:12:45 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} 1
dead 7251548 2023-04-25 14:21:17 2023-04-26 00:08:29 2023-04-26 01:15:38 1:07:09 0:33:55 0:33:14 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

{'smithi039.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'CentOS-PowerTools': Yum repo downloading error: Downloading error(s): repodata/38c2c78c3f89e4d347f07263805d33d8ae17df60ebab2e8a259218c57bcae2fb-comps-PowerTools.x86_64.xml - Cannot download, all mirrors were already tried without success", 'rc': 1, 'results': []}}

pass 7251550 2023-04-25 14:21:17 2023-04-26 00:12:20 2023-04-26 01:23:23 1:11:03 0:59:00 0:12:03 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} 2
pass 7251552 2023-04-25 14:21:18 2023-04-26 00:13:41 2023-04-26 00:35:09 0:21:28 0:14:50 0:06:38 smithi main rhel 8.4 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/readwrite} 2
pass 7251554 2023-04-25 14:21:19 2023-04-26 00:14:02 2023-04-26 00:40:03 0:26:01 0:12:41 0:13:20 smithi main ubuntu 20.04 rados/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr orchestrator_cli} 2
pass 7251556 2023-04-25 14:21:20 2023-04-26 00:18:13 2023-04-26 02:38:31 2:20:18 1:45:46 0:34:32 smithi main centos 8.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 7251558 2023-04-25 14:21:21 2023-04-26 00:25:25 2023-04-26 00:51:43 0:26:18 0:15:56 0:10:22 smithi main rhel 8.4 rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251560 2023-04-25 14:21:22 2023-04-26 00:35:17 2023-04-26 03:51:18 3:16:01 2:45:58 0:30:03 smithi main centos 8.stream rados/cephadm/rbd_iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} 3
pass 7251562 2023-04-25 14:21:22 2023-04-26 00:38:29 2023-04-26 01:34:36 0:56:07 0:46:36 0:09:31 smithi main ubuntu 20.04 rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251564 2023-04-25 14:21:23 2023-04-26 00:39:39 2023-04-26 01:55:12 1:15:33 0:45:22 0:30:11 smithi main centos 8.stream rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/classic start} 2
pass 7251566 2023-04-25 14:21:24 2023-04-26 00:40:10 2023-04-26 01:53:52 1:13:42 0:43:30 0:30:12 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7251568 2023-04-25 14:21:25 2023-04-26 00:40:31 2023-04-26 01:52:20 1:11:49 0:42:05 0:29:44 smithi main centos 8.stream rados/cephadm/smoke-singlehost/{0-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} 1
pass 7251570 2023-04-25 14:21:26 2023-04-26 01:53:39 3410 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} 2
pass 7251572 2023-04-25 14:21:26 2023-04-26 00:46:03 2023-04-26 01:20:32 0:34:29 0:24:03 0:10:26 smithi main ubuntu 20.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} 2
pass 7251574 2023-04-25 14:21:27 2023-04-26 00:46:14 2023-04-26 03:10:44 2:24:30 1:53:40 0:30:50 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
pass 7251576 2023-04-25 14:21:28 2023-04-26 00:48:45 2023-04-26 01:07:38 0:18:53 0:12:53 0:06:00 smithi main rhel 8.4 rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} 1
pass 7251578 2023-04-25 14:21:29 2023-04-26 00:54:17 2023-04-26 01:13:05 0:18:48 0:12:26 0:06:22 smithi main rhel 8.4 rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} 1
pass 7251580 2023-04-25 14:21:30 2023-04-26 01:00:18 2023-04-26 01:46:09 0:45:51 0:32:49 0:13:02 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls mon_election/classic} 2
pass 7251582 2023-04-25 14:21:30 2023-04-26 01:02:39 2023-04-26 01:26:51 0:24:12 0:17:24 0:06:48 smithi main rhel 8.4 rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} 1
fail 7251584 2023-04-25 14:21:31 2023-04-26 01:03:40 2023-04-26 01:25:01 0:21:21 0:06:44 0:14:37 smithi main ubuntu 20.04 rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/master} 3
Failure Reason:

Command failed on smithi019 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull'

pass 7251586 2023-04-25 14:21:32 2023-04-26 01:06:31 2023-04-26 01:56:40 0:50:09 0:40:52 0:09:17 smithi main ubuntu 18.04 rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
pass 7251588 2023-04-25 14:21:33 2023-04-26 01:07:42 2023-04-26 01:48:51 0:41:09 0:32:58 0:08:11 smithi main rhel 8.4 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251590 2023-04-25 14:21:34 2023-04-26 01:11:14 2023-04-26 03:06:29 1:55:15 1:25:10 0:30:05 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_adoption} 1
pass 7251592 2023-04-25 14:21:35 2023-04-26 01:12:34 2023-04-26 02:14:10 1:01:36 0:53:57 0:07:39 smithi main rhel 8.4 rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} 1
pass 7251594 2023-04-25 14:21:35 2023-04-26 01:15:46 2023-04-26 01:52:04 0:36:18 0:28:59 0:07:19 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} 2
pass 7251596 2023-04-25 14:21:36 2023-04-26 01:20:37 2023-04-26 01:46:30 0:25:53 0:16:52 0:09:01 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7251598 2023-04-25 14:21:37 2023-04-26 01:23:28 2023-04-26 01:48:37 0:25:09 0:19:25 0:05:44 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7251600 2023-04-25 14:21:38 2023-04-26 01:25:10 2023-04-26 02:55:00 1:29:50 1:16:49 0:13:01 smithi main ubuntu 20.04 rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251602 2023-04-25 14:21:39 2023-04-26 01:28:01 2023-04-26 01:50:24 0:22:23 0:11:49 0:10:34 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} 1
fail 7251604 2023-04-25 14:21:39 2023-04-26 01:30:32 2023-04-26 03:29:47 1:59:15 1:29:46 0:29:29 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm} 1
Failure Reason:

SELinux denials found on ubuntu@smithi183.front.sepia.ceph.com: ['type=AVC msg=audit(1682479620.749:18665): avc: denied { ioctl } for pid=154198 comm="iptables" path="/var/lib/containers/storage/overlay/e731217cd20c7d46689954630ac7dc235fb02b20d200762d79c51990afa94c6e/merged" dev="overlay" ino=3412077 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']

pass 7251606 2023-04-25 14:21:40 2023-04-26 01:30:33 2023-04-26 01:49:40 0:19:07 0:13:08 0:05:59 smithi main rhel 8.4 rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251608 2023-04-25 14:21:41 2023-04-26 01:40:55 2023-04-26 02:06:18 0:25:23 0:13:13 0:12:10 smithi main ubuntu 20.04 rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} 2
pass 7251610 2023-04-25 14:21:42 2023-04-26 01:46:06 2023-04-26 02:11:58 0:25:52 0:18:23 0:07:29 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/connectivity start} 2
pass 7251612 2023-04-25 14:21:43 2023-04-26 01:46:37 2023-04-26 02:31:47 0:45:10 0:34:00 0:11:10 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} 3
pass 7251614 2023-04-25 14:21:43 2023-04-26 01:48:38 2023-04-26 03:28:51 1:40:13 1:09:37 0:30:36 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/repair_test} 2
pass 7251616 2023-04-25 14:21:44 2023-04-26 01:49:49 2023-04-26 03:57:59 2:08:10 1:37:25 0:30:45 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} 2
fail 7251618 2023-04-25 14:21:45 2023-04-26 01:52:10 2023-04-26 02:20:37 0:28:27 smithi main centos 8.stream rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
Failure Reason:

Command failed on smithi149 with status 1: 'sudo yum install -y kernel'

pass 7251620 2023-04-25 14:21:46 2023-04-26 01:54:01 2023-04-26 04:12:42 2:18:41 1:48:10 0:30:31 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251622 2023-04-25 14:21:47 2023-04-26 01:56:43 2023-04-26 03:55:22 1:58:39 1:26:50 0:31:49 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} 2
pass 7251624 2023-04-25 14:21:47 2023-04-26 01:58:24 2023-04-26 04:02:15 2:03:51 1:31:32 0:32:19 smithi main centos 8.stream rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/prometheus} 2
pass 7251626 2023-04-25 14:21:48 2023-04-26 02:01:25 2023-04-26 03:35:21 1:33:56 1:23:00 0:10:56 smithi main rhel 8.4 rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} 1
pass 7251628 2023-04-25 14:21:49 2023-04-26 02:07:27 2023-04-26 03:15:47 1:08:20 0:57:38 0:10:42 smithi main ubuntu 20.04 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} 2
pass 7251630 2023-04-25 14:21:50 2023-04-26 02:12:08 2023-04-26 02:36:45 0:24:37 0:17:33 0:07:04 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7251632 2023-04-25 14:21:51 2023-04-26 02:14:19 2023-04-26 03:24:00 1:09:41 0:37:50 0:31:51 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm_repos} 1
pass 7251634 2023-04-25 14:21:51 2023-04-26 02:18:00 2023-04-26 02:54:08 0:36:08 0:22:57 0:13:11 smithi main ubuntu 20.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} 2
pass 7251636 2023-04-25 14:21:52 2023-04-26 02:19:27 2023-04-26 04:18:55 1:59:28 1:29:54 0:29:34 smithi main centos 8.stream rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251637 2023-04-25 14:21:53 2023-04-26 02:19:27 2023-04-26 04:47:53 2:28:26 1:56:23 0:32:03 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
pass 7251638 2023-04-25 14:21:54 2023-04-26 02:20:48 2023-04-26 03:00:56 0:40:08 0:30:13 0:09:55 smithi main ubuntu 20.04 rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
pass 7251639 2023-04-25 14:21:55 2023-04-26 02:20:48 2023-04-26 02:44:38 0:23:50 0:11:29 0:12:21 smithi main ubuntu 20.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
pass 7251640 2023-04-25 14:21:56 2023-04-26 02:22:39 2023-04-26 02:48:36 0:25:57 0:12:54 0:13:03 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} 2
pass 7251641 2023-04-25 14:21:56 2023-04-26 02:22:59 2023-04-26 02:48:02 0:25:03 0:15:33 0:09:30 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7251643 2023-04-25 14:21:57 2023-04-26 02:26:31 2023-04-26 02:49:29 0:22:58 0:16:52 0:06:06 smithi main rhel 8.4 rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} 1
pass 7251645 2023-04-25 14:21:58 2023-04-26 02:30:02 2023-04-26 02:55:54 0:25:52 0:18:45 0:07:07 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} 2
pass 7251647 2023-04-25 14:21:59 2023-04-26 02:32:03 2023-04-26 03:01:43 0:29:40 0:17:06 0:12:34 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7251649 2023-04-25 14:21:59 2023-04-26 02:35:44 2023-04-26 04:31:55 1:56:11 1:26:13 0:29:58 smithi main centos 8.stream rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} 1
pass 7251651 2023-04-25 14:22:00 2023-04-26 02:38:36 2023-04-26 03:11:10 0:32:34 0:22:10 0:10:24 smithi main ubuntu 20.04 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects} 2
pass 7251653 2023-04-25 14:22:01 2023-04-26 02:40:47 2023-04-26 03:40:15 0:59:28 0:45:07 0:14:21 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} 3
pass 7251655 2023-04-25 14:22:02 2023-04-26 02:48:09 2023-04-26 03:10:05 0:21:56 0:10:47 0:11:09 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} 1
pass 7251657 2023-04-25 14:22:03 2023-04-26 02:48:39 2023-04-26 04:50:26 2:01:47 1:32:08 0:29:39 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_iscsi_pids_limit} 1
pass 7251659 2023-04-25 14:22:03 2023-04-26 02:53:21 2023-04-26 03:20:06 0:26:45 0:19:00 0:07:45 smithi main rhel 8.4 rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} 1
pass 7251661 2023-04-25 14:22:04 2023-04-26 02:56:02 2023-04-26 03:21:17 0:25:15 0:13:50 0:11:25 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/set-chunks-read} 2
pass 7251663 2023-04-25 14:22:05 2023-04-26 03:02:53 2023-04-26 05:21:13 2:18:20 1:47:32 0:30:48 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251665 2023-04-25 14:22:06 2023-04-26 03:06:35 2023-04-26 05:06:58 2:00:23 1:31:01 0:29:22 smithi main centos 8.stream rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} 1
pass 7251667 2023-04-25 14:22:07 2023-04-26 03:08:06 2023-04-26 05:11:13 2:03:07 1:34:43 0:28:24 smithi main centos 8.stream rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} 1
pass 7251669 2023-04-25 14:22:07 2023-04-26 03:09:57 2023-04-26 04:15:02 1:05:05 0:52:50 0:12:15 smithi main ubuntu 20.04 rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/connectivity} 2
pass 7251671 2023-04-25 14:22:08 2023-04-26 03:15:48 2023-04-26 03:52:29 0:36:41 0:24:48 0:11:53 smithi main ubuntu 20.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7251673 2023-04-25 14:22:09 2023-04-26 03:17:39 2023-04-26 03:49:00 0:31:21 0:18:01 0:13:20 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7251675 2023-04-25 14:22:10 2023-04-26 03:20:11 2023-04-26 04:54:38 1:34:27 1:04:40 0:29:47 smithi main centos 8.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/scrub_test} 2
pass 7251676 2023-04-25 14:22:11 2023-04-26 03:21:21 2023-04-26 03:45:18 0:23:57 0:15:11 0:08:46 smithi main rhel 8.4 rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} 1
pass 7251677 2023-04-25 14:22:11 2023-04-26 03:24:02 2023-04-26 05:38:28 2:14:26 1:43:53 0:30:33 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_nfs} 1
pass 7251678 2023-04-25 14:22:12 2023-04-26 03:24:32 2023-04-26 03:55:54 0:31:22 0:21:13 0:10:09 smithi main ubuntu 18.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} 2
pass 7251679 2023-04-25 14:22:13 2023-04-26 03:24:33 2023-04-26 05:24:26 1:59:53 1:30:20 0:29:33 smithi main centos 8.stream rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} 1
pass 7251680 2023-04-25 14:22:14 2023-04-26 03:24:33 2023-04-26 05:30:18 2:05:45 1:36:05 0:29:40 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} 2
pass 7251681 2023-04-25 14:22:15 2023-04-26 03:25:14 2023-04-26 05:22:42 1:57:28 1:25:55 0:31:33 smithi main centos 8.stream rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} 3
pass 7251682 2023-04-25 14:22:15 2023-04-26 03:27:15 2023-04-26 04:40:49 1:13:34 1:04:42 0:08:52 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/radosbench} 3
dead 7251683 2023-04-25 14:22:16 2023-04-26 03:27:15 2023-04-26 15:36:09 12:08:54 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

hit max job timeout

pass 7251684 2023-04-25 14:22:17 2023-04-26 03:27:15 2023-04-26 03:56:52 0:29:37 0:20:27 0:09:10 smithi main rhel 8.4 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251685 2023-04-25 14:22:18 2023-04-26 03:28:56 2023-04-26 04:11:40 0:42:44 0:31:55 0:10:49 smithi main ubuntu 20.04 rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251686 2023-04-25 14:22:19 2023-04-26 03:29:57 2023-04-26 04:45:21 1:15:24 0:43:24 0:32:00 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7251688 2023-04-25 14:22:19 2023-04-26 03:31:58 2023-04-26 06:07:24 2:35:26 2:05:33 0:29:53 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} 2
pass 7251690 2023-04-25 14:22:20 2023-04-26 03:34:49 2023-04-26 03:56:34 0:21:45 0:14:31 0:07:14 smithi main rhel 8.4 rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251692 2023-04-25 14:22:21 2023-04-26 03:35:00 2023-04-26 03:55:08 0:20:08 0:09:44 0:10:24 smithi main ubuntu 20.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/workunits} 2
dead 7251694 2023-04-25 14:22:22 2023-04-26 03:35:01 2023-04-26 04:41:32 1:06:31 0:36:20 0:30:11 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

{'smithi003.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'CentOS-PowerTools': Yum repo downloading error: Downloading error(s): repodata/38c2c78c3f89e4d347f07263805d33d8ae17df60ebab2e8a259218c57bcae2fb-comps-PowerTools.x86_64.xml - Cannot download, all mirrors were already tried without success", 'rc': 1, 'results': []}}

pass 7251696 2023-04-25 14:22:23 2023-04-26 03:36:12 2023-04-26 05:53:29 2:17:17 1:48:28 0:28:49 smithi main centos 8.stream rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} 2
dead 7251698 2023-04-25 14:22:23 2023-04-26 03:40:23 2023-04-26 03:44:45 0:04:22 smithi main centos 8.stream rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} 1
Failure Reason:

Error reimaging machines: 'NoneType' object has no attribute '_fields'

pass 7251700 2023-04-25 14:22:24 2023-04-26 03:44:54 2023-04-26 06:17:25 2:32:31 1:57:44 0:34:47 smithi main centos 8.stream rados/cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7251702 2023-04-25 14:22:25 2023-04-26 03:51:06 2023-04-26 07:23:39 3:32:33 3:22:42 0:09:51 smithi main ubuntu 20.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} 1
pass 7251704 2023-04-25 14:22:26 2023-04-26 04:22:36 1522 smithi main rhel 8.4 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-localized} 2
pass 7251706 2023-04-25 14:22:27 2023-04-26 03:51:28 2023-04-26 05:59:05 2:07:37 1:37:03 0:30:34 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_orch_cli} 1
pass 7251707 2023-04-25 14:22:28 2023-04-26 04:15:06 985 smithi main rhel 8.4 rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7251710 2023-04-25 14:22:28 2023-04-26 03:55:10 2023-04-26 04:15:48 0:20:38 0:10:35 0:10:03 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} 1
pass 7251712 2023-04-25 14:22:29 2023-04-26 04:37:50 1816 smithi main ubuntu 20.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
pass 7251714 2023-04-25 14:22:30 2023-04-26 03:57:01 2023-04-26 06:18:50 2:21:49 1:49:38 0:32:11 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251716 2023-04-25 14:22:31 2023-04-26 03:58:02 2023-04-26 04:18:22 0:20:20 0:09:17 0:11:03 smithi main ubuntu 20.04 rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251718 2023-04-25 14:22:32 2023-04-26 04:22:11 626 smithi main ubuntu 20.04 rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251719 2023-04-25 14:22:32 2023-04-26 04:02:04 2023-04-26 04:36:48 0:34:44 0:23:16 0:11:28 smithi main ubuntu 20.04 rados/cephadm/smoke/{0-nvme-loop distro/ubuntu_20.04 fixed-2 mon_election/classic start} 2
pass 7251721 2023-04-25 14:22:33 2023-04-26 04:11:46 2023-04-26 04:35:54 0:24:08 0:17:39 0:06:29 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7251724 2023-04-25 14:22:34 2023-04-26 04:15:08 2023-04-26 06:28:24 2:13:16 1:43:52 0:29:24 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects} 2
pass 7251726 2023-04-25 14:22:35 2023-04-26 04:18:09 2023-04-26 06:35:11 2:17:02 1:46:46 0:30:16 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_orch_cli_mon} 5
pass 7251728 2023-04-25 14:22:36 2023-04-26 04:19:40 2023-04-26 05:03:44 0:44:04 0:36:19 0:07:45 smithi main rhel 8.4 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 2
pass 7251730 2023-04-25 14:22:36 2023-04-26 04:22:21 2023-04-26 04:50:25 0:28:04 0:16:54 0:11:10 smithi main ubuntu 20.04 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} 2
pass 7251732 2023-04-25 14:22:37 2023-04-26 04:22:42 2023-04-26 05:18:27 0:55:45 0:34:59 0:20:46 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/rbd_cls} 3
fail 7251734 2023-04-25 14:22:38 2023-04-26 04:36:05 2023-04-26 05:00:24 0:24:19 smithi main centos 8.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} 2
Failure Reason:

Command failed on smithi112 with status 1: 'sudo yum install -y kernel'

pass 7251736 2023-04-25 14:22:39 2023-04-26 04:36:55 2023-04-26 05:01:43 0:24:48 0:17:28 0:07:20 smithi main rhel 8.4 rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7251738 2023-04-25 14:22:40 2023-04-26 04:38:56 2023-04-26 05:03:41 0:24:45 0:18:49 0:05:56 smithi main rhel 8.4 rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} 1
fail 7251740 2023-04-25 14:22:41 2023-04-26 04:39:37 2023-04-26 04:59:18 0:19:41 0:07:35 0:12:06 smithi main rados/cephadm/dashboard/{0-distro/ignorelist_health task/test_e2e} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=5b35a461e1c1b935b2b3fc7c43d68a58c1a41547

pass 7251742 2023-04-25 14:22:41 2023-04-26 04:40:58 2023-04-26 05:02:17 0:21:19 0:14:15 0:07:04 smithi main rhel 8.4 rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} 1
pass 7251744 2023-04-25 14:22:42 2023-04-26 04:44:20 2023-04-26 05:23:44 0:39:24 0:27:44 0:11:40 smithi main ubuntu 20.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
dead 7251746 2023-04-25 14:22:43 2023-04-26 04:48:01 2023-04-26 16:59:15 12:11:14 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
Failure Reason:

hit max job timeout

pass 7251748 2023-04-25 14:22:44 2023-04-26 04:50:32 2023-04-26 05:30:41 0:40:09 0:29:08 0:11:01 smithi main ubuntu 20.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} 2
pass 7251750 2023-04-25 14:22:45 2023-04-26 04:54:43 2023-04-26 06:14:24 1:19:41 0:45:13 0:34:28 smithi main centos 8.stream rados/cephadm/smoke/{0-nvme-loop distro/centos_8.stream_container_tools fixed-2 mon_election/connectivity start} 2
pass 7251752 2023-04-25 14:22:46 2023-04-26 04:59:25 2023-04-26 05:36:25 0:37:00 0:30:13 0:06:47 smithi main rhel 8.4 rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} 1
pass 7251754 2023-04-25 14:22:46 2023-04-26 04:59:25 2023-04-26 05:22:25 0:23:00 0:15:50 0:07:10 smithi main rhel 8.4 rados/cephadm/smoke-singlehost/{0-distro$/{rhel_8.4_container_tools_3.0} 1-start 2-services/rgw 3-final} 1
pass 7251756 2023-04-25 14:22:47 2023-04-26 05:00:26 2023-04-26 05:20:35 0:20:09 0:09:55 0:10:14 smithi main ubuntu 20.04 rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251758 2023-04-25 14:22:48 2023-04-26 05:01:27 2023-04-26 05:23:34 0:22:07 0:14:35 0:07:32 smithi main rhel 8.4 rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} 3
pass 7251760 2023-04-25 14:22:49 2023-04-26 05:03:48 2023-04-26 07:49:17 2:45:29 2:16:43 0:28:46 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7251762 2023-04-25 14:22:50 2023-04-26 05:12:25 2023-04-26 06:57:49 1:45:24 1:15:09 0:30:15 smithi main centos 8.stream rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls mon_election/classic} 2
pass 7251764 2023-04-25 14:22:50 2023-04-26 05:14:06 2023-04-26 05:40:13 0:26:07 0:10:52 0:15:15 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} 1
pass 7251766 2023-04-25 14:22:51 2023-04-26 05:18:27 2023-04-26 07:24:41 2:06:14 1:35:20 0:30:54 smithi main centos 8.stream rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} 1
pass 7251768 2023-04-25 14:22:52 2023-04-26 05:20:38 2023-04-26 05:47:13 0:26:35 0:13:04 0:13:31 smithi main ubuntu 20.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
pass 7251770 2023-04-25 14:22:53 2023-04-26 05:22:49 2023-04-26 07:34:16 2:11:27 1:41:08 0:30:19 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} 2
pass 7251772 2023-04-25 14:22:54 2023-04-26 05:23:40 2023-04-26 05:58:37 0:34:57 0:28:05 0:06:52 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
fail 7251774 2023-04-25 14:22:55 2023-04-26 05:24:31 2023-04-26 07:32:48 2:08:17 1:37:44 0:30:33 smithi main centos 8.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} 2
Failure Reason:

"2023-04-26T07:28:05.415999+0000 mon.a (mon.0) 473 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7251776 2023-04-25 14:22:55 2023-04-26 05:25:52 2023-04-26 07:31:29 2:05:37 1:32:20 0:33:17 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_adoption} 1
pass 7251778 2023-04-25 14:22:56 2023-04-26 05:30:44 2023-04-26 05:58:29 0:27:45 0:11:07 0:16:38 smithi main ubuntu 20.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} 2
pass 7251780 2023-04-25 14:22:57 2023-04-26 06:26:03 2228 smithi main rhel 8.4 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} 2
pass 7251781 2023-04-25 14:22:58 2023-04-26 05:42:57 2023-04-26 06:14:33 0:31:36 0:17:40 0:13:56 smithi main ubuntu 18.04 rados/cephadm/osds/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7251783 2023-04-25 14:22:59 2023-04-26 05:46:58 2023-04-26 06:23:05 0:36:07 0:25:05 0:11:02 smithi main ubuntu 20.04 rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251785 2023-04-25 14:22:59 2023-04-26 05:47:19 2023-04-26 06:17:31 0:30:12 0:19:47 0:10:25 smithi main ubuntu 18.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7251787 2023-04-25 14:23:00 2023-04-26 05:48:09 2023-04-26 06:52:29 1:04:20 0:51:22 0:12:58 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} 3
pass 7251789 2023-04-25 14:23:01 2023-04-26 06:07:27 2023-04-26 08:04:48 1:57:21 1:28:03 0:29:18 smithi main centos 8.stream rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} 1
pass 7251791 2023-04-25 14:23:02 2023-04-26 06:13:28 2023-04-26 08:25:23 2:11:55 1:42:51 0:29:04 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects} 2
pass 7251793 2023-04-25 14:23:03 2023-04-26 06:14:29 2023-04-26 08:21:32 2:07:03 1:33:03 0:34:00 smithi main centos 8.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} 3
fail 7251795 2023-04-25 14:23:03 2023-04-26 08:20:53 5682 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/classic task/test_cephadm} 1
Failure Reason:

SELinux denials found on ubuntu@smithi188.front.sepia.ceph.com: ['type=AVC msg=audit(1682497140.009:18694): avc: denied { ioctl } for pid=154298 comm="iptables" path="/var/lib/containers/storage/overlay/262b092bfa64b44d47e5bf6a32463bedb8a8f5df5ebaf863ebe054e24f14d75d/merged" dev="overlay" ino=3412108 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']

pass 7251797 2023-04-25 14:23:04 2023-04-26 06:17:41 2023-04-26 06:56:24 0:38:43 0:30:29 0:08:14 smithi main rhel 8.4 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} 2
pass 7251799 2023-04-25 14:23:05 2023-04-26 06:26:13 2023-04-26 06:55:48 0:29:35 0:20:07 0:09:28 smithi main rhel 8.4 rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_3.0 fixed-2 mon_election/classic start} 2
pass 7251801 2023-04-25 14:23:06 2023-04-26 06:35:13 2023-04-26 09:08:28 2:33:15 2:04:52 0:28:23 smithi main centos 8.stream rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} 1
pass 7251804 2023-04-25 14:23:07 2023-04-26 07:17:52 1550 smithi main ubuntu 20.04 rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7251806 2023-04-25 14:23:07 2023-04-26 06:38:05 2023-04-26 08:57:02 2:18:57 1:45:50 0:33:07 smithi main centos 8.stream rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} 2
pass 7251808 2023-04-25 14:23:08 2023-04-26 06:42:06 2023-04-26 07:25:35 0:43:29 0:35:21 0:08:08 smithi main rhel 8.4 rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} 2
pass 7251810 2023-04-25 14:23:09 2023-04-26 06:44:38 2023-04-26 07:55:25 1:10:47 0:42:13 0:28:34 smithi main centos 8.stream rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools mon_election/connectivity task/test_cephadm_repos} 1
pass 7251812 2023-04-25 14:23:10 2023-04-26 06:46:28 2023-04-26 07:34:41 0:48:13 0:40:07 0:08:06 smithi main rhel 8.4 rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} 2
pass 7251814 2023-04-25 14:23:11 2023-04-26 07:11:52 2023-04-26 09:17:06 2:05:14 1:29:55 0:35:19 smithi main centos 8.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} 2
pass 7251816 2023-04-25 14:23:12 2023-04-26 07:24:45 2023-04-26 08:17:17 0:52:32 0:37:39 0:14:53 smithi main ubuntu 18.04 rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} 3
pass 7251818 2023-04-25 14:23:12 2023-04-26 07:29:26 2023-04-26 11:02:43 3:33:17 3:03:42 0:29:35 smithi main centos 8.stream rados/standalone/{supported-random-distro$/{centos_8} workloads/scrub} 1
dead 7251820 2023-04-25 14:23:13 2023-04-26 07:31:37 2023-04-26 19:41:24 12:09:47 smithi main centos 8.stream rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

hit max job timeout

pass 7251822 2023-04-25 14:23:14 2023-04-26 07:34:19 2023-04-26 08:12:20 0:38:01 0:24:50 0:13:11 smithi main ubuntu 20.04 rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} 1
pass 7251824 2023-04-25 14:23:15 2023-04-26 07:37:40 2023-04-26 08:05:41 0:28:01 0:16:12 0:11:49 smithi main ubuntu 20.04 rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} 1
pass 7251826 2023-04-25 14:23:16 2023-04-26 07:39:51 2023-04-26 09:53:28 2:13:37 1:43:06 0:30:31 smithi main centos 8.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
fail 7251828 2023-04-25 14:23:16 2023-04-26 07:40:12 2023-04-26 08:14:32 0:34:20 smithi main centos 8.stream rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

Command failed on smithi178 with status 1: 'sudo yum install -y kernel'

pass 7251830 2023-04-25 14:23:17 2023-04-26 07:47:03 2023-04-26 10:10:40 2:23:37 1:53:56 0:29:41 smithi main centos 8.stream rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
fail 7251832 2023-04-25 14:23:18 2023-04-26 07:47:54 2023-04-26 08:13:11 0:25:17 0:11:27 0:13:50 smithi main ubuntu 18.04 rados/rook/smoke/{0-distro/ubuntu_18.04 0-kubeadm 1-rook 2-workload/radosbench 3-final cluster/3-node k8s/1.21 net/calico rook/1.6.2} 3
Failure Reason:

Command failed on smithi171 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7251834 2023-04-25 14:23:19 2023-04-26 07:54:16 2023-04-26 08:44:43 0:50:27 0:42:56 0:07:31 smithi main rhel 8.4 rados/cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2