Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7382286 2023-08-28 16:10:36 2023-08-28 17:57:25 2023-08-28 18:30:19 0:32:54 0:25:29 0:07:25 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

Command failed on smithi031 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.31:/foobucket /mnt/foo'"

pass 7382287 2023-08-28 16:10:37 2023-08-28 17:57:26 2023-08-28 18:26:34 0:29:08 0:17:41 0:11:27 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7382288 2023-08-28 16:10:37 2023-08-28 17:57:26 2023-08-28 18:28:09 0:30:43 0:21:48 0:08:55 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7382289 2023-08-28 16:10:38 2023-08-28 17:57:26 2023-08-28 18:25:54 0:28:28 0:17:47 0:10:41 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
fail 7382290 2023-08-28 16:10:39 2023-08-28 17:57:27 2023-08-28 18:30:31 0:33:04 0:20:54 0:12:10 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Command failed on smithi037 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55c633ce-45cf-11ee-9b8b-d70336b2926f -- bash -c 'while ! ceph orch ls | grep nfs | grep 2/2 ; do sleep 1 ; done'"

pass 7382291 2023-08-28 16:10:39 2023-08-28 17:57:27 2023-08-28 18:28:09 0:30:42 0:20:10 0:10:32 smithi main centos 8.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli} 2
pass 7382292 2023-08-28 16:10:40 2023-08-28 17:57:27 2023-08-28 18:52:33 0:55:06 0:43:25 0:11:41 smithi main centos 8.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} 3
pass 7382293 2023-08-28 16:10:40 2023-08-28 17:57:28 2023-08-28 18:29:30 0:32:02 0:20:22 0:11:40 smithi main ubuntu 20.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} 1
pass 7382294 2023-08-28 16:10:41 2023-08-28 17:57:28 2023-08-28 18:26:32 0:29:04 0:18:14 0:10:50 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
dead 7382295 2023-08-28 16:10:42 2023-08-28 17:57:28 2023-08-28 18:15:41 0:18:13 0:09:56 0:08:17 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

["dd: failed to open '/home/ubuntu/.ansible/tmp/ansible-tmp-1693245853.2524476-26393-9428471280628/AnsiballZ_user.py': No such file or directory", 'failed to transfer file to /home/teuthworker/.ansible/tmp/ansible-local-23711q6nmx9w7/tmph9k0x4lu /home/ubuntu/.ansible/tmp/ansible-tmp-1693245853.2524476-26393-9428471280628/AnsiballZ_user.py:']

fail 7382296 2023-08-28 16:10:42 2023-08-28 17:57:29 2023-08-28 18:06:07 0:08:38 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

Stale jobs detected, aborting.

pass 7382297 2023-08-28 16:10:43 2023-08-28 17:57:29 2023-08-28 18:38:27 0:40:58 0:26:31 0:14:27 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7382298 2023-08-28 16:10:44 2023-08-28 17:57:29 2023-08-28 19:18:40 1:21:11 1:06:50 0:14:21 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7382299 2023-08-28 16:10:44 2023-08-28 17:57:30 2023-08-28 18:50:01 0:52:31 0:39:35 0:12:56 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_nfs} 1
pass 7382300 2023-08-28 16:10:45 2023-08-28 17:57:30 2023-08-28 18:52:09 0:54:39 0:42:15 0:12:24 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
pass 7382301 2023-08-28 16:10:46 2023-08-28 17:57:30 2023-08-28 18:31:24 0:33:54 0:20:10 0:13:44 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7382302 2023-08-28 16:10:46 2023-08-28 17:57:31 2023-08-28 18:51:16 0:53:45 0:39:12 0:14:33 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi155 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 803d7f7c-45cf-11ee-9b8b-d70336b2926f -- bash -c 'ceph orch upgrade ls'"

pass 7382303 2023-08-28 16:10:47 2023-08-28 17:57:31 2023-08-28 18:27:40 0:30:09 0:21:13 0:08:56 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7382304 2023-08-28 16:10:48 2023-08-28 17:57:31 2023-08-28 18:27:11 0:29:40 0:20:49 0:08:51 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7382305 2023-08-28 16:10:48 2023-08-28 17:57:32 2023-08-28 18:28:08 0:30:36 0:17:45 0:12:51 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} 1
pass 7382306 2023-08-28 16:10:49 2023-08-28 17:57:32 2023-08-28 18:27:58 0:30:26 0:18:41 0:11:45 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7382307 2023-08-28 16:10:50 2023-08-28 17:57:32 2023-08-28 18:43:21 0:45:49 0:34:17 0:11:32 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382308 2023-08-28 16:10:50 2023-08-28 17:57:33 2023-08-28 18:46:38 0:49:05 0:35:07 0:13:58 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7382309 2023-08-28 16:10:51 2023-08-28 17:57:33 2023-08-28 18:48:59 0:51:26 0:39:57 0:11:29 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
pass 7382310 2023-08-28 16:10:52 2023-08-28 17:57:34 2023-08-28 18:29:43 0:32:09 0:23:03 0:09:06 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7382311 2023-08-28 16:10:52 2023-08-28 17:57:34 2023-08-28 18:40:54 0:43:20 0:29:51 0:13:29 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} 5
pass 7382312 2023-08-28 16:10:53 2023-08-28 17:57:34 2023-08-28 18:28:58 0:31:24 0:21:41 0:09:43 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7382313 2023-08-28 16:10:54 2023-08-28 17:57:35 2023-08-28 18:39:17 0:41:42 0:26:28 0:15:14 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7382314 2023-08-28 16:10:54 2023-08-28 17:57:35 2023-08-28 18:29:18 0:31:43 0:18:18 0:13:25 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
pass 7382315 2023-08-28 16:10:55 2023-08-28 17:57:35 2023-08-28 18:31:40 0:34:05 0:22:47 0:11:18 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7382316 2023-08-28 16:10:56 2023-08-28 17:57:36 2023-08-28 18:41:50 0:44:14 0:29:14 0:15:00 smithi main ubuntu 20.04 orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7382317 2023-08-28 16:10:56 2023-08-28 17:57:36 2023-08-28 19:30:14 1:32:38 0:24:11 1:08:27 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7382318 2023-08-28 16:10:57 2023-08-28 17:57:36 2023-08-28 19:30:23 1:32:47 0:18:39 1:14:08 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7382319 2023-08-28 16:10:58 2023-08-28 17:57:37 2023-08-28 18:51:53 0:54:16 0:40:36 0:13:40 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7382320 2023-08-28 16:10:58 2023-08-28 17:57:37 2023-08-28 18:46:38 0:49:01 0:34:21 0:14:40 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382321 2023-08-28 16:10:59 2023-08-28 17:57:38 2023-08-28 19:26:14 1:28:36 0:17:34 1:11:02 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7382322 2023-08-28 16:10:59 2023-08-28 17:57:38 2023-08-28 18:58:38 1:01:00 0:46:33 0:14:27 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi084 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cd92160c-45cf-11ee-9b8b-d70336b2926f -- bash -c 'ceph orch upgrade ls'"

pass 7382323 2023-08-28 16:11:00 2023-08-28 17:57:39 2023-08-28 18:45:37 0:47:58 0:37:10 0:10:48 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7382324 2023-08-28 16:11:01 2023-08-28 17:57:39 2023-08-28 18:38:00 0:40:21 0:29:21 0:11:00 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7382325 2023-08-28 16:11:01 2023-08-28 17:57:39 2023-08-28 18:54:53 0:57:14 0:41:30 0:15:44 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7382326 2023-08-28 16:11:02 2023-08-28 17:57:40 2023-08-28 18:30:37 0:32:57 0:17:39 0:15:18 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7382327 2023-08-28 16:11:03 2023-08-28 17:57:40 2023-08-28 18:29:15 0:31:35 0:20:32 0:11:03 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7382328 2023-08-28 16:11:03 2023-08-28 17:57:40 2023-08-28 18:30:41 0:33:01 0:17:33 0:15:28 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7382329 2023-08-28 16:11:04 2023-08-28 17:57:41 2023-08-28 19:26:35 1:28:54 0:20:03 1:08:51 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7382330 2023-08-28 16:11:05 2023-08-28 17:57:41 2023-08-28 18:57:33 0:59:52 0:48:52 0:11:00 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7382331 2023-08-28 16:11:05 2023-08-28 17:57:42 2023-08-28 18:57:13 0:59:31 0:47:06 0:12:25 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7382332 2023-08-28 16:11:06 2023-08-28 18:02:53 2023-08-28 18:30:32 0:27:39 0:21:18 0:06:21 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption} 1
pass 7382333 2023-08-28 16:11:06 2023-08-28 18:03:23 2023-08-28 18:33:17 0:29:54 0:21:50 0:08:04 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7382334 2023-08-28 16:11:07 2023-08-28 18:03:33 2023-08-28 18:49:14 0:45:41 0:35:32 0:10:09 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382335 2023-08-28 16:11:08 2023-08-28 18:04:14 2023-08-28 18:32:16 0:28:02 0:18:20 0:09:42 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_ca_signed_key} 2
pass 7382336 2023-08-28 16:11:08 2023-08-28 18:04:14 2023-08-28 18:49:22 0:45:08 0:33:55 0:11:13 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7382337 2023-08-28 16:11:09 2023-08-28 18:05:05 2023-08-28 18:35:05 0:30:00 0:20:28 0:09:32 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7382338 2023-08-28 16:11:10 2023-08-28 18:05:05 2023-08-28 18:36:42 0:31:37 0:25:40 0:05:57 smithi main rhel 8.6 orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli} 2
pass 7382339 2023-08-28 16:11:10 2023-08-28 18:05:15 2023-08-28 18:30:04 0:24:49 0:15:52 0:08:57 smithi main centos 8.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} 1
pass 7382340 2023-08-28 16:11:11 2023-08-28 18:05:16 2023-08-28 18:30:47 0:25:31 0:14:31 0:11:00 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
fail 7382341 2023-08-28 16:11:12 2023-08-28 18:05:26 2023-08-28 18:26:39 0:21:13 0:11:43 0:09:30 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
Failure Reason:

['Failed to connect to the host via ssh: ssh: connect to host smithi002.front.sepia.ceph.com port 22: No route to host']

fail 7382342 2023-08-28 16:11:12 2023-08-28 18:06:17 2023-08-28 18:55:07 0:48:50 0:29:02 0:19:48 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

fail 7382343 2023-08-28 16:11:13 2023-08-28 18:15:41 2023-08-28 18:27:34 0:11:53 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
pass 7382344 2023-08-28 16:11:13 2023-08-28 18:15:52 2023-08-28 18:53:50 0:37:58 0:18:09 0:19:49 smithi main ubuntu 20.04 orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7382345 2023-08-28 16:11:14 2023-08-28 18:25:56 2023-08-28 18:51:37 0:25:41 0:16:03 0:09:38 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7382346 2023-08-28 16:11:15 2023-08-28 18:26:37 2023-08-28 18:55:31 0:28:54 0:19:16 0:09:38 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm} 1
pass 7382347 2023-08-28 16:11:15 2023-08-28 18:26:37 2023-08-28 19:03:48 0:37:11 0:26:42 0:10:29 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
dead 7382348 2023-08-28 16:11:16 2023-08-28 18:26:37 2023-08-28 18:32:21 0:05:44 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

Error reimaging machines: Expected smithi002's OS to be centos 8 but found rhel 8.6

pass 7382349 2023-08-28 16:11:17 2023-08-28 18:26:48 2023-08-28 19:08:22 0:41:34 0:31:50 0:09:44 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382350 2023-08-28 16:11:17 2023-08-28 18:27:18 2023-08-28 18:46:24 0:19:06 0:09:27 0:09:39 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_cephadm_repos} 1
fail 7382351 2023-08-28 16:11:18 2023-08-28 18:27:38 2023-08-28 20:36:47 2:09:09 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

machine smithi002.front.sepia.ceph.com is locked by scheduled_zack@teuthology, not scheduled_adking@teuthology

pass 7382352 2023-08-28 16:11:19 2023-08-28 18:27:49 2023-08-28 19:24:06 0:56:17 0:47:01 0:09:16 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7382353 2023-08-28 16:11:19 2023-08-28 18:27:59 2023-08-28 19:15:25 0:47:26 0:37:15 0:10:11 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
fail 7382354 2023-08-28 16:11:20 2023-08-28 18:28:10 2023-08-28 18:55:13 0:27:03 0:20:24 0:06:39 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
Failure Reason:

Command failed on smithi081 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.81:/foobucket /mnt/foo'"

fail 7382355 2023-08-28 16:11:20 2023-08-28 18:28:10 2023-08-28 19:17:11 0:49:01 0:39:55 0:09:06 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

'2023-08-28T19:08:51.611735+00:00 smithi070 kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: ' in syslog

pass 7382356 2023-08-28 16:11:21 2023-08-28 18:28:10 2023-08-28 18:54:01 0:25:51 0:15:26 0:10:25 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7382357 2023-08-28 16:11:22 2023-08-28 18:29:01 2023-08-28 18:54:35 0:25:34 0:19:45 0:05:49 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7382358 2023-08-28 16:11:22 2023-08-28 18:29:21 2023-08-28 18:58:40 0:29:19 0:18:01 0:11:18 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
fail 7382359 2023-08-28 16:11:23 2023-08-28 18:29:22 2023-08-28 19:03:33 0:34:11 0:23:48 0:10:23 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

Command failed on smithi107 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.107:/foouser /mnt/foo'"

pass 7382360 2023-08-28 16:11:24 2023-08-28 18:29:22 2023-08-28 19:04:48 0:35:26 0:28:08 0:07:18 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7382361 2023-08-28 16:11:24 2023-08-28 18:29:53 2023-08-28 19:11:09 0:41:16 0:31:45 0:09:31 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382362 2023-08-28 16:11:25 2023-08-28 18:30:14 2023-08-28 18:57:03 0:26:49 0:18:02 0:08:47 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7382363 2023-08-28 16:11:25 2023-08-28 18:30:24 2023-08-28 18:56:15 0:25:51 0:14:32 0:11:19 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7382364 2023-08-28 16:11:26 2023-08-28 18:30:35 2023-08-28 19:21:09 0:50:34 0:38:15 0:12:19 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
fail 7382365 2023-08-28 16:11:27 2023-08-28 18:30:45 2023-08-28 19:07:49 0:37:04 0:26:48 0:10:16 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi115 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4bfec7ee-45d3-11ee-9b8b-d70336b2926f -- bash -c 'ceph orch upgrade ls'"

pass 7382366 2023-08-28 16:11:27 2023-08-28 18:30:45 2023-08-28 19:11:36 0:40:51 0:35:16 0:05:35 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_nfs} 1
pass 7382367 2023-08-28 16:11:28 2023-08-28 18:30:56 2023-08-28 19:06:37 0:35:41 0:29:39 0:06:02 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7382368 2023-08-28 16:11:29 2023-08-28 18:30:57 2023-08-28 18:58:46 0:27:49 0:16:56 0:10:53 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7382369 2023-08-28 16:11:29 2023-08-28 18:31:28 2023-08-28 18:57:20 0:25:52 0:15:26 0:10:26 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7382370 2023-08-28 16:11:30 2023-08-28 18:31:48 2023-08-28 18:57:14 0:25:26 0:18:37 0:06:49 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7382371 2023-08-28 16:11:30 2023-08-28 18:32:18 2023-08-28 19:02:24 0:30:06 0:19:00 0:11:06 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli} 1
fail 7382372 2023-08-28 16:11:31 2023-08-28 18:32:39 2023-08-28 18:56:08 0:23:29 0:17:10 0:06:19 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

Command failed on smithi033 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7ad454e2787b18b20fbc6be4ced6d1b76d089c0a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8121935c-45d3-11ee-9b8b-d70336b2926f -- ceph osd stat -f json'

pass 7382373 2023-08-28 16:11:32 2023-08-28 18:33:20 2023-08-28 19:21:20 0:48:00 0:37:52 0:10:08 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
pass 7382374 2023-08-28 16:11:32 2023-08-28 18:33:20 2023-08-28 19:29:22 0:56:02 0:43:57 0:12:05 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7382375 2023-08-28 16:11:33 2023-08-28 18:35:11 2023-08-28 19:01:40 0:26:29 0:18:17 0:08:12 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7382376 2023-08-28 16:11:34 2023-08-28 18:36:22 2023-08-28 19:13:46 0:37:24 0:24:36 0:12:48 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon} 5
pass 7382377 2023-08-28 16:11:34 2023-08-28 18:38:03 2023-08-28 19:24:06 0:46:03 0:34:44 0:11:19 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7382378 2023-08-28 16:11:35 2023-08-28 18:38:33 2023-08-28 19:09:37 0:31:04 0:18:40 0:12:24 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7382379 2023-08-28 16:11:35 2023-08-28 18:39:24 2023-08-28 19:17:21 0:37:57 0:24:43 0:13:14 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7382380 2023-08-28 16:11:36 2023-08-28 18:41:05 2023-08-28 19:06:40 0:25:35 0:19:23 0:06:12 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2