Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 7351886 2023-07-25 19:57:38 2023-07-25 19:59:10 2023-07-25 20:03:51 0:04:41 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

Error reimaging machines: Expected smithi104's OS to be centos 8 but found ubuntu 22.04

dead 7351887 2023-07-25 19:57:39 2023-07-25 19:59:33 2023-07-25 20:03:06 0:03:33 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi123

dead 7351888 2023-07-25 19:57:41 2023-07-25 19:59:33 2023-07-25 20:14:09 0:14:36 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

Error reimaging machines: Cannot connect to remote host smithi101

pass 7351889 2023-07-25 19:57:57 2023-07-25 19:59:44 2023-07-25 20:44:20 0:44:36 0:35:09 0:09:27 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_nfs} 1
dead 7351890 2023-07-25 19:58:13 2023-07-25 19:59:54 2023-07-25 20:03:48 0:03:54 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Error reimaging machines: Expected smithi162's OS to be centos 8 but found centos 9

dead 7351891 2023-07-25 19:58:22 2023-07-25 20:00:00 2023-07-25 20:05:00 0:05:00 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Error reimaging machines: Expected smithi161's OS to be centos 8 but found ubuntu 20.04

pass 7351892 2023-07-25 19:58:27 2023-07-25 20:00:00 2023-07-25 21:28:34 1:28:34 0:17:36 1:10:58 smithi main ubuntu 20.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} 2
dead 7351893 2023-07-25 19:58:28 2023-07-25 20:00:00 2023-07-25 20:14:09 0:14:09 smithi main centos 8.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} 3
Failure Reason:

Error reimaging machines: Cannot connect to remote host smithi050

pass 7351894 2023-07-25 19:58:44 2023-07-25 20:00:01 2023-07-25 21:28:27 1:28:26 0:17:56 1:10:30 smithi main ubuntu 20.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} 1
pass 7351897 2023-07-25 19:59:00 2023-07-25 20:01:21 2023-07-25 20:29:53 0:28:32 0:14:57 0:13:35 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7351899 2023-07-25 19:59:11 2023-07-25 20:03:53 2023-07-25 21:08:46 1:04:53 0:45:34 0:19:19 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi060 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 11fef29e-2b2a-11ee-9b35-001a4aab830c -- bash -c 'ceph orch upgrade ls'"

pass 7351901 2023-07-25 19:59:27 2023-07-25 20:04:19 2023-07-25 20:45:27 0:41:08 0:30:43 0:10:25 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7351903 2023-07-25 19:59:32 2023-07-25 20:42:52 1675 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli} 1
Failure Reason:

Test failure: test_daemon_restart (tasks.cephadm_cases.test_cli.TestCephadmCLI)

pass 7351905 2023-07-25 19:59:43 2023-07-25 20:06:16 2023-07-25 20:33:58 0:27:42 0:18:18 0:09:24 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7351906 2023-07-25 19:59:59 2023-07-25 20:06:16 2023-07-25 20:33:28 0:27:12 0:19:07 0:08:05 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7351908 2023-07-25 20:00:00 2023-07-25 21:03:30 2846 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
pass 7351910 2023-07-25 20:00:01 2023-07-25 20:07:03 2023-07-25 21:06:41 0:59:38 0:45:56 0:13:42 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7351912 2023-07-25 20:00:12 2023-07-25 20:10:07 2023-07-25 22:07:43 1:57:36 0:41:03 1:16:33 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli_mon} 5
pass 7351914 2023-07-25 20:00:28 2023-07-25 20:11:14 2023-07-25 20:48:18 0:37:04 0:21:49 0:15:15 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7351916 2023-07-25 20:00:36 2023-07-25 20:12:39 2023-07-25 20:50:36 0:37:57 0:25:13 0:12:44 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7351918 2023-07-25 20:00:47 2023-07-25 20:14:02 2023-07-25 20:47:49 0:33:47 0:18:08 0:15:39 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7351920 2023-07-25 20:00:53 2023-07-25 20:15:10 2023-07-25 20:45:27 0:30:17 0:14:53 0:15:24 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7351922 2023-07-25 20:01:04 2023-07-25 20:15:46 2023-07-25 21:08:38 0:52:52 0:37:35 0:15:17 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7351924 2023-07-25 20:01:15 2023-07-25 20:17:46 2023-07-25 20:49:26 0:31:40 0:16:14 0:15:26 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7351927 2023-07-25 20:01:20 2023-07-25 21:05:05 1915 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7351929 2023-07-25 20:01:37 2023-07-25 20:22:07 2023-07-25 21:04:08 0:42:01 0:25:00 0:17:01 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_set_mon_crush_locations} 3
pass 7351930 2023-07-25 20:01:47 2023-07-25 20:25:14 2023-07-25 21:05:58 0:40:44 0:26:07 0:14:37 smithi main ubuntu 20.04 orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7351933 2023-07-25 20:01:54 2023-07-25 20:58:56 1028 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7351935 2023-07-25 20:01:56 2023-07-25 21:43:58 3696 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
dead 7351937 2023-07-25 20:02:06 2023-07-25 20:30:43 2023-07-25 20:31:42 0:00:59 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
Failure Reason:

Error reimaging machines: 500 Server Error: Internal Server Error for url: http://fog.front.sepia.ceph.com/fog/host/113/task

pass 7351939 2023-07-25 20:02:17 2023-07-25 21:02:31 1392 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption} 1
pass 7351941 2023-07-25 20:02:20 2023-07-25 21:17:23 2127 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7351943 2023-07-25 20:02:26 2023-07-25 20:33:06 2023-07-25 21:17:01 0:43:55 0:35:36 0:08:19 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm} 1
pass 7351945 2023-07-25 20:02:31 2023-07-25 20:33:44 2023-07-25 21:18:03 0:44:19 0:33:04 0:11:15 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7351946 2023-07-25 20:02:37 2023-07-25 21:00:26 807 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
fail 7351949 2023-07-25 20:02:43 2023-07-25 20:34:49 2023-07-25 21:24:41 0:49:52 0:36:07 0:13:45 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi005 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 462193bc-2b2d-11ee-9b35-001a4aab830c -- bash -c 'ceph orch upgrade ls'"

pass 7351950 2023-07-25 20:02:54 2023-07-25 21:05:17 1333 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
dead 7351953 2023-07-25 20:03:00 2023-07-25 20:36:24 2023-07-25 20:40:03 0:03:39 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

Error reimaging machines: Failed to power on smithi089

fail 7351955 2023-07-25 20:03:06 2023-07-25 20:38:01 2023-07-25 20:49:57 0:11:56 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
Failure Reason:

Command failed on smithi155 with status 100: 'sudo apt-get clean'

dead 7351956 2023-07-25 20:03:22 2023-07-25 20:39:36 2023-07-25 20:53:44 0:14:08 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

SSH connection to smithi155 was lost: 'sudo apt-get update'

pass 7351959 2023-07-25 20:03:31 2023-07-25 21:11:08 1146 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7351962 2023-07-25 20:03:37 2023-07-25 20:46:15 2023-07-25 21:39:50 0:53:35 0:41:09 0:12:26 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7351964 2023-07-25 20:03:43 2023-07-25 21:20:49 988 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
dead 7351966 2023-07-25 20:03:53 2023-07-25 20:52:43 2023-07-25 21:02:06 0:09:23 0:03:12 0:06:11 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

{"Creating volume group 'vg_nvme' failed", "Run `vgcreate --help' for more information.", '/dev/vg_nvme: already exists in filesystem'}

pass 7351967 2023-07-25 20:03:58 2023-07-25 20:53:04 2023-07-25 21:47:10 0:54:06 0:39:34 0:14:32 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7351969 2023-07-25 20:04:03 2023-07-25 20:55:40 2023-07-25 21:26:22 0:30:42 0:19:16 0:11:26 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
pass 7351972 2023-07-25 20:04:09 2023-07-25 21:32:56 1310 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7351974 2023-07-25 20:04:19 2023-07-25 20:59:31 2023-07-25 21:50:10 0:50:39 0:37:08 0:13:31 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7351975 2023-07-25 20:04:30 2023-07-25 21:00:12 2023-07-25 21:42:11 0:41:59 0:26:15 0:15:44 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7351977 2023-07-25 20:04:40 2023-07-25 21:00:49 2023-07-25 21:49:25 0:48:36 0:36:29 0:12:07 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} 1
pass 7351979 2023-07-25 20:04:56 2023-07-25 21:04:10 2023-07-25 21:50:00 0:45:50 0:34:30 0:11:20 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7351982 2023-07-25 20:05:12 2023-07-25 21:40:06 1496 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} 1
pass 7351983 2023-07-25 20:05:18 2023-07-25 21:06:36 2023-07-25 21:34:26 0:27:50 0:14:56 0:12:54 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7351985 2023-07-25 20:05:34 2023-07-25 21:06:45 2023-07-25 21:37:45 0:31:00 0:21:32 0:09:28 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7351987 2023-07-25 20:05:40 2023-07-25 21:06:56 2023-07-25 21:37:45 0:30:49 0:15:48 0:15:01 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7351989 2023-07-25 20:05:46 2023-07-25 21:09:07 2023-07-25 22:03:48 0:54:41 0:41:14 0:13:27 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7351991 2023-07-25 20:05:50 2023-07-25 22:10:02 3147 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7351994 2023-07-25 20:06:01 2023-07-25 21:12:52 2023-07-25 22:03:56 0:51:04 0:38:35 0:12:29 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
pass 7351996 2023-07-25 20:06:07 2023-07-25 21:16:14 2023-07-25 21:45:19 0:29:05 0:17:42 0:11:23 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7351998 2023-07-25 20:06:12 2023-07-25 21:17:43 2023-07-25 22:02:57 0:45:14 0:34:28 0:10:46 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7351999 2023-07-25 20:06:15 2023-07-25 21:17:58 2023-07-25 21:46:17 0:28:19 0:16:39 0:11:40 smithi main ubuntu 20.04 orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7352001 2023-07-25 20:06:21 2023-07-25 21:18:20 2023-07-25 21:45:32 0:27:12 0:18:39 0:08:33 smithi main rhel 8.6 orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} 1
pass 7352003 2023-07-25 20:06:32 2023-07-25 21:18:31 2023-07-25 21:48:08 0:29:37 0:14:31 0:15:06 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
fail 7352005 2023-07-25 20:06:43 2023-07-25 21:19:54 2023-07-25 22:20:10 1:00:16 0:45:54 0:14:22 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi067 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6534a2c-2b33-11ee-9b35-001a4aab830c -- bash -c 'ceph orch upgrade ls'"

pass 7352009 2023-07-25 20:06:55 2023-07-25 21:21:24 2023-07-25 22:07:30 0:46:06 0:33:31 0:12:35 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
dead 7352013 2023-07-25 20:07:02 2023-07-25 21:29:18 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

Error reimaging machines: Failed to power on smithi110

pass 7352016 2023-07-25 20:07:03 2023-07-25 21:25:31 2023-07-25 21:57:53 0:32:22 0:22:35 0:09:47 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} 2
dead 7352019 2023-07-25 20:07:14 2023-07-25 21:32:22 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

Error reimaging machines: Expected smithi159's OS to be centos 8 but found ubuntu 20.04

dead 7352024 2023-07-25 20:07:20 2023-07-25 21:29:54 2023-07-25 21:33:47 0:03:53 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
Failure Reason:

Error reimaging machines: Expected smithi155's OS to be centos 8 but found ubuntu 20.04

pass 7352026 2023-07-25 20:07:36 2023-07-25 21:32:46 2023-07-25 22:00:46 0:28:00 0:17:22 0:10:38 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} 2
fail 7352030 2023-07-25 20:07:41 2023-07-25 22:11:39 1681 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

timeout expired in wait_for_all_osds_up

pass 7352033 2023-07-25 20:07:48 2023-07-25 22:06:58 1309 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7352037 2023-07-25 20:07:58 2023-07-25 21:38:01 2023-07-25 22:28:42 0:50:41 0:43:11 0:07:30 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7352038 2023-07-25 20:08:04 2023-07-25 21:38:01 2023-07-25 22:32:36 0:54:35 0:43:08 0:11:27 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7352040 2023-07-25 20:08:15 2023-07-25 21:39:32 2023-07-25 22:06:01 0:26:29 0:16:08 0:10:21 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} 1
pass 7352043 2023-07-25 20:08:21 2023-07-25 22:55:07 2023-07-25 23:42:30 0:47:23 0:35:33 0:11:50 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7352045 2023-07-25 20:08:27 2023-07-25 23:32:52 1747 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7352046 2023-07-25 20:08:33 2023-07-25 23:28:35 1544 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} 1
pass 7352049 2023-07-25 20:08:36 2023-07-25 23:29:36 1187 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7352051 2023-07-25 20:08:47 2023-07-25 22:59:00 2023-07-25 23:48:51 0:49:51 0:39:00 0:10:51 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7352052 2023-07-25 20:08:52 2023-07-25 23:00:31 2023-07-25 23:43:08 0:42:37 0:35:34 0:07:03 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7352054 2023-07-25 20:09:02 2023-07-25 23:01:02 2023-07-25 23:20:29 0:19:27 0:11:48 0:07:39 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} 1
pass 7352055 2023-07-25 20:09:08 2023-07-25 23:01:02 2023-07-25 23:26:56 0:25:54 0:15:25 0:10:29 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7352057 2023-07-25 20:09:14 2023-07-25 23:02:03 2023-07-25 23:30:55 0:28:52 0:21:06 0:07:46 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7352059 2023-07-25 20:09:30 2023-07-25 23:03:04 2023-07-26 00:10:41 1:07:37 0:59:58 0:07:39 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7352061 2023-07-25 20:09:36 2023-07-25 23:03:35 2023-07-26 00:04:09 1:00:34 0:47:14 0:13:20 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7352063 2023-07-25 20:09:51 2023-07-25 23:35:05 1053 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
pass 7352066 2023-07-25 20:09:57 2023-07-25 23:08:18 2023-07-25 23:53:58 0:45:40 0:33:29 0:12:11 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7352067 2023-07-25 20:10:03 2023-07-25 23:08:58 2023-07-25 23:40:46 0:31:48 0:13:27 0:18:21 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
fail 7352068 2023-07-25 20:10:14 2023-07-25 23:11:59 2023-07-26 00:00:14 0:48:15 0:36:02 0:12:13 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

Command failed on smithi102 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea6a7c4e-2b42-11ee-9b35-001a4aab830c -- bash -c 'ceph orch upgrade ls'"

pass 7352071 2023-07-25 20:10:18 2023-07-25 23:49:54 1178 smithi main ubuntu 20.04 orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
fail 7352073 2023-07-25 20:10:29 2023-07-25 23:24:14 2023-07-26 00:07:48 0:43:34 0:30:53 0:12:41 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

pass 7352075 2023-07-25 20:10:38 2023-07-26 00:03:09 1501 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
pass 7352077 2023-07-25 20:10:54 2023-07-26 00:15:30 2100 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_nfs} 1
pass 7352079 2023-07-25 20:11:10 2023-07-26 00:08:29 1664 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7352080 2023-07-25 20:11:21 2023-07-25 23:27:25 2023-07-26 00:01:05 0:33:40 0:20:57 0:12:43 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli} 1
pass 7352082 2023-07-25 20:11:27 2023-07-25 23:28:01 2023-07-25 23:58:28 0:30:27 0:19:36 0:10:51 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7352084 2023-07-25 20:11:41 2023-07-25 23:29:18 2023-07-26 00:06:56 0:37:38 0:25:24 0:12:14 smithi main ubuntu 20.04 orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7352086 2023-07-25 20:11:52 2023-07-25 23:29:44 2023-07-26 00:28:52 0:59:08 0:45:31 0:13:37 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7352088 2023-07-25 20:12:08 2023-07-25 23:30:12 2023-07-26 00:27:31 0:57:19 0:47:30 0:09:49 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7352090 2023-07-25 20:12:14 2023-07-25 23:30:58 2023-07-26 00:22:56 0:51:58 0:41:29 0:10:29 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli_mon} 5
pass 7352092 2023-07-25 20:12:30 2023-07-25 23:33:28 2023-07-26 00:20:15 0:46:47 0:33:54 0:12:53 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7352094 2023-07-25 20:12:38 2023-07-25 23:35:48 2023-07-26 00:06:10 0:30:22 0:20:59 0:09:23 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7352097 2023-07-25 20:12:43 2023-07-26 00:17:55 1761 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7352098 2023-07-25 20:12:44 2023-07-25 23:40:35 2023-07-26 00:29:55 0:49:20 0:39:02 0:10:18 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7352100 2023-07-25 20:12:50 2023-07-25 23:40:56 2023-07-26 00:10:44 0:29:48 0:15:33 0:14:15 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
dead 7352102 2023-07-25 20:12:55 2023-07-25 23:44:40 2023-07-26 00:19:18 0:34:38 0:18:13 0:16:25 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
Failure Reason:

{'Failed to update apt cache: unknown reason'}