Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7389865 2023-09-06 09:26:47 2023-09-06 09:27:43 2023-09-06 09:51:08 0:23:25 0:14:07 0:09:18 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7389866 2023-09-06 09:26:48 2023-09-06 09:27:44 2023-09-06 09:56:35 0:28:51 0:20:26 0:08:25 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
pass 7389867 2023-09-06 09:26:48 2023-09-06 09:27:44 2023-09-06 10:07:23 0:39:39 0:32:29 0:07:10 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7389868 2023-09-06 09:26:49 2023-09-06 09:28:15 2023-09-06 10:14:07 0:45:52 0:32:20 0:13:32 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7389869 2023-09-06 09:26:50 2023-09-06 09:30:35 2023-09-06 10:31:36 1:01:01 0:51:19 0:09:42 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

pass 7389870 2023-09-06 09:26:51 2023-09-06 09:30:36 2023-09-06 09:59:01 0:28:25 0:17:54 0:10:31 smithi main centos 8.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/off orchestrator_cli} 2
pass 7389871 2023-09-06 09:26:52 2023-09-06 09:31:26 2023-09-06 10:22:25 0:50:59 0:40:22 0:10:37 smithi main centos 8.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} 3
pass 7389872 2023-09-06 09:26:52 2023-09-06 09:33:17 2023-09-06 09:59:25 0:26:08 0:15:45 0:10:23 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7389873 2023-09-06 09:26:53 2023-09-06 09:33:37 2023-09-06 09:54:51 0:21:14 0:14:42 0:06:32 smithi main rhel 8.6 orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} 1
pass 7389874 2023-09-06 09:26:54 2023-09-06 09:33:37 2023-09-06 09:59:47 0:26:10 0:13:37 0:12:33 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
fail 7389875 2023-09-06 09:26:55 2023-09-06 09:36:38 2023-09-06 10:33:00 0:56:22 0:44:43 0:11:39 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi156 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bb5a89a-4c9b-11ee-9ab3-7b867c8bd7da -- bash -c 'ceph orch upgrade ls'"

fail 7389876 2023-09-06 09:26:56 2023-09-06 09:37:29 2023-09-06 10:14:11 0:36:42 0:30:05 0:06:37 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_nfs} 1
Failure Reason:

Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS)

pass 7389877 2023-09-06 09:26:57 2023-09-06 09:37:29 2023-09-06 10:13:55 0:36:26 0:28:59 0:07:27 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7389878 2023-09-06 09:26:57 2023-09-06 09:38:40 2023-09-06 10:08:45 0:30:05 0:19:09 0:10:56 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli} 1
fail 7389879 2023-09-06 09:26:58 2023-09-06 09:38:40 2023-09-06 10:12:11 0:33:31 0:17:41 0:15:50 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

Command failed on smithi146 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.146:/d1 /mnt/happy'"

pass 7389880 2023-09-06 09:26:59 2023-09-06 09:42:11 2023-09-06 10:07:12 0:25:01 0:18:22 0:06:39 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7389881 2023-09-06 09:27:00 2023-09-06 09:42:11 2023-09-06 10:35:56 0:53:45 0:40:04 0:13:41 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
pass 7389882 2023-09-06 09:27:00 2023-09-06 09:47:42 2023-09-06 10:44:52 0:57:10 0:44:14 0:12:56 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7389883 2023-09-06 09:27:01 2023-09-06 09:50:33 2023-09-06 10:31:30 0:40:57 0:24:54 0:16:03 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon} 5
pass 7389884 2023-09-06 09:27:02 2023-09-06 09:56:04 2023-09-06 10:23:19 0:27:15 0:20:03 0:07:12 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7389885 2023-09-06 09:27:03 2023-09-06 09:56:04 2023-09-06 10:31:51 0:35:47 0:24:41 0:11:06 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7389886 2023-09-06 09:27:04 2023-09-06 09:56:45 2023-09-06 10:21:59 0:25:14 0:15:09 0:10:05 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7389887 2023-09-06 09:27:04 2023-09-06 09:56:45 2023-09-06 10:25:38 0:28:53 0:16:44 0:12:09 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7389888 2023-09-06 09:27:05 2023-09-06 09:57:26 2023-09-06 10:40:52 0:43:26 0:31:43 0:11:43 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7389889 2023-09-06 09:27:06 2023-09-06 09:59:06 2023-09-06 10:25:14 0:26:08 0:19:00 0:07:08 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7389890 2023-09-06 09:27:07 2023-09-06 09:59:26 2023-09-06 10:35:04 0:35:38 0:25:43 0:09:55 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7389891 2023-09-06 09:27:08 2023-09-06 09:59:57 2023-09-06 10:29:08 0:29:11 0:23:41 0:05:30 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7389892 2023-09-06 09:27:08 2023-09-06 09:59:57 2023-09-06 10:40:19 0:40:22 0:25:08 0:15:14 smithi main ubuntu 20.04 orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

Command failed on smithi061 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 7389893 2023-09-06 09:27:09 2023-09-06 10:04:08 2023-09-06 10:41:13 0:37:05 0:23:18 0:13:47 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

Command failed on smithi032 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.32:/fake /mnt/foo'"

pass 7389894 2023-09-06 09:27:10 2023-09-06 10:07:19 2023-09-06 11:17:54 1:10:35 1:00:42 0:09:53 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7389895 2023-09-06 09:27:11 2023-09-06 10:07:29 2023-09-06 10:56:00 0:48:31 0:41:28 0:07:03 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
pass 7389896 2023-09-06 09:27:12 2023-09-06 10:08:50 2023-09-06 10:35:25 0:26:35 0:16:08 0:10:27 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} 1
pass 7389897 2023-09-06 09:27:12 2023-09-06 10:12:21 2023-09-06 10:36:21 0:24:00 0:17:29 0:06:31 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_ca_signed_key} 2
pass 7389898 2023-09-06 09:27:13 2023-09-06 10:12:31 2023-09-06 10:45:49 0:33:18 0:26:16 0:07:02 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
pass 7389899 2023-09-06 09:27:14 2023-09-06 10:12:31 2023-09-06 10:53:11 0:40:40 0:30:31 0:10:09 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7389900 2023-09-06 09:27:15 2023-09-06 10:14:02 2023-09-06 10:43:39 0:29:37 0:18:59 0:10:38 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
Failure Reason:

Command failed on smithi002 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.2:/fake /mnt/foo -o port=2999'"

pass 7389901 2023-09-06 09:27:16 2023-09-06 10:14:12 2023-09-06 10:38:19 0:24:07 0:13:08 0:10:59 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
fail 7389902 2023-09-06 09:27:16 2023-09-06 10:14:23 2023-09-06 11:00:53 0:46:30 0:35:05 0:11:25 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi026 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 47cb38c8-4ca0-11ee-9ab3-7b867c8bd7da -- bash -c 'ceph orch upgrade ls'"

pass 7389903 2023-09-06 09:27:17 2023-09-06 10:14:43 2023-09-06 10:47:02 0:32:19 0:17:48 0:14:31 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7389904 2023-09-06 09:27:18 2023-09-06 10:21:14 2023-09-06 10:48:36 0:27:22 0:18:34 0:08:48 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} 1
pass 7389905 2023-09-06 09:27:19 2023-09-06 10:21:15 2023-09-06 11:11:46 0:50:31 0:38:10 0:12:21 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
fail 7389906 2023-09-06 09:27:19 2023-09-06 10:21:35 2023-09-06 10:48:29 0:26:54 0:17:28 0:09:26 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
Failure Reason:

Command failed on smithi042 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.42:/fake /mnt/foo'"

pass 7389907 2023-09-06 09:27:20 2023-09-06 10:21:35 2023-09-06 10:40:47 0:19:12 0:09:25 0:09:47 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} 1
fail 7389908 2023-09-06 09:27:21 2023-09-06 10:21:36 2023-09-06 10:41:14 0:19:38 0:10:37 0:09:01 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

Command failed on smithi140 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8f608264-4ca1-11ee-9ab3-7b867c8bd7da -- ceph mon dump -f json'

pass 7389909 2023-09-06 09:27:22 2023-09-06 10:22:06 2023-09-06 10:47:27 0:25:21 0:16:38 0:08:43 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7389910 2023-09-06 09:27:22 2023-09-06 10:22:27 2023-09-06 11:06:10 0:43:43 0:32:53 0:10:50 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7389911 2023-09-06 09:27:23 2023-09-06 10:22:27 2023-09-06 11:10:46 0:48:19 0:37:53 0:10:26 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7389912 2023-09-06 09:27:24 2023-09-06 10:23:27 2023-09-06 10:49:02 0:25:35 0:19:24 0:06:11 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7389913 2023-09-06 09:27:25 2023-09-06 10:23:38 2023-09-06 10:51:04 0:27:26 0:17:17 0:10:09 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
pass 7389914 2023-09-06 09:27:26 2023-09-06 10:23:38 2023-09-06 11:08:58 0:45:20 0:32:32 0:12:48 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7389915 2023-09-06 09:27:26 2023-09-06 10:25:19 2023-09-06 10:59:54 0:34:35 0:23:39 0:10:56 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7389916 2023-09-06 09:27:27 2023-09-06 10:25:39 2023-09-06 10:52:22 0:26:43 0:17:45 0:08:58 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7389917 2023-09-06 09:27:28 2023-09-06 10:29:10 2023-09-06 11:05:54 0:36:44 0:30:00 0:06:44 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} 1
Failure Reason:

Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS)

pass 7389918 2023-09-06 09:27:29 2023-09-06 10:29:10 2023-09-06 11:08:40 0:39:30 0:30:29 0:09:01 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7389919 2023-09-06 09:27:29 2023-09-06 10:31:31 2023-09-06 10:56:52 0:25:21 0:17:47 0:07:34 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7389920 2023-09-06 09:27:30 2023-09-06 10:31:31 2023-09-06 10:56:39 0:25:08 0:19:31 0:05:37 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} 1
pass 7389921 2023-09-06 09:27:31 2023-09-06 10:31:31 2023-09-06 11:01:37 0:30:06 0:19:09 0:10:57 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7389922 2023-09-06 09:27:32 2023-09-06 10:31:42 2023-09-06 10:57:00 0:25:18 0:15:54 0:09:24 smithi main centos 8.stream orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7389923 2023-09-06 09:27:32 2023-09-06 10:31:52 2023-09-06 11:22:23 0:50:31 0:38:51 0:11:40 smithi main centos 8.stream orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
fail 7389924 2023-09-06 09:27:33 2023-09-06 10:33:03 2023-09-06 11:21:16 0:48:13 0:40:47 0:07:26 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
Failure Reason:

Command failed on smithi079 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 7389925 2023-09-06 09:27:34 2023-09-06 10:34:03 2023-09-06 11:25:30 0:51:27 0:38:16 0:13:11 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

Command failed on smithi119 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 7389926 2023-09-06 09:27:35 2023-09-06 10:35:14 2023-09-06 11:14:53 0:39:39 0:30:02 0:09:37 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7389927 2023-09-06 09:27:36 2023-09-06 10:35:34 2023-09-06 11:04:20 0:28:46 0:17:50 0:10:56 smithi main centos 8.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli} 2
pass 7389928 2023-09-06 09:27:36 2023-09-06 10:36:05 2023-09-06 11:04:20 0:28:15 0:17:29 0:10:46 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7389929 2023-09-06 09:27:37 2023-09-06 10:36:25 2023-09-06 11:04:22 0:27:57 0:18:02 0:09:55 smithi main rhel 8.6 orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} 1
pass 7389930 2023-09-06 09:27:38 2023-09-06 10:38:26 2023-09-06 11:03:35 0:25:09 0:12:57 0:12:12 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
fail 7389931 2023-09-06 09:27:39 2023-09-06 10:40:26 2023-09-06 11:36:36 0:56:10 0:44:32 0:11:38 smithi main centos 8.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi072 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6a17fafc-4ca4-11ee-9ab3-7b867c8bd7da -- bash -c 'ceph orch upgrade ls'"

pass 7389932 2023-09-06 09:27:39 2023-09-06 10:40:57 2023-09-06 11:23:02 0:42:05 0:31:55 0:10:10 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
pass 7389933 2023-09-06 09:27:40 2023-09-06 10:40:57 2023-09-06 11:10:27 0:29:30 0:19:25 0:10:05 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7389934 2023-09-06 09:27:41 2023-09-06 10:41:18 2023-09-06 11:11:08 0:29:50 0:16:48 0:13:02 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7389935 2023-09-06 09:27:42 2023-09-06 10:44:04 2023-09-06 11:22:04 0:38:00 0:26:57 0:11:03 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7389936 2023-09-06 09:27:42 2023-09-06 10:44:55 2023-09-06 11:19:18 0:34:23 0:22:30 0:11:53 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7389937 2023-09-06 09:27:43 2023-09-06 10:45:55 2023-09-06 11:10:53 0:24:58 0:16:18 0:08:40 smithi main rhel 8.6 orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7389938 2023-09-06 09:27:44 2023-09-06 11:14:45 1192 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7389939 2023-09-06 09:27:44 2023-09-06 10:47:36 2023-09-06 11:13:40 0:26:04 0:18:31 0:07:33 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7389940 2023-09-06 09:27:45 2023-09-06 10:48:37 2023-09-06 11:38:08 0:49:31 0:42:13 0:07:18 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
pass 7389941 2023-09-06 09:27:46 2023-09-06 10:49:07 2023-09-06 11:42:42 0:53:35 0:40:17 0:13:18 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7389942 2023-09-06 09:27:47 2023-09-06 10:51:08 2023-09-06 11:15:45 0:24:37 0:14:28 0:10:09 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} 1
pass 7389943 2023-09-06 09:27:48 2023-09-06 10:52:28 2023-09-06 11:36:37 0:44:09 0:33:31 0:10:38 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7389944 2023-09-06 09:27:48 2023-09-06 10:53:19 2023-09-06 11:21:18 0:27:59 0:18:42 0:09:17 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_ca_signed_key} 2
pass 7389945 2023-09-06 09:27:49 2023-09-06 10:56:09 2023-09-06 11:31:49 0:35:40 0:27:50 0:07:50 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
pass 7389946 2023-09-06 09:27:50 2023-09-06 10:56:40 2023-09-06 11:20:06 0:23:26 0:17:05 0:06:21 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7389947 2023-09-06 09:27:51 2023-09-06 10:57:00 2023-09-06 12:00:18 1:03:18 0:52:11 0:11:07 smithi main centos 8.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

fail 7389948 2023-09-06 09:27:51 2023-09-06 10:57:11 2023-09-06 11:29:41 0:32:30 0:21:54 0:10:36 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm} 1
Failure Reason:

SELinux denials found on ubuntu@smithi027.front.sepia.ceph.com: ['type=AVC msg=audit(1693999574.691:21265): avc: denied { ioctl } for pid=120263 comm="iptables" path="/var/lib/containers/storage/overlay/fb6484a51c3dfd3bb78793a77565ff4856424b7150bc34279a8130d1470d542d/merged" dev="overlay" ino=2754125 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']

pass 7389949 2023-09-06 09:27:52 2023-09-06 11:00:01 2023-09-06 11:38:03 0:38:02 0:30:27 0:07:35 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7389950 2023-09-06 09:27:53 2023-09-06 11:01:02 2023-09-06 11:31:36 0:30:34 0:19:11 0:11:23 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7389951 2023-09-06 09:27:54 2023-09-06 11:01:42 2023-09-06 11:19:45 0:18:03 0:07:23 0:10:40 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
pass 7389952 2023-09-06 09:27:54 2023-09-06 11:01:43 2023-09-06 11:29:10 0:27:27 0:18:05 0:09:22 smithi main rhel 8.6 orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7389953 2023-09-06 09:27:55 2023-09-06 11:03:44 2023-09-06 12:08:29 1:04:45 0:56:32 0:08:13 smithi main rhel 8.6 orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7389954 2023-09-06 09:27:56 2023-09-06 11:04:24 2023-09-06 11:59:45 0:55:21 0:43:58 0:11:23 smithi main ubuntu 20.04 orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7389955 2023-09-06 09:27:57 2023-09-06 11:04:25 2023-09-06 11:47:44 0:43:19 0:32:54 0:10:25 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7389956 2023-09-06 09:27:57 2023-09-06 11:04:25 2023-09-06 11:47:10 0:42:45 0:31:30 0:11:15 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
Failure Reason:

reached maximum tries (301) after waiting for 300 seconds

pass 7389957 2023-09-06 09:27:58 2023-09-06 11:06:16 2023-09-06 11:33:08 0:26:52 0:12:46 0:14:06 smithi main centos 8.stream orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
fail 7389958 2023-09-06 09:27:59 2023-09-06 11:08:46 2023-09-06 11:55:46 0:47:00 0:34:40 0:12:20 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

Command failed on smithi050 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 021a3088-4ca8-11ee-9ab3-7b867c8bd7da -- bash -c 'ceph orch upgrade ls'"

pass 7389959 2023-09-06 09:28:00 2023-09-06 11:09:07 2023-09-06 11:40:50 0:31:43 0:18:55 0:12:48 smithi main ubuntu 20.04 orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7389960 2023-09-06 09:28:00 2023-09-06 11:10:37 2023-09-06 11:37:21 0:26:44 0:17:53 0:08:51 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} 1
pass 7389961 2023-09-06 09:28:01 2023-09-06 11:10:38 2023-09-06 11:44:06 0:33:28 0:23:09 0:10:19 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
fail 7389962 2023-09-06 09:28:02 2023-09-06 11:10:48 2023-09-06 11:36:46 0:25:58 0:14:34 0:11:24 smithi main centos 8.stream orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} 2
Failure Reason:

Command failed on smithi118 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

fail 7389963 2023-09-06 09:28:03 2023-09-06 11:10:58 2023-09-06 11:46:35 0:35:37 0:25:31 0:10:06 smithi main centos 8.stream orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} 1
Failure Reason:

Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS)

fail 7389964 2023-09-06 09:28:03 2023-09-06 11:11:09 2023-09-06 11:48:49 0:37:40 0:27:34 0:10:06 smithi main centos 8.stream orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
Failure Reason:

Command failed on smithi173 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 7389965 2023-09-06 09:28:04 2023-09-06 11:11:49 2023-09-06 11:38:42 0:26:53 0:20:27 0:06:26 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} 1
fail 7389966 2023-09-06 09:28:05 2023-09-06 11:11:49 2023-09-06 11:44:44 0:32:55 0:24:02 0:08:53 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
Failure Reason:

Command failed on smithi042 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.42:/d1 /mnt/happy'"

pass 7389967 2023-09-06 09:28:06 2023-09-06 11:14:52 2023-09-06 11:51:07 0:36:15 0:25:14 0:11:01 smithi main ubuntu 20.04 orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7389968 2023-09-06 09:28:06 2023-09-06 11:15:02 2023-09-06 12:10:03 0:55:01 0:44:36 0:10:25 smithi main ubuntu 20.04 orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7389969 2023-09-06 09:28:07 2023-09-06 11:15:53 2023-09-06 12:07:23 0:51:30 0:42:31 0:08:59 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7389970 2023-09-06 09:28:08 2023-09-06 11:18:03 2023-09-06 12:02:38 0:44:35 0:33:15 0:11:20 smithi main centos 8.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7389971 2023-09-06 09:28:09 2023-09-06 11:19:24 2023-09-06 11:58:25 0:39:01 0:30:55 0:08:06 smithi main rhel 8.6 orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

Command failed on smithi086 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 7389972 2023-09-06 09:28:09 2023-09-06 11:21:15 2023-09-06 11:46:15 0:25:00 0:19:34 0:05:26 smithi main rhel 8.6 orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7389973 2023-09-06 09:28:10 2023-09-06 11:21:15 2023-09-06 11:54:31 0:33:16 0:27:02 0:06:14 smithi main rhel 8.6 orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
Failure Reason:

Command failed on smithi092 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

pass 7389974 2023-09-06 09:28:11 2023-09-06 11:21:25 2023-09-06 11:44:34 0:23:09 0:14:18 0:08:51 smithi main centos 8.stream orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7389975 2023-09-06 09:28:11 2023-09-06 11:21:26 2023-09-06 11:50:31 0:29:05 0:18:17 0:10:48 smithi main ubuntu 20.04 orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7389976 2023-09-06 09:28:12 2023-09-06 11:22:26 2023-09-06 11:55:31 0:33:05 0:21:03 0:12:02 smithi main ubuntu 20.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2