Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7913676 2024-09-21 03:45:03 2024-09-21 03:47:01 2024-09-21 04:18:36 0:31:35 0:21:32 0:10:03 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7913677 2024-09-21 03:45:04 2024-09-21 03:47:51 2024-09-21 04:13:49 0:25:58 0:16:25 0:09:33 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
fail 7913678 2024-09-21 03:45:05 2024-09-21 03:48:52 2024-09-21 04:32:02 0:43:10 0:33:02 0:10:08 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-09-21T04:20:00.000128+0000 mon.smithi110 (mon.0) 370 : cluster [WRN] osd.2 (root=default,host=smithi110) is down" in cluster log

pass 7913679 2024-09-21 03:45:07 2024-09-21 03:48:52 2024-09-21 04:31:59 0:43:07 0:32:37 0:10:30 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7913680 2024-09-21 03:45:08 2024-09-21 03:49:02 2024-09-21 04:34:28 0:45:26 0:36:03 0:09:23 smithi main centos 9.stream orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/nfs} 1
pass 7913681 2024-09-21 03:45:09 2024-09-21 03:49:23 2024-09-21 04:12:13 0:22:50 0:13:24 0:09:26 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
pass 7913682 2024-09-21 03:45:10 2024-09-21 03:49:23 2024-09-21 04:12:11 0:22:48 0:11:24 0:11:24 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
pass 7913683 2024-09-21 03:45:11 2024-09-21 03:51:24 2024-09-21 04:21:13 0:29:49 0:20:07 0:09:42 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7913684 2024-09-21 03:45:12 2024-09-21 03:51:24 2024-09-21 04:24:35 0:33:11 0:24:02 0:09:09 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7913685 2024-09-21 03:45:15 2024-09-21 03:51:34 2024-09-21 04:16:43 0:25:09 0:15:48 0:09:21 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
fail 7913686 2024-09-21 03:45:16 2024-09-21 03:51:45 2024-09-21 04:13:19 0:21:34 0:11:07 0:10:27 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
Failure Reason:

"2024-09-21T04:09:01.721600+0000 mon.smithi167 (mon.0) 293 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7913687 2024-09-21 03:45:17 2024-09-21 03:52:15 2024-09-21 04:15:14 0:22:59 0:10:42 0:12:17 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7913688 2024-09-21 03:45:18 2024-09-21 03:54:16 2024-09-21 04:59:26 1:05:10 0:54:03 0:11:07 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
pass 7913689 2024-09-21 03:45:19 2024-09-21 03:55:46 2024-09-21 04:41:04 0:45:18 0:35:15 0:10:03 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
fail 7913690 2024-09-21 03:45:21 2024-09-21 03:55:56 2024-09-21 04:25:57 0:30:01 0:16:13 0:13:48 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_mgr_ctdb_res_ips} 4
Failure Reason:

SELinux denials found on ubuntu@smithi112.front.sepia.ceph.com: ['type=AVC msg=audit(1726892467.852:10875): avc: denied { nlmsg_read } for pid=61328 comm="ss" scontext=system_u:system_r:container_t:s0:c383,c420 tcontext=system_u:system_r:container_t:s0:c383,c420 tclass=netlink_tcpdiag_socket permissive=1']

pass 7913691 2024-09-21 03:45:22 2024-09-21 03:58:47 2024-09-21 04:32:30 0:33:43 0:22:56 0:10:47 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7913692 2024-09-21 03:45:23 2024-09-21 03:59:18 2024-09-21 04:20:01 0:20:43 0:12:21 0:08:22 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7913693 2024-09-21 03:45:24 2024-09-21 03:59:18 2024-09-21 04:24:42 0:25:24 0:15:46 0:09:38 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7913694 2024-09-21 03:45:25 2024-09-21 03:59:38 2024-09-21 04:31:19 0:31:41 0:21:36 0:10:05 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7913695 2024-09-21 03:45:26 2024-09-21 03:59:49 2024-09-21 04:22:19 0:22:30 0:12:13 0:10:17 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7913696 2024-09-21 03:45:27 2024-09-21 04:01:19 2024-09-21 04:32:11 0:30:52 0:19:34 0:11:18 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_mgr_domain} 2
pass 7913697 2024-09-21 03:45:29 2024-09-21 04:01:30 2024-09-21 04:39:56 0:38:26 0:26:34 0:11:52 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7913698 2024-09-21 03:45:30 2024-09-21 04:02:10 2024-09-21 04:41:40 0:39:30 0:28:50 0:10:40 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7913699 2024-09-21 03:45:31 2024-09-21 04:02:40 2024-09-21 04:58:27 0:55:47 0:44:58 0:10:49 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7913700 2024-09-21 03:45:32 2024-09-21 04:02:41 2024-09-21 04:38:38 0:35:57 0:26:17 0:09:40 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7913701 2024-09-21 03:45:33 2024-09-21 04:02:51 2024-09-21 05:26:31 1:23:40 1:06:30 0:17:10 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7913702 2024-09-21 03:45:34 2024-09-21 04:09:22 2024-09-21 04:32:34 0:23:12 0:13:36 0:09:36 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
fail 7913703 2024-09-21 03:45:36 2024-09-21 04:09:43 2024-09-21 05:03:25 0:53:42 0:43:01 0:10:41 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-squid 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

Command failed on smithi018 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 275ef78e-77d1-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c \'ceph versions | jq -e \'"\'"\'.rgw | length == 1\'"\'"\'\''

pass 7913704 2024-09-21 03:45:37 2024-09-21 04:09:43 2024-09-21 04:45:40 0:35:57 0:24:50 0:11:07 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
pass 7913705 2024-09-21 03:45:38 2024-09-21 04:10:24 2024-09-21 04:35:12 0:24:48 0:13:17 0:11:31 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7913706 2024-09-21 03:45:39 2024-09-21 04:12:14 2024-09-21 04:31:09 0:18:55 0:09:53 0:09:02 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_mgr_res_basic} 2
pass 7913707 2024-09-21 03:45:40 2024-09-21 04:12:25 2024-09-21 04:43:23 0:30:58 0:21:30 0:09:28 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} 2
fail 7913708 2024-09-21 03:45:41 2024-09-21 04:12:25 2024-09-21 04:36:05 0:23:40 0:12:16 0:11:24 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

Command failed on smithi070 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e91f46da-77d1-11ef-baf5-efdc52797490 -- bash -c \'set -e\nset -x\nwhile true; do TOKEN=$(ceph rgw realm tokens | jq -r \'"\'"\'.[0].token\'"\'"\'); echo $TOKEN; if [ "$TOKEN" != "master zone has no endpoint" ]; then break; fi; sleep 5; done\nTOKENS=$(ceph rgw realm tokens)\necho $TOKENS | jq --exit-status \'"\'"\'.[0].realm == "myrealm1"\'"\'"\'\necho $TOKENS | jq --exit-status \'"\'"\'.[0].token\'"\'"\'\nTOKEN_JSON=$(ceph rgw realm tokens | jq -r \'"\'"\'.[0].token\'"\'"\' | base64 --decode)\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.realm_name == "myrealm1"\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.endpoint | test("http://.+:\\\\d+")\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.realm_id | test("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$")\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.access_key\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.secret\'"\'"\'\n\''

pass 7913709 2024-09-21 03:45:42 2024-09-21 04:12:55 2024-09-21 04:43:43 0:30:48 0:20:25 0:10:23 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7913710 2024-09-21 03:45:44 2024-09-21 04:13:46 2024-09-21 04:37:10 0:23:24 0:13:10 0:10:14 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7913711 2024-09-21 03:45:45 2024-09-21 04:14:06 2024-09-21 05:12:30 0:58:24 0:47:15 0:11:09 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
fail 7913712 2024-09-21 03:45:46 2024-09-21 04:14:07 2024-09-21 04:34:16 0:20:09 0:09:06 0:11:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (30) after waiting for 300 seconds

dead 7913713 2024-09-21 03:45:47 2024-09-21 04:15:17 2024-09-21 12:23:56 8:08:39 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7913714 2024-09-21 03:45:48 2024-09-21 04:15:27 2024-09-21 04:37:58 0:22:31 0:10:31 0:12:00 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
pass 7913715 2024-09-21 03:45:49 2024-09-21 04:16:58 2024-09-21 05:20:31 1:03:33 0:51:59 0:11:34 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
pass 7913716 2024-09-21 03:45:51 2024-09-21 04:18:49 2024-09-21 04:39:43 0:20:54 0:11:06 0:09:48 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_mgr_res_dom} 2
pass 7913717 2024-09-21 03:45:52 2024-09-21 04:18:59 2024-09-21 04:51:49 0:32:50 0:23:22 0:09:28 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7913718 2024-09-21 03:45:53 2024-09-21 04:18:59 2024-09-21 04:46:34 0:27:35 0:15:08 0:12:27 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7913719 2024-09-21 03:45:54 2024-09-21 04:21:30 2024-09-21 04:47:40 0:26:10 0:14:31 0:11:39 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7913720 2024-09-21 03:45:55 2024-09-21 04:22:41 2024-09-21 04:46:29 0:23:48 0:12:57 0:10:51 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
pass 7913721 2024-09-21 03:45:56 2024-09-21 04:24:51 2024-09-21 04:45:51 0:21:00 0:12:03 0:08:57 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/deploy-raw} 2
pass 7913722 2024-09-21 03:45:57 2024-09-21 04:24:51 2024-09-21 04:46:01 0:21:10 0:11:49 0:09:21 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
pass 7913723 2024-09-21 03:45:58 2024-09-21 04:25:02 2024-09-21 04:50:26 0:25:24 0:14:46 0:10:38 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7913724 2024-09-21 03:46:00 2024-09-21 04:25:22 2024-09-21 04:56:02 0:30:40 0:19:03 0:11:37 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7913725 2024-09-21 03:46:01 2024-09-21 04:26:13 2024-09-21 05:09:14 0:43:01 0:33:55 0:09:06 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
pass 7913726 2024-09-21 03:46:02 2024-09-21 04:26:13 2024-09-21 04:50:13 0:24:00 0:10:37 0:13:23 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
fail 7913727 2024-09-21 03:46:03 2024-09-21 04:28:04 2024-09-21 05:32:13 1:04:09 0:50:52 0:13:17 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7913728 2024-09-21 03:46:04 2024-09-21 04:31:24 2024-09-21 04:54:32 0:23:08 0:12:49 0:10:19 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7913729 2024-09-21 03:46:05 2024-09-21 04:31:35 2024-09-21 05:46:04 1:14:29 1:04:36 0:09:53 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04-squid 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

Command failed on smithi089 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfdfee20-77d4-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c \'ceph versions | jq -e \'"\'"\'.rgw | length == 1\'"\'"\'\''

pass 7913730 2024-09-21 03:46:06 2024-09-21 04:32:15 2024-09-21 05:01:55 0:29:40 0:19:16 0:10:24 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
fail 7913731 2024-09-21 03:46:08 2024-09-21 04:32:15 2024-09-21 05:17:05 0:44:50 0:33:01 0:11:49 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} 2
Failure Reason:

"2024-09-21T05:13:09.618148+0000 mon.a (mon.0) 1369 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7913732 2024-09-21 03:46:09 2024-09-21 04:32:26 2024-09-21 05:01:46 0:29:20 0:19:48 0:09:32 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
pass 7913733 2024-09-21 03:46:10 2024-09-21 04:32:26 2024-09-21 04:55:59 0:23:33 0:12:53 0:10:40 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7913734 2024-09-21 03:46:11 2024-09-21 04:32:46 2024-09-21 05:00:21 0:27:35 0:18:43 0:08:52 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7913735 2024-09-21 03:46:12 2024-09-21 04:32:47 2024-09-21 05:38:59 1:06:12 0:56:16 0:09:56 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
pass 7913736 2024-09-21 03:46:13 2024-09-21 04:33:57 2024-09-21 05:06:02 0:32:05 0:22:02 0:10:03 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7913737 2024-09-21 03:46:14 2024-09-21 04:34:28 2024-09-21 04:54:22 0:19:54 0:10:09 0:09:45 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_mgr_basic} 2
pass 7913738 2024-09-21 03:46:15 2024-09-21 04:35:18 2024-09-21 05:08:27 0:33:09 0:23:22 0:09:47 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} 2
pass 7913739 2024-09-21 03:46:17 2024-09-21 04:35:28 2024-09-21 04:50:47 0:15:19 0:05:40 0:09:39 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
fail 7913740 2024-09-21 03:46:18 2024-09-21 04:35:29 2024-09-21 04:55:28 0:19:59 0:08:51 0:11:08 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (30) after waiting for 300 seconds

pass 7913741 2024-09-21 03:46:19 2024-09-21 04:36:19 2024-09-21 04:57:17 0:20:58 0:12:40 0:08:18 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
pass 7913742 2024-09-21 03:46:20 2024-09-21 04:36:20 2024-09-21 04:58:07 0:21:47 0:11:43 0:10:04 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7913743 2024-09-21 03:46:21 2024-09-21 04:37:20 2024-09-21 05:01:08 0:23:48 0:12:57 0:10:51 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
fail 7913744 2024-09-21 03:46:22 2024-09-21 04:38:11 2024-09-21 04:59:12 0:21:01 0:11:28 0:09:33 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
Failure Reason:

"2024-09-21T04:53:54.268655+0000 mon.smithi116 (mon.0) 290 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7913745 2024-09-21 03:46:24 2024-09-21 04:38:21 2024-09-21 05:00:00 0:21:39 0:10:57 0:10:42 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
pass 7913746 2024-09-21 03:46:25 2024-09-21 04:38:51 2024-09-21 05:24:52 0:46:01 0:34:25 0:11:36 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
pass 7913747 2024-09-21 03:46:26 2024-09-21 04:40:02 2024-09-21 05:03:22 0:23:20 0:13:47 0:09:33 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7913748 2024-09-21 03:46:27 2024-09-21 04:40:12 2024-09-21 05:04:16 0:24:04 0:12:54 0:11:10 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7913749 2024-09-21 03:46:28 2024-09-21 04:41:23 2024-09-21 05:07:54 0:26:31 0:15:16 0:11:15 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_mgr_ctdb_res_basic} 4
pass 7913750 2024-09-21 03:46:29 2024-09-21 04:42:53 2024-09-21 05:06:28 0:23:35 0:14:10 0:09:25 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7913751 2024-09-21 03:46:31 2024-09-21 04:43:44 2024-09-21 05:24:34 0:40:50 0:30:37 0:10:13 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
pass 7913752 2024-09-21 03:46:32 2024-09-21 04:43:44 2024-09-21 05:27:15 0:43:31 0:33:49 0:09:42 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} 2
pass 7913753 2024-09-21 03:46:33 2024-09-21 04:44:04 2024-09-21 05:06:55 0:22:51 0:12:31 0:10:20 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
pass 7913754 2024-09-21 03:46:34 2024-09-21 04:45:55 2024-09-21 05:17:32 0:31:37 0:22:06 0:09:31 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
fail 7913755 2024-09-21 03:46:35 2024-09-21 04:45:55 2024-09-21 05:33:29 0:47:34 0:37:18 0:10:16 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7913756 2024-09-21 03:46:37 2024-09-21 04:45:56 2024-09-21 05:09:06 0:23:10 0:14:00 0:09:10 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
fail 7913757 2024-09-21 03:46:38 2024-09-21 04:46:06 2024-09-21 05:39:23 0:53:17 0:43:16 0:10:01 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-squid 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi077 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1db6b6a4-77d6-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c \'ceph versions | jq -e \'"\'"\'.rgw | length == 1\'"\'"\'\''

pass 7913758 2024-09-21 03:46:39 2024-09-21 04:46:17 2024-09-21 05:24:41 0:38:24 0:27:08 0:11:16 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_mgr_ctdb_res_dom} 4
fail 7913759 2024-09-21 03:46:40 2024-09-21 04:46:47 2024-09-21 05:29:28 0:42:41 0:32:45 0:09:56 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} 2
Failure Reason:

"2024-09-21T05:26:33.654907+0000 mon.a (mon.0) 1390 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7913760 2024-09-21 03:46:41 2024-09-21 04:47:57 2024-09-21 05:25:25 0:37:28 0:23:10 0:14:18 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
pass 7913761 2024-09-21 03:46:43 2024-09-21 04:50:38 2024-09-21 05:13:42 0:23:04 0:13:40 0:09:24 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
pass 7913762 2024-09-21 03:46:44 2024-09-21 04:50:39 2024-09-21 05:15:08 0:24:29 0:12:36 0:11:53 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7913763 2024-09-21 03:46:45 2024-09-21 04:51:59 2024-09-21 05:19:48 0:27:49 0:15:40 0:12:09 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7913764 2024-09-21 03:46:46 2024-09-21 04:54:40 2024-09-21 05:52:27 0:57:47 0:46:58 0:10:49 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
fail 7913765 2024-09-21 03:46:49 2024-09-21 04:54:50 2024-09-21 05:23:25 0:28:35 0:16:34 0:12:01 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_mgr_ctdb_res_ips} 4
Failure Reason:

"2024-09-21T05:20:52.720437+0000 mon.a (mon.0) 794 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7913766 2024-09-21 03:46:50 2024-09-21 04:56:11 2024-09-21 05:29:01 0:32:50 0:22:55 0:09:55 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} 2
pass 7913767 2024-09-21 03:46:51 2024-09-21 04:56:11 2024-09-21 05:17:10 0:20:59 0:12:32 0:08:27 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7913768 2024-09-21 03:46:52 2024-09-21 04:56:21 2024-09-21 05:31:53 0:35:32 0:24:12 0:11:20 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
fail 7913769 2024-09-21 03:46:54 2024-09-21 04:58:22 2024-09-21 05:17:51 0:19:29 0:08:50 0:10:39 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (30) after waiting for 300 seconds

pass 7913770 2024-09-21 03:46:55 2024-09-21 04:58:42 2024-09-21 05:22:37 0:23:55 0:13:28 0:10:27 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7913771 2024-09-21 03:46:56 2024-09-21 04:59:33 2024-09-21 05:19:58 0:20:25 0:10:20 0:10:05 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7913772 2024-09-21 03:46:57 2024-09-21 05:00:13 2024-09-21 05:57:52 0:57:39 0:47:25 0:10:14 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
pass 7913773 2024-09-21 03:46:58 2024-09-21 05:00:14 2024-09-21 05:30:48 0:30:34 0:19:40 0:10:54 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7913774 2024-09-21 03:46:59 2024-09-21 05:00:44 2024-09-21 05:20:28 0:19:44 0:10:45 0:08:59 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_mgr_domain} 2
pass 7913775 2024-09-21 03:47:01 2024-09-21 05:37:31 1575 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7913776 2024-09-21 03:47:02 2024-09-21 05:02:05 2024-09-21 05:46:33 0:44:28 0:34:09 0:10:19 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} 2
pass 7913777 2024-09-21 03:47:03 2024-09-21 05:02:45 2024-09-21 05:30:45 0:28:00 0:18:30 0:09:30 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
pass 7913778 2024-09-21 03:47:04 2024-09-21 05:03:36 2024-09-21 05:26:50 0:23:14 0:13:48 0:09:26 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7913779 2024-09-21 03:47:05 2024-09-21 05:03:46 2024-09-21 06:01:29 0:57:43 0:42:12 0:15:31 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7913780 2024-09-21 03:47:07 2024-09-21 05:06:37 2024-09-21 05:35:59 0:29:22 0:20:08 0:09:14 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7913781 2024-09-21 03:47:08 2024-09-21 05:06:37 2024-09-21 05:34:12 0:27:35 0:18:25 0:09:10 smithi main ubuntu 22.04 orch:cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_mgr_res_basic} 2
pass 7913782 2024-09-21 03:47:09 2024-09-21 05:06:38 2024-09-21 05:48:43 0:42:05 0:32:46 0:09:19 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} 2
dead 7913783 2024-09-21 03:47:10 2024-09-21 05:07:08 2024-09-21 13:17:51 8:10:43 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

hit max job timeout

fail 7913784 2024-09-21 03:47:12 2024-09-21 05:08:09 2024-09-21 05:56:17 0:48:08 0:36:50 0:11:18 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7913785 2024-09-21 03:47:13 2024-09-21 05:08:39 2024-09-21 05:32:17 0:23:38 0:12:53 0:10:45 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
fail 7913786 2024-09-21 03:47:14 2024-09-21 05:09:19 2024-09-21 06:23:08 1:13:49 1:04:47 0:09:02 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04-squid 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

Command failed on smithi099 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ee3410ee-77d9-11ef-baf5-efdc52797490 -e sha1=f36283e7c91cac1c01cb9a7c4ff21bd697dbd4f2 -- bash -c \'ceph versions | jq -e \'"\'"\'.rgw | length == 1\'"\'"\'\''

pass 7913787 2024-09-21 03:47:15 2024-09-21 05:09:30 2024-09-21 05:32:45 0:23:15 0:13:08 0:10:07 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7913788 2024-09-21 03:47:16 2024-09-21 05:09:50 2024-09-21 06:37:24 1:27:34 1:17:45 0:09:49 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
dead 7913789 2024-09-21 03:47:17 2024-09-21 05:09:51 2024-09-21 13:22:11 8:12:20 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7913790 2024-09-21 03:47:18 2024-09-21 05:12:51 2024-09-21 05:35:31 0:22:40 0:10:43 0:11:57 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_mgr_res_dom} 2
pass 7913791 2024-09-21 03:47:19 2024-09-21 05:14:32 2024-09-21 05:48:21 0:33:49 0:23:10 0:10:39 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7913792 2024-09-21 03:47:21 2024-09-21 05:15:22 2024-09-21 05:40:32 0:25:10 0:14:27 0:10:43 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_set_mon_crush_locations} 3
pass 7913793 2024-09-21 03:47:22 2024-09-21 05:17:23 2024-09-21 05:50:27 0:33:04 0:22:00 0:11:04 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2