Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7882481 2024-08-30 21:08:51 2024-08-31 01:25:07 2024-08-31 02:26:36 1:01:29 0:52:45 0:08:44 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

Command failed on smithi144 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 195bb82c-673a-11ef-bcd4-c7b262605968 -e sha1=34b9d27bc925f169ace27b9f1f204536aa49fc0b -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

pass 7882482 2024-08-30 21:08:52 2024-08-31 01:25:08 2024-08-31 01:41:56 0:16:48 0:09:51 0:06:57 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7882483 2024-08-30 21:08:53 2024-08-31 01:25:08 2024-08-31 01:42:13 0:17:05 0:10:41 0:06:24 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7882484 2024-08-30 21:08:55 2024-08-31 01:25:09 2024-08-31 01:42:48 0:17:39 0:07:48 0:09:51 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7882485 2024-08-30 21:08:56 2024-08-31 01:27:49 2024-08-31 02:00:54 0:33:05 0:26:18 0:06:47 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
pass 7882486 2024-08-30 21:08:57 2024-08-31 01:27:50 2024-08-31 02:38:28 1:10:38 1:00:45 0:09:53 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
pass 7882487 2024-08-30 21:08:58 2024-08-31 01:28:40 2024-08-31 01:45:31 0:16:51 0:10:01 0:06:50 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_extra_daemon_features} 2
pass 7882488 2024-08-30 21:09:00 2024-08-31 01:28:41 2024-08-31 01:47:01 0:18:20 0:11:21 0:06:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
fail 7882489 2024-08-30 21:09:01 2024-08-31 01:29:31 2024-08-31 02:25:24 0:55:53 0:49:28 0:06:25 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882490 2024-08-30 21:09:02 2024-08-31 01:30:32 2024-08-31 02:20:51 0:50:19 0:40:11 0:10:08 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7882491 2024-08-30 21:09:04 2024-08-31 01:31:12 2024-08-31 01:58:52 0:27:40 0:18:28 0:09:12 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
pass 7882492 2024-08-30 21:09:05 2024-08-31 01:31:23 2024-08-31 02:00:58 0:29:35 0:20:36 0:08:59 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7882493 2024-08-30 21:09:06 2024-08-31 01:31:23 2024-08-31 02:07:16 0:35:53 0:25:03 0:10:50 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_host_drain} 3
fail 7882494 2024-08-30 21:09:08 2024-08-31 01:31:33 2024-08-31 01:49:58 0:18:25 0:07:58 0:10:27 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/1.7.2} 3
Failure Reason:

Command failed on smithi052 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7882495 2024-08-30 21:09:09 2024-08-31 01:32:14 2024-08-31 02:02:29 0:30:15 0:20:56 0:09:19 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7882496 2024-08-30 21:09:10 2024-08-31 01:32:24 2024-08-31 01:49:37 0:17:13 0:10:05 0:07:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
dead 7882497 2024-08-30 21:09:11 2024-08-31 01:32:25 2024-08-31 09:41:10 8:08:45 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7882498 2024-08-30 21:09:13 2024-08-31 01:32:55 2024-08-31 02:09:46 0:36:51 0:26:43 0:10:08 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7882499 2024-08-30 21:09:14 2024-08-31 01:34:05 2024-08-31 01:50:45 0:16:40 0:09:57 0:06:43 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7882500 2024-08-30 21:09:15 2024-08-31 01:34:06 2024-08-31 02:31:40 0:57:34 0:48:03 0:09:31 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7882501 2024-08-30 21:09:17 2024-08-31 01:34:06 2024-08-31 02:06:22 0:32:16 0:22:12 0:10:04 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7882502 2024-08-30 21:09:18 2024-08-31 01:34:47 2024-08-31 01:59:45 0:24:58 0:18:31 0:06:27 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} 2
pass 7882503 2024-08-30 21:09:20 2024-08-31 01:34:47 2024-08-31 02:09:49 0:35:02 0:28:36 0:06:26 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7882504 2024-08-30 21:09:21 2024-08-31 01:34:47 2024-08-31 02:14:17 0:39:30 0:31:16 0:08:14 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7882505 2024-08-30 21:09:22 2024-08-31 01:35:28 2024-08-31 02:37:33 1:02:05 0:52:18 0:09:47 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7882506 2024-08-30 21:09:23 2024-08-31 01:36:08 2024-08-31 01:50:55 0:14:47 0:06:55 0:07:52 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_adoption} 1
pass 7882507 2024-08-30 21:09:25 2024-08-31 01:37:59 2024-08-31 02:03:00 0:25:01 0:14:18 0:10:43 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/off orchestrator_cli} 2
pass 7882508 2024-08-30 21:09:26 2024-08-31 01:39:19 2024-08-31 01:54:18 0:14:59 0:09:13 0:05:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7882509 2024-08-30 21:09:27 2024-08-31 01:39:30 2024-08-31 02:07:14 0:27:44 0:20:52 0:06:52 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7882510 2024-08-30 21:09:29 2024-08-31 01:40:00 2024-08-31 01:57:20 0:17:20 0:11:07 0:06:13 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
pass 7882511 2024-08-30 21:09:30 2024-08-31 01:40:01 2024-08-31 01:57:58 0:17:57 0:09:39 0:08:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7882512 2024-08-30 21:09:31 2024-08-31 01:40:41 2024-08-31 01:55:40 0:14:59 0:08:38 0:06:21 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} 1
pass 7882513 2024-08-30 21:09:32 2024-08-31 01:40:52 2024-08-31 01:57:50 0:16:58 0:09:16 0:07:42 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7882514 2024-08-30 21:09:34 2024-08-31 01:42:12 2024-08-31 02:13:13 0:31:01 0:24:51 0:06:10 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
pass 7882515 2024-08-30 21:09:35 2024-08-31 01:42:13 2024-08-31 02:05:12 0:22:59 0:15:42 0:07:17 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_monitoring_stack_basic} 3
pass 7882516 2024-08-30 21:09:36 2024-08-31 01:42:23 2024-08-31 01:59:53 0:17:30 0:11:20 0:06:10 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7882517 2024-08-30 21:09:38 2024-08-31 01:43:04 2024-08-31 02:12:48 0:29:44 0:19:44 0:10:00 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
dead 7882518 2024-08-30 21:09:39 2024-08-31 01:43:04 2024-08-31 09:54:17 8:11:13 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

hit max job timeout

pass 7882519 2024-08-30 21:09:40 2024-08-31 01:43:14 2024-08-31 02:00:13 0:16:59 0:11:12 0:05:47 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7882520 2024-08-30 21:09:41 2024-08-31 01:43:25 2024-08-31 02:00:01 0:16:36 0:10:54 0:05:42 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7882521 2024-08-30 21:09:42 2024-08-31 01:43:25 2024-08-31 02:55:25 1:12:00 1:02:06 0:09:54 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} 2
fail 7882522 2024-08-30 21:09:44 2024-08-31 01:43:25 2024-08-31 02:28:31 0:45:06 0:35:26 0:09:40 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882523 2024-08-30 21:09:45 2024-08-31 01:45:46 2024-08-31 02:02:25 0:16:39 0:10:54 0:05:45 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_cephadm_timeout} 1
pass 7882524 2024-08-30 21:09:46 2024-08-31 01:45:46 2024-08-31 02:01:36 0:15:50 0:07:41 0:08:09 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7882525 2024-08-30 21:09:48 2024-08-31 01:46:37 2024-08-31 02:03:41 0:17:04 0:10:36 0:06:28 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7882526 2024-08-30 21:09:49 2024-08-31 01:46:37 2024-08-31 02:13:58 0:27:21 0:20:20 0:07:01 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rotate-keys} 2
pass 7882527 2024-08-30 21:09:50 2024-08-31 01:46:58 2024-08-31 02:04:05 0:17:07 0:10:47 0:06:20 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7882528 2024-08-30 21:09:51 2024-08-31 01:47:08 2024-08-31 02:06:28 0:19:20 0:11:57 0:07:23 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_set_mon_crush_locations} 3
pass 7882529 2024-08-30 21:09:53 2024-08-31 01:47:18 2024-08-31 02:19:02 0:31:44 0:21:17 0:10:27 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7882530 2024-08-30 21:09:54 2024-08-31 01:47:19 2024-08-31 02:07:02 0:19:43 0:11:10 0:08:33 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7882531 2024-08-30 21:09:55 2024-08-31 01:49:50 2024-08-31 02:05:19 0:15:29 0:07:44 0:07:45 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
pass 7882532 2024-08-30 21:09:57 2024-08-31 01:50:20 2024-08-31 02:20:57 0:30:37 0:20:09 0:10:28 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7882533 2024-08-30 21:09:58 2024-08-31 01:51:10 2024-08-31 02:44:34 0:53:24 0:40:34 0:12:50 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
pass 7882534 2024-08-30 21:09:59 2024-08-31 01:54:21 2024-08-31 02:09:25 0:15:04 0:08:02 0:07:02 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_ca_signed_key} 2
pass 7882535 2024-08-30 21:10:00 2024-08-31 01:54:22 2024-08-31 02:13:19 0:18:57 0:11:49 0:07:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7882536 2024-08-30 21:10:02 2024-08-31 01:54:32 2024-08-31 02:59:55 1:05:23 0:58:45 0:06:38 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} 2
fail 7882537 2024-08-30 21:10:03 2024-08-31 01:54:52 2024-08-31 02:30:27 0:35:35 0:28:46 0:06:49 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-08-31T02:20:00.000226+0000 mon.smithi073 (mon.0) 421 : cluster [WRN] osd.3 (root=default,host=smithi178) is down" in cluster log

pass 7882538 2024-08-30 21:10:04 2024-08-31 01:54:53 2024-08-31 02:27:28 0:32:35 0:23:04 0:09:31 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli} 1
fail 7882539 2024-08-30 21:10:05 2024-08-31 01:55:53 2024-08-31 02:16:14 0:20:21 0:08:12 0:12:09 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/master} 1
Failure Reason:

Command failed on smithi029 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7882540 2024-08-30 21:10:07 2024-08-31 01:57:34 2024-08-31 02:25:30 0:27:56 0:18:51 0:09:05 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
pass 7882541 2024-08-30 21:10:08 2024-08-31 01:58:04 2024-08-31 02:32:18 0:34:14 0:23:42 0:10:32 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7882542 2024-08-30 21:10:09 2024-08-31 01:58:05 2024-08-31 02:54:05 0:56:00 0:44:39 0:11:21 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} 2
pass 7882543 2024-08-30 21:10:10 2024-08-31 01:58:15 2024-08-31 02:15:54 0:17:39 0:10:52 0:06:47 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_cephadm} 1
pass 7882544 2024-08-30 21:10:11 2024-08-31 01:59:05 2024-08-31 02:18:24 0:19:19 0:12:33 0:06:46 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7882545 2024-08-30 21:10:13 2024-08-31 01:59:36 2024-08-31 02:16:45 0:17:09 0:09:51 0:07:18 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7882546 2024-08-30 21:10:14 2024-08-31 01:59:36 2024-08-31 02:16:45 0:17:09 0:11:03 0:06:06 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7882547 2024-08-30 21:10:15 2024-08-31 01:59:57 2024-08-31 02:18:56 0:18:59 0:11:29 0:07:30 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
fail 7882548 2024-08-30 21:10:16 2024-08-31 02:00:07 2024-08-31 02:17:28 0:17:21 0:09:07 0:08:14 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi176 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244a32d3583bb0aa259286ff61157888a54a3ca5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

pass 7882549 2024-08-30 21:10:18 2024-08-31 02:00:07 2024-08-31 02:31:58 0:31:51 0:21:03 0:10:48 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7882550 2024-08-30 21:10:19 2024-08-31 02:00:18 2024-08-31 02:33:18 0:33:00 0:25:27 0:07:33 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} 2
fail 7882551 2024-08-30 21:10:20 2024-08-31 02:00:28 2024-08-31 02:44:09 0:43:41 0:35:47 0:07:54 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882552 2024-08-30 21:10:22 2024-08-31 02:01:09 2024-08-31 02:29:14 0:28:05 0:19:44 0:08:21 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli_mon} 5
pass 7882553 2024-08-30 21:10:23 2024-08-31 02:01:59 2024-08-31 02:18:11 0:16:12 0:08:53 0:07:19 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7882554 2024-08-30 21:10:24 2024-08-31 02:02:50 2024-08-31 02:17:53 0:15:03 0:07:53 0:07:10 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7882555 2024-08-30 21:10:25 2024-08-31 02:02:50 2024-08-31 02:20:08 0:17:18 0:10:19 0:06:59 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7882556 2024-08-30 21:10:27 2024-08-31 02:03:10 2024-08-31 02:33:00 0:29:50 0:19:39 0:10:11 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} 1
pass 7882557 2024-08-30 21:10:28 2024-08-31 02:03:21 2024-08-31 02:18:59 0:15:38 0:07:56 0:07:42 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
pass 7882558 2024-08-30 21:10:29 2024-08-31 02:04:01 2024-08-31 02:21:21 0:17:20 0:10:20 0:07:00 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7882559 2024-08-30 21:10:30 2024-08-31 02:04:21 2024-08-31 02:22:22 0:18:01 0:10:07 0:07:54 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_extra_daemon_features} 2
pass 7882560 2024-08-30 21:10:32 2024-08-31 02:05:22 2024-08-31 02:22:33 0:17:11 0:10:06 0:07:05 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7882561 2024-08-30 21:10:33 2024-08-31 02:05:33 2024-08-31 02:30:23 0:24:50 0:18:31 0:06:19 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
fail 7882562 2024-08-30 21:10:34 2024-08-31 02:05:33 2024-08-31 02:54:37 0:49:04 0:41:54 0:07:10 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

Command failed on smithi070 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 862b0c56-673e-11ef-bcd4-c7b262605968 -e sha1=34b9d27bc925f169ace27b9f1f204536aa49fc0b -- bash -c \'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e \'"\'"\'.up_to_date | length == 7\'"\'"\'\''

dead 7882563 2024-08-30 21:10:35 2024-08-31 02:05:43 2024-08-31 10:15:56 8:10:13 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7882564 2024-08-30 21:10:37 2024-08-31 02:06:34 2024-08-31 02:23:58 0:17:24 0:11:23 0:06:01 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_host_drain} 3
pass 7882565 2024-08-30 21:10:38 2024-08-31 02:06:44 2024-08-31 02:25:54 0:19:10 0:11:41 0:07:29 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw 3-final} 2
fail 7882566 2024-08-30 21:10:39 2024-08-31 02:06:55 2024-08-31 02:48:47 0:41:52 0:34:52 0:07:00 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882567 2024-08-30 21:10:40 2024-08-31 02:07:15 2024-08-31 02:20:22 0:13:07 0:06:46 0:06:21 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_adoption} 1
pass 7882568 2024-08-30 21:10:41 2024-08-31 02:07:25 2024-08-31 02:35:19 0:27:54 0:17:57 0:09:57 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7882569 2024-08-30 21:10:43 2024-08-31 02:07:26 2024-08-31 02:22:48 0:15:22 0:09:26 0:05:56 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
pass 7882570 2024-08-30 21:10:44 2024-08-31 02:07:26 2024-08-31 02:43:09 0:35:43 0:24:43 0:11:00 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
pass 7882571 2024-08-30 21:10:45 2024-08-31 02:07:36 2024-08-31 02:26:41 0:19:05 0:09:34 0:09:31 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7882572 2024-08-30 21:10:47 2024-08-31 02:09:37 2024-08-31 03:05:37 0:56:00 0:46:57 0:09:03 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} 2
pass 7882573 2024-08-30 21:10:48 2024-08-31 02:09:58 2024-08-31 02:39:48 0:29:50 0:19:56 0:09:54 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7882574 2024-08-30 21:10:49 2024-08-31 02:10:08 2024-08-31 02:30:02 0:19:54 0:09:45 0:10:09 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7882575 2024-08-30 21:10:51 2024-08-31 02:12:59 2024-08-31 02:31:13 0:18:14 0:09:53 0:08:21 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7882576 2024-08-30 21:10:52 2024-08-31 02:13:29 2024-08-31 02:28:38 0:15:09 0:08:09 0:07:00 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
pass 7882577 2024-08-30 21:10:53 2024-08-31 02:13:29 2024-08-31 02:41:08 0:27:39 0:20:31 0:07:08 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7882578 2024-08-30 21:10:54 2024-08-31 02:14:10 2024-08-31 02:31:38 0:17:28 0:10:30 0:06:58 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7882579 2024-08-30 21:10:56 2024-08-31 02:14:40 2024-08-31 02:40:10 0:25:30 0:15:36 0:09:54 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_monitoring_stack_basic} 3
pass 7882580 2024-08-30 21:10:57 2024-08-31 02:17:01 2024-08-31 02:34:11 0:17:10 0:10:15 0:06:55 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7882581 2024-08-30 21:10:58 2024-08-31 02:17:01 2024-08-31 02:54:32 0:37:31 0:29:15 0:08:16 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7882582 2024-08-30 21:10:59 2024-08-31 02:17:42 2024-08-31 02:47:47 0:30:05 0:19:24 0:10:41 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_cephadm_timeout} 1
pass 7882583 2024-08-30 21:11:01 2024-08-31 02:18:12 2024-08-31 02:47:58 0:29:46 0:18:47 0:10:59 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_domain} 2
fail 7882584 2024-08-30 21:11:02 2024-08-31 02:18:22 2024-08-31 02:35:44 0:17:22 0:07:57 0:09:25 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} 1
Failure Reason:

Command failed on smithi012 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7882585 2024-08-30 21:11:03 2024-08-31 02:18:33 2024-08-31 02:48:45 0:30:12 0:19:54 0:10:18 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7882586 2024-08-30 21:11:05 2024-08-31 02:18:43 2024-08-31 03:10:10 0:51:27 0:43:49 0:07:38 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} 2
pass 7882587 2024-08-30 21:11:06 2024-08-31 02:19:14 2024-08-31 03:12:47 0:53:33 0:44:05 0:09:28 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
pass 7882588 2024-08-30 21:11:07 2024-08-31 02:19:14 2024-08-31 02:38:44 0:19:30 0:11:38 0:07:52 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
dead 7882589 2024-08-30 21:11:08 2024-08-31 02:19:24 2024-08-31 10:29:43 8:10:19 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_rgw_multisite} 3
Failure Reason:

hit max job timeout

pass 7882590 2024-08-30 21:11:10 2024-08-31 02:20:25 2024-08-31 02:40:03 0:19:38 0:11:35 0:08:03 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7882591 2024-08-30 21:11:11 2024-08-31 02:21:05 2024-08-31 02:39:56 0:18:51 0:12:00 0:06:51 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7882592 2024-08-30 21:11:12 2024-08-31 02:21:06 2024-08-31 02:38:04 0:16:58 0:10:12 0:06:46 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7882593 2024-08-30 21:11:14 2024-08-31 02:21:06 2024-08-31 02:52:49 0:31:43 0:21:38 0:10:05 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7882594 2024-08-30 21:11:15 2024-08-31 02:21:16 2024-08-31 03:15:29 0:54:13 0:44:03 0:10:10 smithi main ubuntu 22.04 orch/cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} 2
pass 7882595 2024-08-30 21:11:16 2024-08-31 02:21:37 2024-08-31 02:54:23 0:32:46 0:21:28 0:11:18 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7882596 2024-08-30 21:11:17 2024-08-31 02:22:47 2024-08-31 03:04:27 0:41:40 0:34:29 0:07:11 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882597 2024-08-30 21:11:19 2024-08-31 02:22:58 2024-08-31 03:03:14 0:40:16 0:31:34 0:08:42 smithi main centos 9.stream orch/cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
pass 7882598 2024-08-30 21:11:20 2024-08-31 02:24:08 2024-08-31 03:25:30 1:01:22 0:51:34 0:09:48 smithi main ubuntu 22.04 orch/cephadm/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
pass 7882599 2024-08-30 21:11:21 2024-08-31 02:24:09 2024-08-31 02:41:18 0:17:09 0:10:32 0:06:37 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_orch_cli} 1
pass 7882600 2024-08-30 21:11:23 2024-08-31 02:24:19 2024-08-31 02:40:33 0:16:14 0:09:10 0:07:04 smithi main centos 9.stream orch/cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream} 2-node-mgr agent/off orchestrator_cli} 2
fail 7882601 2024-08-30 21:11:24 2024-08-31 02:25:40 2024-08-31 02:53:15 0:27:35 0:20:19 0:07:16 smithi main centos 9.stream orch/cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-08-31T02:36:27.519386+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

pass 7882602 2024-08-30 21:11:25 2024-08-31 02:26:10 2024-08-31 02:43:44 0:17:34 0:09:28 0:08:06 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_basic} 2
pass 7882603 2024-08-30 21:11:26 2024-08-31 02:26:50 2024-08-31 02:45:48 0:18:58 0:12:23 0:06:35 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7882604 2024-08-30 21:11:28 2024-08-31 02:26:51 2024-08-31 02:54:35 0:27:44 0:18:23 0:09:21 smithi main ubuntu 22.04 orch/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
pass 7882605 2024-08-30 21:11:29 2024-08-31 02:26:51 2024-08-31 02:43:47 0:16:56 0:08:03 0:08:53 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7882606 2024-08-30 21:11:30 2024-08-31 02:28:52 2024-08-31 03:37:47 1:08:55 1:03:28 0:05:27 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
pass 7882607 2024-08-30 21:11:32 2024-08-31 02:28:52 2024-08-31 02:48:12 0:19:20 0:12:54 0:06:26 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7882608 2024-08-30 21:11:33 2024-08-31 02:29:33 2024-08-31 02:57:17 0:27:44 0:16:54 0:10:50 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_ca_signed_key} 2
pass 7882609 2024-08-30 21:11:34 2024-08-31 02:29:33 2024-08-31 02:59:59 0:30:26 0:19:38 0:10:48 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7882610 2024-08-30 21:11:35 2024-08-31 02:29:33 2024-08-31 03:00:04 0:30:31 0:20:41 0:09:50 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7882611 2024-08-30 21:11:37 2024-08-31 02:30:34 2024-08-31 02:48:08 0:17:34 0:11:04 0:06:30 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7882612 2024-08-30 21:11:38 2024-08-31 02:30:45 2024-08-31 02:47:44 0:16:59 0:10:57 0:06:02 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_cephadm} 1
pass 7882613 2024-08-30 21:11:39 2024-08-31 02:30:45 2024-08-31 03:24:38 0:53:53 0:45:54 0:07:59 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
pass 7882614 2024-08-30 21:11:40 2024-08-31 02:31:35 2024-08-31 02:48:56 0:17:21 0:11:07 0:06:14 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7882615 2024-08-30 21:11:42 2024-08-31 02:31:56 2024-08-31 02:57:42 0:25:46 0:18:18 0:07:28 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} 2
fail 7882616 2024-08-30 21:11:43 2024-08-31 02:31:57 2024-08-31 03:09:22 0:37:25 0:29:33 0:07:52 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-08-31T03:00:00.000247+0000 mon.smithi022 (mon.0) 436 : cluster [WRN] osd.4 (root=default,host=smithi089) is down" in cluster log

pass 7882617 2024-08-30 21:11:44 2024-08-31 02:32:17 2024-08-31 03:01:29 0:29:12 0:19:51 0:09:21 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli_mon} 5
pass 7882618 2024-08-30 21:11:46 2024-08-31 02:33:38 2024-08-31 02:49:47 0:16:09 0:08:03 0:08:06 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_domain} 2
pass 7882619 2024-08-30 21:11:47 2024-08-31 02:34:28 2024-08-31 02:52:51 0:18:23 0:10:05 0:08:18 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7882620 2024-08-30 21:11:48 2024-08-31 02:35:39 2024-08-31 02:54:24 0:18:45 0:10:51 0:07:54 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7882621 2024-08-30 21:11:49 2024-08-31 02:37:49 2024-08-31 03:29:52 0:52:03 0:41:19 0:10:44 smithi main ubuntu 22.04 orch/cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
pass 7882622 2024-08-30 21:11:51 2024-08-31 02:38:20 2024-08-31 03:08:27 0:30:07 0:20:05 0:10:02 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7882623 2024-08-30 21:11:52 2024-08-31 02:38:50 2024-08-31 02:48:19 0:09:29 0:03:24 0:06:05 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/classic task/test_cephadm_repos} 1
dead 7882624 2024-08-30 21:11:53 2024-08-31 02:39:01 2024-08-31 10:48:41 8:09:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
Failure Reason:

hit max job timeout

pass 7882625 2024-08-30 21:11:55 2024-08-31 02:40:01 2024-08-31 02:55:25 0:15:24 0:08:24 0:07:00 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
pass 7882626 2024-08-30 21:11:56 2024-08-31 02:40:12 2024-08-31 02:59:16 0:19:04 0:12:00 0:07:04 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7882627 2024-08-30 21:11:57 2024-08-31 02:40:22 2024-08-31 03:10:17 0:29:55 0:18:47 0:11:08 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/connectivity task/test_extra_daemon_features} 2
fail 7882628 2024-08-30 21:11:58 2024-08-31 02:40:22 2024-08-31 03:21:34 0:41:12 0:34:20 0:06:52 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7882629 2024-08-30 21:12:00 2024-08-31 02:40:43 2024-08-31 02:59:11 0:18:28 0:08:05 0:10:23 smithi main ubuntu 22.04 orch/rook/smoke/{0-distro/ubuntu_22.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} 3
Failure Reason:

Command failed on smithi003 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils"

pass 7882630 2024-08-30 21:12:01 2024-08-31 02:41:23 2024-08-31 03:03:07 0:21:44 0:12:42 0:09:02 smithi main ubuntu 22.04 orch/cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_adoption} 1
pass 7882631 2024-08-30 21:12:02 2024-08-31 02:41:33 2024-08-31 02:59:26 0:17:53 0:09:26 0:08:27 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7882632 2024-08-30 21:12:03 2024-08-31 02:41:44 2024-08-31 03:13:08 0:31:24 0:19:14 0:12:10 smithi main ubuntu 22.04 orch/cephadm/smb/{0-distro/ubuntu_22.04 tasks/deploy_smb_basic} 2
pass 7882633 2024-08-30 21:12:05 2024-08-31 02:43:24 2024-08-31 03:11:40 0:28:16 0:18:48 0:09:28 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7882634 2024-08-30 21:12:06 2024-08-31 02:43:55 2024-08-31 03:10:58 0:27:03 0:20:15 0:06:48 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} 2
pass 7882635 2024-08-30 21:12:07 2024-08-31 02:44:05 2024-08-31 03:01:23 0:17:18 0:09:38 0:07:40 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
pass 7882636 2024-08-30 21:12:08 2024-08-31 02:44:26 2024-08-31 03:21:13 0:36:47 0:29:44 0:07:03 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
pass 7882637 2024-08-30 21:12:10 2024-08-31 02:44:46 2024-08-31 03:05:11 0:20:25 0:12:03 0:08:22 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/classic task/test_host_drain} 3
pass 7882638 2024-08-30 21:12:11 2024-08-31 02:46:07 2024-08-31 03:21:44 0:35:37 0:24:27 0:11:10 smithi main ubuntu 22.04 orch/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7882639 2024-08-30 21:12:12 2024-08-31 02:48:08 2024-08-31 03:05:42 0:17:34 0:10:41 0:06:53 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
pass 7882640 2024-08-30 21:12:14 2024-08-31 02:48:18 2024-08-31 03:57:53 1:09:35 1:00:10 0:09:25 smithi main ubuntu 22.04 orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
pass 7882641 2024-08-30 21:12:15 2024-08-31 02:48:18 2024-08-31 03:20:08 0:31:50 0:21:09 0:10:41 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7882642 2024-08-30 21:12:16 2024-08-31 02:48:29 2024-08-31 03:18:04 0:29:35 0:19:43 0:09:52 smithi main ubuntu 22.04 orch/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
pass 7882643 2024-08-30 21:12:17 2024-08-31 02:48:29 2024-08-31 03:05:48 0:17:19 0:09:50 0:07:29 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7882644 2024-08-30 21:12:19 2024-08-31 02:48:29 2024-08-31 03:26:26 0:37:57 0:29:45 0:08:12 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7882645 2024-08-30 21:12:20 2024-08-31 02:49:00 2024-08-31 03:06:09 0:17:09 0:10:24 0:06:45 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_cephadm_timeout} 1
pass 7882646 2024-08-30 21:12:21 2024-08-31 02:49:00 2024-08-31 03:14:46 0:25:46 0:14:43 0:11:03 smithi main ubuntu 22.04 orch/cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_22.04} 2-node-mgr agent/on orchestrator_cli} 2
pass 7882647 2024-08-30 21:12:23 2024-08-31 02:49:11 2024-08-31 03:04:24 0:15:13 0:07:43 0:07:30 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_domain} 2
pass 7882648 2024-08-30 21:12:24 2024-08-31 02:49:11 2024-08-31 03:07:16 0:18:05 0:11:05 0:07:00 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7882649 2024-08-30 21:12:25 2024-08-31 02:50:01 2024-08-31 03:04:52 0:14:51 0:09:21 0:05:30 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} 1
pass 7882650 2024-08-30 21:12:26 2024-08-31 02:50:02 2024-08-31 03:08:15 0:18:13 0:08:10 0:10:03 smithi main centos 9.stream orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
pass 7882651 2024-08-30 21:12:28 2024-08-31 02:53:02 2024-08-31 03:10:22 0:17:20 0:11:07 0:06:13 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
pass 7882652 2024-08-30 21:12:29 2024-08-31 02:53:33 2024-08-31 03:40:38 0:47:05 0:40:58 0:06:07 smithi main centos 9.stream orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
pass 7882653 2024-08-30 21:12:30 2024-08-31 02:53:33 2024-08-31 03:49:40 0:56:07 0:48:02 0:08:05 smithi main centos 9.stream orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} 2
pass 7882654 2024-08-30 21:12:31 2024-08-31 02:54:24 2024-08-31 03:30:14 0:35:50 0:24:55 0:10:55 smithi main ubuntu 22.04 orch/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/classic task/test_monitoring_stack_basic} 3
pass 7882655 2024-08-30 21:12:33 2024-08-31 02:54:34 2024-08-31 03:26:14 0:31:40 0:20:56 0:10:44 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7882656 2024-08-30 21:12:34 2024-08-31 02:54:45 2024-08-31 03:11:54 0:17:09 0:10:15 0:06:54 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7882657 2024-08-30 21:12:35 2024-08-31 02:54:45 2024-08-31 03:11:55 0:17:10 0:11:02 0:06:08 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7882658 2024-08-30 21:12:37 2024-08-31 02:54:55 2024-08-31 03:12:20 0:17:25 0:11:27 0:05:58 smithi main centos 9.stream orch/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
fail 7882659 2024-08-30 21:12:38 2024-08-31 02:55:36 2024-08-31 03:13:06 0:17:30 0:09:54 0:07:36 smithi main centos 9.stream orch/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/connectivity task/test_rgw_multisite} 3
Failure Reason:

Command failed on smithi138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:34b9d27bc925f169ace27b9f1f204536aa49fc0b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f5a4d326-6745-11ef-bcd4-c7b262605968 -- bash -c \'set -e\nset -x\nwhile true; do TOKEN=$(ceph rgw realm tokens | jq -r \'"\'"\'.[0].token\'"\'"\'); echo $TOKEN; if [ "$TOKEN" != "master zone has no endpoint" ]; then break; fi; sleep 5; done\nTOKENS=$(ceph rgw realm tokens)\necho $TOKENS | jq --exit-status \'"\'"\'.[0].realm == "myrealm1"\'"\'"\'\necho $TOKENS | jq --exit-status \'"\'"\'.[0].token\'"\'"\'\nTOKEN_JSON=$(ceph rgw realm tokens | jq -r \'"\'"\'.[0].token\'"\'"\' | base64 --decode)\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.realm_name == "myrealm1"\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.endpoint | test("http://.+:\\\\d+")\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.realm_id | test("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$")\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.access_key\'"\'"\'\necho $TOKEN_JSON | jq --exit-status \'"\'"\'.secret\'"\'"\'\n\''

fail 7882660 2024-08-30 21:12:39 2024-08-31 02:55:46 2024-08-31 03:37:17 0:41:31 0:34:58 0:06:33 smithi main centos 9.stream orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7882661 2024-08-30 21:12:41 2024-08-31 02:56:17 2024-08-31 03:14:39 0:18:22 0:10:40 0:07:42 smithi main centos 9.stream orch/cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli} 1
pass 7882662 2024-08-30 21:12:42 2024-08-31 02:57:27 2024-08-31 03:14:47 0:17:20 0:09:45 0:07:35 smithi main centos 9.stream orch/cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_basic} 2
pass 7882663 2024-08-30 21:12:43 2024-08-31 02:57:59 2024-08-31 03:18:27 0:20:28 0:13:01 0:07:27 smithi main centos 9.stream orch/cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7882664 2024-08-30 21:12:45 2024-08-31 02:59:29 2024-08-31 03:33:03 0:33:34 0:23:30 0:10:04 smithi main ubuntu 22.04 orch/cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7882665 2024-08-30 21:12:46 2024-08-31 02:59:30 2024-08-31 03:36:35 0:37:05 0:29:21 0:07:44 smithi main centos 9.stream orch/cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2