Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7574458 2024-02-27 05:08:26 2024-02-27 05:42:31 2024-02-27 06:08:20 0:25:49 0:14:59 0:10:50 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
pass 7574459 2024-02-27 05:08:27 2024-02-27 05:42:31 2024-02-27 06:14:17 0:31:46 0:21:40 0:10:06 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
fail 7574460 2024-02-27 05:08:28 2024-02-27 05:42:32 2024-02-27 06:59:53 1:17:21 1:00:58 0:16:23 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-27T06:40:00.000211+0000 mon.a (mon.0) 1620 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log

pass 7574461 2024-02-27 05:08:30 2024-02-27 05:48:13 2024-02-27 06:29:16 0:41:03 0:25:11 0:15:52 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
fail 7574462 2024-02-27 05:08:31 2024-02-27 05:53:34 2024-02-27 06:30:43 0:37:09 0:25:44 0:11:25 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi057 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f869c4fa-d536-11ee-95c3-87774f69a715 -e sha1=635ee6850c57c98cc6ba932cade45f9cc015aa32 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''

fail 7574463 2024-02-27 05:08:32 2024-02-27 05:53:35 2024-02-27 06:45:11 0:51:36 0:35:10 0:16:26 smithi main centos 9.stream orch:cephadm/mgr-nfs-upgrade/{0-centos_9.stream 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} 2
Failure Reason:

"2024-02-27T06:21:27.967462+0000 mon.smithi131 (mon.0) 743 : cluster [WRN] Health check failed: Degraded data redundancy: 1/723 objects degraded (0.138%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7574464 2024-02-27 05:08:33 2024-02-27 05:55:55 2024-02-27 06:28:14 0:32:19 0:22:23 0:09:56 smithi main ubuntu 22.04 orch:cephadm/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} 1
Failure Reason:

Test failure: test_create_and_delete_export (tasks.cephfs.test_nfs.TestNFS)

pass 7574465 2024-02-27 05:08:34 2024-02-27 05:55:55 2024-02-27 06:22:51 0:26:56 0:13:50 0:13:06 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/off orchestrator_cli} 2
pass 7574466 2024-02-27 05:08:36 2024-02-27 05:57:36 2024-02-27 06:37:23 0:39:47 0:30:33 0:09:14 smithi main centos 9.stream orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
pass 7574467 2024-02-27 05:08:37 2024-02-27 05:58:06 2024-02-27 06:29:22 0:31:16 0:18:04 0:13:12 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7574468 2024-02-27 05:08:38 2024-02-27 05:58:07 2024-02-27 06:26:22 0:28:15 0:18:16 0:09:59 smithi main ubuntu 22.04 orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/basic 3-final} 1
pass 7574469 2024-02-27 05:08:40 2024-02-27 05:58:37 2024-02-27 06:21:13 0:22:36 0:12:00 0:10:36 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 3
pass 7574470 2024-02-27 05:08:41 2024-02-27 05:59:58 2024-02-27 06:40:34 0:40:36 0:25:29 0:15:07 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
pass 7574471 2024-02-27 05:08:42 2024-02-27 06:01:18 2024-02-27 06:50:22 0:49:04 0:38:42 0:10:22 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} 2
pass 7574472 2024-02-27 05:08:44 2024-02-27 06:01:59 2024-02-27 06:27:08 0:25:09 0:14:57 0:10:12 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
pass 7574473 2024-02-27 05:08:45 2024-02-27 06:02:09 2024-02-27 06:31:15 0:29:06 0:17:10 0:11:56 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7574474 2024-02-27 05:08:46 2024-02-27 06:02:10 2024-02-27 06:32:05 0:29:55 0:19:06 0:10:49 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7574475 2024-02-27 05:08:48 2024-02-27 06:02:20 2024-02-27 06:38:10 0:35:50 0:21:40 0:14:10 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7574476 2024-02-27 05:08:49 2024-02-27 06:06:11 2024-02-27 06:51:12 0:45:01 0:32:47 0:12:14 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7574477 2024-02-27 05:08:50 2024-02-27 06:06:11 2024-02-27 06:56:47 0:50:36 0:39:40 0:10:56 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} 2
Failure Reason:

"2024-02-27T06:25:44.349078+0000 mon.a (mon.0) 318 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7574478 2024-02-27 05:08:52 2024-02-27 06:06:32 2024-02-27 06:36:59 0:30:27 0:16:59 0:13:28 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7574479 2024-02-27 05:08:53 2024-02-27 06:07:52 2024-02-27 06:34:10 0:26:18 0:14:49 0:11:29 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7574480 2024-02-27 05:08:54 2024-02-27 06:08:22 2024-02-27 06:30:06 0:21:44 0:12:26 0:09:18 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/classic task/test_adoption} 1
pass 7574481 2024-02-27 05:08:56 2024-02-27 06:08:23 2024-02-27 06:46:29 0:38:06 0:25:17 0:12:49 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
pass 7574482 2024-02-27 05:08:57 2024-02-27 06:10:23 2024-02-27 06:37:27 0:27:04 0:14:05 0:12:59 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
pass 7574483 2024-02-27 05:08:58 2024-02-27 06:10:24 2024-02-27 06:58:20 0:47:56 0:33:18 0:14:38 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} 2
pass 7574484 2024-02-27 05:09:00 2024-02-27 06:12:54 2024-02-27 07:24:48 1:11:54 1:01:14 0:10:40 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} 2
pass 7574485 2024-02-27 05:09:01 2024-02-27 06:12:55 2024-02-27 06:37:13 0:24:18 0:12:03 0:12:15 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} 2
pass 7574486 2024-02-27 05:09:02 2024-02-27 06:13:55 2024-02-27 06:39:12 0:25:17 0:14:53 0:10:24 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
fail 7574487 2024-02-27 05:09:04 2024-02-27 06:14:06 2024-02-27 07:33:54 1:19:48 1:07:32 0:12:16 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7574488 2024-02-27 05:09:05 2024-02-27 06:14:06 2024-02-27 06:44:08 0:30:02 0:19:54 0:10:08 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
pass 7574489 2024-02-27 05:09:06 2024-02-27 06:14:26 2024-02-27 06:50:55 0:36:29 0:19:53 0:16:36 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
pass 7574490 2024-02-27 05:09:08 2024-02-27 06:21:18 2024-02-27 06:50:32 0:29:14 0:18:07 0:11:07 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
pass 7574491 2024-02-27 05:09:09 2024-02-27 06:21:18 2024-02-27 06:47:48 0:26:30 0:12:04 0:14:26 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 3
pass 7574492 2024-02-27 05:09:10 2024-02-27 06:26:29 2024-02-27 06:54:13 0:27:44 0:16:20 0:11:24 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw 3-final} 2
pass 7574493 2024-02-27 05:09:11 2024-02-27 06:27:10 2024-02-27 07:02:31 0:35:21 0:24:46 0:10:35 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_python} 2
pass 7574494 2024-02-27 05:09:13 2024-02-27 06:27:30 2024-02-27 06:44:35 0:17:05 0:06:49 0:10:16 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm_repos} 1
pass 7574495 2024-02-27 05:09:14 2024-02-27 06:27:31 2024-02-27 07:12:17 0:44:46 0:32:51 0:11:55 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7574496 2024-02-27 05:09:15 2024-02-27 06:29:21 2024-02-27 06:54:07 0:24:46 0:15:03 0:09:43 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_cephadm_timeout} 1
fail 7574497 2024-02-27 05:09:16 2024-02-27 06:29:21 2024-02-27 07:32:54 1:03:33 0:51:35 0:11:58 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} 2
Failure Reason:

"2024-02-27T06:47:33.821991+0000 mon.a (mon.0) 200 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

pass 7574498 2024-02-27 05:09:18 2024-02-27 06:29:32 2024-02-27 06:53:26 0:23:54 0:13:32 0:10:22 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
pass 7574499 2024-02-27 05:09:19 2024-02-27 06:30:12 2024-02-27 07:00:19 0:30:07 0:19:13 0:10:54 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/basic 3-final} 2
pass 7574500 2024-02-27 05:09:20 2024-02-27 06:30:43 2024-02-27 07:22:31 0:51:48 0:40:09 0:11:39 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} 2
pass 7574501 2024-02-27 05:09:21 2024-02-27 06:31:23 2024-02-27 06:57:19 0:25:56 0:15:07 0:10:49 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2
pass 7574502 2024-02-27 05:09:23 2024-02-27 06:32:14 2024-02-27 07:00:02 0:27:48 0:14:28 0:13:20 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
fail 7574503 2024-02-27 05:09:24 2024-02-27 06:34:14 2024-02-27 07:04:11 0:29:57 0:15:16 0:14:41 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-02-27T07:00:00.000189+0000 mon.a (mon.0) 1029 : cluster 3 [WRN] CEPHADM_AGENT_DOWN: 2 Cephadm Agent(s) are not reporting. Hosts may be offline" in cluster log

pass 7574504 2024-02-27 05:09:25 2024-02-27 06:37:05 2024-02-27 07:02:39 0:25:34 0:14:23 0:11:11 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/iscsi 3-final} 2
fail 7574505 2024-02-27 05:09:26 2024-02-27 06:37:16 2024-02-27 07:26:36 0:49:20 0:39:47 0:09:33 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7574506 2024-02-27 05:09:28 2024-02-27 06:37:26 2024-02-27 07:14:42 0:37:16 0:28:02 0:09:14 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rotate-keys} 2
fail 7574507 2024-02-27 05:09:29 2024-02-27 06:37:26 2024-02-27 07:12:18 0:34:52 0:23:06 0:11:46 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_host_drain} 3
Failure Reason:

"2024-02-27T07:10:00.000216+0000 mon.a (mon.0) 706 : cluster 3 [WRN] CEPHADM_STRAY_HOST: 1 stray host(s) with 1 daemon(s) not managed by cephadm" in cluster log

pass 7574508 2024-02-27 05:09:30 2024-02-27 06:38:17 2024-02-27 07:00:58 0:22:41 0:13:24 0:09:17 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_orch_cli} 1
pass 7574509 2024-02-27 05:09:31 2024-02-27 06:38:17 2024-02-27 07:10:54 0:32:37 0:22:22 0:10:15 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/jaeger 3-final} 2
pass 7574510 2024-02-27 05:09:33 2024-02-27 06:39:18 2024-02-27 07:10:14 0:30:56 0:19:22 0:11:34 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
dead 7574511 2024-02-27 05:09:34 2024-02-27 06:40:38 2024-02-27 18:49:01 12:08:23 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

hit max job timeout

pass 7574512 2024-02-27 05:09:35 2024-02-27 06:40:38 2024-02-27 07:03:29 0:22:51 0:13:09 0:09:42 smithi main centos 9.stream orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_9.stream_runc} 2-node-mgr agent/on orchestrator_cli} 2
pass 7574513 2024-02-27 05:09:36 2024-02-27 06:40:39 2024-02-27 07:08:23 0:27:44 0:14:26 0:13:18 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/mirror 3-final} 2
pass 7574514 2024-02-27 05:09:38 2024-02-27 06:41:49 2024-02-27 07:04:39 0:22:50 0:12:51 0:09:59 smithi main centos 9.stream orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
pass 7574515 2024-02-27 05:09:39 2024-02-27 06:41:50 2024-02-27 07:07:46 0:25:56 0:12:38 0:13:18 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
fail 7574516 2024-02-27 05:09:40 2024-02-27 06:44:41 2024-02-27 07:28:05 0:43:24 0:31:53 0:11:31 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-02-27T07:20:00.000137+0000 mon.smithi089 (mon.0) 481 : cluster 3 [WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid" in cluster log

fail 7574517 2024-02-27 05:09:42 2024-02-27 06:47:50 2024-02-27 07:38:42 0:50:52 0:37:39 0:13:13 smithi main ubuntu 20.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-02-27T07:07:56.784412+0000 mon.a (mon.0) 172 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log

pass 7574518 2024-02-27 05:09:43 2024-02-27 06:50:30 2024-02-27 07:15:50 0:25:20 0:16:29 0:08:51 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} 2
fail 7574519 2024-02-27 05:09:44 2024-02-27 06:50:41 2024-02-27 08:05:01 1:14:20 1:04:20 0:10:00 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} 2
Failure Reason:

"2024-02-27T07:30:00.003362+0000 mon.a (mon.0) 2063 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log

fail 7574520 2024-02-27 05:09:45 2024-02-27 06:51:21 2024-02-27 08:05:33 1:14:12 1:01:24 0:12:48 smithi main ubuntu 22.04 orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} 2
Failure Reason:

"2024-02-27T07:30:00.217333+0000 mon.a (mon.0) 1268 : cluster 3 [WRN] CEPHADM_STRAY_DAEMON: 2 stray daemon(s) not managed by cephadm" in cluster log

pass 7574521 2024-02-27 05:09:47 2024-02-27 06:53:32 2024-02-27 07:25:23 0:31:51 0:20:12 0:11:39 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_monitoring_stack_basic} 3
pass 7574522 2024-02-27 05:09:48 2024-02-27 06:54:22 2024-02-27 07:29:09 0:34:47 0:21:55 0:12:52 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} 2
pass 7574523 2024-02-27 05:09:49 2024-02-27 06:57:23 2024-02-27 07:21:51 0:24:28 0:13:24 0:11:04 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
pass 7574524 2024-02-27 05:09:51 2024-02-27 06:58:24 2024-02-27 07:48:59 0:50:35 0:37:16 0:13:19 smithi main ubuntu 22.04 orch:cephadm/no-agent-workunits/{0-distro/ubuntu_22.04 mon_election/connectivity task/test_orch_cli_mon} 5
pass 7574525 2024-02-27 05:09:52 2024-02-27 07:00:24 2024-02-27 07:32:12 0:31:48 0:19:55 0:11:53 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_rgw_multisite} 3
pass 7574526 2024-02-27 05:09:53 2024-02-27 07:02:35 2024-02-27 07:28:13 0:25:38 0:15:44 0:09:54 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} 2
pass 7574527 2024-02-27 05:09:55 2024-02-27 07:02:45 2024-02-27 07:28:26 0:25:41 0:14:59 0:10:42 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
fail 7574528 2024-02-27 05:09:56 2024-02-27 07:03:36 2024-02-27 07:44:52 0:41:16 0:26:13 0:15:03 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed on smithi077 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1a691c5e-d541-11ee-95c3-87774f69a715 -e sha1=635ee6850c57c98cc6ba932cade45f9cc015aa32 -- bash -c \'ceph versions | jq -e \'"\'"\'.mgr | keys\'"\'"\' | grep $sha1\''

pass 7574529 2024-02-27 05:09:57 2024-02-27 07:07:47 2024-02-27 07:34:57 0:27:10 0:17:21 0:09:49 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
pass 7574530 2024-02-27 05:09:59 2024-02-27 07:07:47 2024-02-27 07:42:50 0:35:03 0:22:48 0:12:15 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} 2
pass 7574531 2024-02-27 05:10:00 2024-02-27 07:08:28 2024-02-27 07:40:03 0:31:35 0:17:03 0:14:32 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_set_mon_crush_locations} 3
pass 7574532 2024-02-27 05:10:01 2024-02-27 07:10:58 2024-02-27 07:45:56 0:34:58 0:24:04 0:10:54 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} 2
pass 7574533 2024-02-27 05:10:03 2024-02-27 07:12:19 2024-02-27 07:38:04 0:25:45 0:13:22 0:12:23 smithi main centos 9.stream orch:cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
fail 7574534 2024-02-27 05:10:04 2024-02-27 07:14:50 2024-02-27 08:05:12 0:50:22 0:39:50 0:10:32 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

pass 7574535 2024-02-27 05:10:05 2024-02-27 07:16:00 2024-02-27 07:36:57 0:20:57 0:09:50 0:11:07 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/connectivity task/test_adoption} 1
pass 7574536 2024-02-27 05:10:07 2024-02-27 07:16:01 2024-02-27 07:47:31 0:31:30 0:15:03 0:16:27 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} 2
pass 7574537 2024-02-27 05:10:08 2024-02-27 07:22:02 2024-02-27 07:44:36 0:22:34 0:11:57 0:10:37 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
fail 7574538 2024-02-27 05:10:09 2024-02-27 07:22:42 2024-02-27 08:31:46 1:09:04 0:56:43 0:12:21 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} 2
Failure Reason:

"2024-02-27T07:50:00.000142+0000 mon.a (mon.0) 896 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 3284/201426 objects degraded (1.630%), 2 pgs degraded" in cluster log

fail 7574539 2024-02-27 05:10:11 2024-02-27 07:24:33 2024-02-27 08:04:03 0:39:30 0:28:19 0:11:11 smithi main centos 9.stream orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} 2
Failure Reason:

"2024-02-27T07:51:40.059228+0000 mon.a (mon.0) 948 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

pass 7574540 2024-02-27 05:10:12 2024-02-27 07:24:53 2024-02-27 07:46:23 0:21:30 0:11:40 0:09:50 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_ca_signed_key} 2
pass 7574541 2024-02-27 05:10:13 2024-02-27 07:25:24 2024-02-27 07:53:38 0:28:14 0:14:07 0:14:07 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/nfs 3-final} 2
pass 7574542 2024-02-27 05:10:15 2024-02-27 07:28:14 2024-02-27 08:03:35 0:35:21 0:24:36 0:10:45 smithi main centos 9.stream orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} 2
pass 7574543 2024-02-27 05:10:16 2024-02-27 07:28:35 2024-02-27 07:55:16 0:26:41 0:17:37 0:09:04 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_cephadm} 1
pass 7574544 2024-02-27 05:10:17 2024-02-27 07:28:35 2024-02-27 07:59:09 0:30:34 0:19:43 0:10:51 smithi main ubuntu 22.04 orch:cephadm/smoke-roleless/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-services/nfs2 3-final} 2
fail 7574545 2024-02-27 05:10:18 2024-02-27 07:29:16 2024-02-27 08:11:08 0:41:52 0:26:52 0:15:00 smithi main centos 9.stream orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

Command failed (workunit test suites/fsstress.sh) on smithi006 with status 126: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=635ee6850c57c98cc6ba932cade45f9cc015aa32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

pass 7574546 2024-02-27 05:10:20 2024-02-27 07:32:16 2024-02-27 08:04:56 0:32:40 0:20:49 0:11:51 smithi main ubuntu 22.04 orch:cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
pass 7574547 2024-02-27 05:10:21 2024-02-27 07:35:07 2024-02-27 08:12:43 0:37:36 0:25:31 0:12:05 smithi main ubuntu 22.04 orch:cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
pass 7574548 2024-02-27 05:10:22 2024-02-27 07:36:58 2024-02-27 08:01:23 0:24:25 0:14:21 0:10:04 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nvmeof 3-final} 2
fail 7574549 2024-02-27 05:10:24 2024-02-27 07:38:08 2024-02-27 07:57:48 0:19:40 0:08:39 0:11:01 smithi main ubuntu 22.04 orch:cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm_repos} 1
Failure Reason:

Command failed (workunit test cephadm/test_repos.sh) on smithi094 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=635ee6850c57c98cc6ba932cade45f9cc015aa32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

pass 7574550 2024-02-27 05:10:25 2024-02-27 07:40:09 2024-02-27 08:03:03 0:22:54 0:14:15 0:08:39 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/classic task/test_cephadm_timeout} 1
pass 7574551 2024-02-27 05:10:26 2024-02-27 07:40:09 2024-02-27 08:13:30 0:33:21 0:18:48 0:14:33 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} 2
fail 7574552 2024-02-27 05:10:28 2024-02-27 07:43:00 2024-02-27 08:26:30 0:43:30 0:31:41 0:11:49 smithi main centos 9.stream orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} 2
Failure Reason:

"2024-02-27T08:20:00.000210+0000 mon.a (mon.0) 1841 : cluster 3 [WRN] OSD_DOWN: 1 osds down" in cluster log