Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi165.front.sepia.ceph.com smithi True True 2024-04-25 04:53:16.797911 scheduled_adking@teuthology x86_64 /home/teuthworker/archive/adking-2024-04-25_03:53:55-orch:cephadm-wip-adk-testing-2024-04-24-1625-distro-default-smithi/7672805
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
waiting 7672805 2024-04-25 03:55:12 2024-04-25 04:52:56 2024-04-25 04:53:17 0:02:46 0:02:46 smithi main ubuntu 22.04 orch:cephadm/thrash/{0-distro/ubuntu_22.04 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} 2
pass 7672771 2024-04-25 03:54:38 2024-04-25 04:30:01 2024-04-25 04:53:13 0:23:12 0:14:29 0:08:43 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
pass 7672751 2024-04-25 03:54:17 2024-04-25 04:11:04 2024-04-25 04:32:09 0:21:05 0:12:44 0:08:21 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
pass 7672452 2024-04-24 21:49:10 2024-04-25 03:07:49 2024-04-25 03:41:00 0:33:11 0:15:31 0:17:40 smithi main centos 8.stream krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 7672413 2024-04-24 21:48:31 2024-04-25 02:27:19 2024-04-25 03:09:05 0:41:46 0:22:08 0:19:38 smithi main centos 8.stream krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 7672348 2024-04-24 21:28:11 2024-04-25 01:38:31 2024-04-25 02:27:34 0:49:03 0:40:41 0:08:22 smithi main centos 9.stream fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} 2
fail 7672124 2024-04-24 20:45:17 2024-04-24 21:57:39 2024-04-25 01:27:43 3:30:04 3:18:56 0:11:08 smithi main centos 9.stream rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_latest} tasks/test_multi} 2
Failure Reason:

rgw multisite test failures

fail 7671826 2024-04-24 15:50:26 2024-04-24 16:52:33 2024-04-24 17:11:40 0:19:07 0:12:09 0:06:58 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} 3
Failure Reason:

"2024-04-24T17:07:45.782213+0000 mon.a (mon.0) 468 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) ['daemon osd.1 on smithi114 is in unknown state'] (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7671806 2024-04-24 15:50:05 2024-04-24 16:36:44 2024-04-24 16:50:00 0:13:16 0:06:18 0:06:58 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} 2
Failure Reason:

Command failed on smithi165 with status 125: "sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5904d29475f5be602879d9fb26280e89b808d5cc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a3c7838-025a-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi114:172.21.15.114=a;smithi114:[v2:172.21.15.114:3301,v1:172.21.15.114:6790]=c;smithi165:172.21.15.165=b'"

pass 7671659 2024-04-24 14:13:13 2024-04-24 17:55:32 2024-04-24 19:52:46 1:57:14 1:46:30 0:10:44 smithi main centos 8.stream rbd/maintenance/{base/install clusters/{fixed-3 openstack} conf/{disable-pool-app} objectstore/bluestore-comp-zlib qemu/xfstests supported-random-distro$/{centos_8} workloads/dynamic_features_no_cache} 3
pass 7671609 2024-04-24 14:12:18 2024-04-24 17:23:27 2024-04-24 17:56:46 0:33:19 0:20:55 0:12:24 smithi main ubuntu 20.04 rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} conf/{disable-pool-app} features/journaling msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_bonnie} 3
pass 7671437 2024-04-24 13:00:25 2024-04-24 13:13:32 2024-04-24 14:09:31 0:55:59 0:45:04 0:10:55 smithi main ubuntu 22.04 rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} 2
fail 7671321 2024-04-24 11:41:52 2024-04-24 12:51:00 2024-04-24 13:06:36 0:15:36 0:08:35 0:07:01 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream mon_election/classic task/test_orch_cli} 1
Failure Reason:

Command failed on smithi165 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:23fcfb96e7e1a49d12a94e3f87a8e3f06db2a1ec ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0a8bd1e2-023b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4'

pass 7671238 2024-04-24 09:12:27 2024-04-24 09:21:31 2024-04-24 10:05:01 0:43:30 0:24:56 0:18:34 smithi main ubuntu 22.04 rados:thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} 4
fail 7671160 2024-04-24 05:22:53 2024-04-24 05:36:29 2024-04-24 07:08:23 1:31:54 1:14:12 0:17:42 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} 3
Failure Reason:

"2024-04-24T06:30:14.155843+0000 mds.i (mds.2) 54 : cluster [WRN] Scrub error on inode 0x3000000e419 (/volumes/qa/sv_1/cb877da6-ac73-47a5-b966-1a86a4cbbb26/client.0/tmp/payload.1/multiple_rsync_payload.193225/firmware/nvidia/tu106/gr) see mds.i log and `damage ls` output for details" in cluster log

pass 7671092 2024-04-24 01:17:51 2024-04-24 19:52:45 2024-04-24 22:02:18 2:09:33 2:00:17 0:09:16 smithi main centos 8.stream upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
pass 7671069 2024-04-24 01:17:27 2024-04-24 14:09:02 2024-04-24 16:37:04 2:28:02 2:20:04 0:07:58 smithi main rhel 8.6 upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
pass 7671019 2024-04-24 01:16:35 2024-04-24 10:00:41 2024-04-24 12:51:19 2:50:38 2:35:52 0:14:46 smithi main ubuntu 20.04 upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
pass 7670991 2024-04-24 01:10:46 2024-04-24 07:17:46 2024-04-24 09:29:04 2:11:18 2:02:29 0:08:49 smithi main centos 8.stream upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
pass 7670925 2024-04-24 01:09:36 2024-04-24 03:10:17 2024-04-24 05:45:37 2:35:20 2:25:18 0:10:02 smithi main centos 8.stream upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2