Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi003.front.sepia.ceph.com smithi True True 2024-05-21 21:32:03.152068 scheduled_leonidus@teuthology centos 9 x86_64 /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719044
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
running 7719044 2024-05-21 21:14:40 2024-05-21 21:31:32 2024-05-21 21:55:35 0:25:39 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} 3
pass 7718795 2024-05-21 18:11:12 2024-05-21 18:32:22 2024-05-21 19:29:28 0:57:06 0:44:08 0:12:58 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} 3
fail 7718693 2024-05-21 14:53:37 2024-05-21 15:59:00 2024-05-21 16:18:54 0:19:54 0:09:42 0:10:12 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
Failure Reason:

Command failed on smithi003 with status 1: 'set -ex\ndd of=/root/logrotate.conf'

fail 7718660 2024-05-21 14:53:03 2024-05-21 15:36:45 2024-05-21 15:59:35 0:22:50 0:10:21 0:12:29 smithi main centos 9.stream orch:cephadm/no-agent-workunits/{0-distro/centos_9.stream_runc mon_election/connectivity task/test_orch_cli_mon} 5
Failure Reason:

Command failed on smithi003 with status 1: 'set -ex\ndd of=/root/logrotate.conf'

fail 7718576 2024-05-21 13:19:04 2024-05-21 15:01:33 2024-05-21 15:38:21 0:36:48 0:25:11 0:11:37 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} 3
Failure Reason:

error during scrub thrashing: rank damage found: {'backtrace'}

pass 7718509 2024-05-21 13:02:24 2024-05-21 13:19:27 2024-05-21 15:03:08 1:43:41 1:31:28 0:12:13 smithi install-fix centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} 3
pass 7717812 2024-05-21 01:15:21 2024-05-21 21:06:45 2024-05-21 21:31:57 0:25:12 0:12:01 0:13:11 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-agent-small} 4
pass 7717756 2024-05-21 01:14:17 2024-05-21 20:35:34 2024-05-21 21:07:53 0:32:19 0:21:24 0:10:55 smithi main ubuntu 22.04 rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 2
pass 7717695 2024-05-21 01:13:05 2024-05-21 19:50:33 2024-05-21 20:35:41 0:45:08 0:32:06 0:13:02 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} 4
pass 7717655 2024-05-21 01:12:19 2024-05-21 19:29:24 2024-05-21 19:53:29 0:24:05 0:13:59 0:10:06 smithi main ubuntu 22.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{ubuntu_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} tasks/{1-install 2-ceph 3-mgrmodules 4-units/insights}} 2
pass 7717578 2024-05-21 01:10:57 2024-05-21 18:02:28 2024-05-21 18:36:00 0:33:32 0:21:16 0:12:16 smithi main centos 9.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} 4
pass 7717554 2024-05-21 01:10:31 2024-05-21 17:43:29 2024-05-21 18:04:13 0:20:44 0:12:37 0:08:07 smithi main centos 9.stream rados/rest/{mgr-restful supported-random-distro$/{centos_latest}} 1
pass 7717528 2024-05-21 01:10:02 2024-05-21 17:24:26 2024-05-21 17:43:25 0:18:59 0:08:02 0:10:57 smithi main centos 9.stream rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/{1-install 2-ceph 3-mgrmodules 4-units/workunits}} 2
pass 7716922 2024-05-20 20:52:55 2024-05-21 17:01:39 2024-05-21 17:26:10 0:24:31 0:12:29 0:12:02 smithi main ubuntu 22.04 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} 2
fail 7716887 2024-05-20 20:52:17 2024-05-21 16:39:02 2024-05-21 17:02:12 0:23:10 0:11:17 0:11:53 smithi main ubuntu 22.04 rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi003 with status 5: 'sudo systemctl stop ceph-64f4a2c2-1793-11ef-bc9a-c7b262605968@mon.a'

pass 7716846 2024-05-20 20:51:32 2024-05-21 16:18:44 2024-05-21 16:40:14 0:21:30 0:11:41 0:09:49 smithi main centos 9.stream rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/readwrite} 2
pass 7716828 2024-05-20 20:51:13 2024-05-21 12:49:52 2024-05-21 13:19:41 0:29:49 0:20:07 0:09:42 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
fail 7716778 2024-05-20 20:50:20 2024-05-21 12:17:17 2024-05-21 12:49:47 0:32:30 0:21:27 0:11:03 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} 3
Failure Reason:

Command failed on smithi149 with status 1: 'CEPH_CLIENT_ID=2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --max-attr-len 20000 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0'

dead 7716771 2024-05-20 20:50:12 2024-05-21 12:11:34 2024-05-21 12:16:49 0:05:15 smithi main centos 9.stream rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
Failure Reason:

Error reimaging machines: Failed to power on smithi071

pass 7716740 2024-05-20 20:49:40 2024-05-21 11:44:52 2024-05-21 12:15:41 0:30:49 0:20:27 0:10:22 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} 4