Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi076.front.sepia.ceph.com smithi True True 2024-05-21 16:22:07.085806 scheduled_yuriw@teuthology ubuntu 22.04 x86_64 /home/teuthworker/archive/yuriw-2024-05-20_20:45:46-rados-wip-yuri3-testing-2024-05-20-1050-distro-default-smithi/7716851
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7718709 2024-05-21 14:53:54 2024-05-21 16:04:47 2024-05-21 16:22:06 0:17:19 0:06:33 0:10:46 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream tasks/deploy_smb_mgr_basic} 2
Failure Reason:

Command failed on smithi076 with status 1: 'set -ex\ndd of=/root/logrotate.conf'

fail 7718675 2024-05-21 14:53:19 2024-05-21 15:46:32 2024-05-21 16:04:51 0:18:19 0:07:28 0:10:51 smithi main centos 9.stream orch:cephadm/smb/{0-distro/centos_9.stream_runc tasks/deploy_smb_mgr_res_dom} 2
Failure Reason:

Command failed on smithi076 with status 1: 'set -ex\ndd of=/root/logrotate.conf'

fail 7718653 2024-05-21 14:52:55 2024-05-21 15:31:12 2024-05-21 15:47:31 0:16:19 0:06:33 0:09:46 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} 2
Failure Reason:

Command failed on smithi076 with status 1: 'set -ex\ndd of=/root/logrotate.conf'

pass 7718564 2024-05-21 13:18:51 2024-05-21 14:52:37 2024-05-21 15:32:14 0:39:37 0:30:32 0:09:05 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
fail 7718522 2024-05-21 13:02:27 2024-05-21 13:27:14 2024-05-21 14:43:18 1:16:04 1:03:32 0:12:32 smithi install-fix centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} 3
Failure Reason:

"2024-05-21T14:14:03.832409+0000 mds.b (mds.0) 144 : cluster [WRN] Scrub error on inode 0x10000003397 (/volumes/qa/sv_0/5d6c08a4-abcf-43fb-a798-3083025c35e7/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details" in cluster log

pass 7718484 2024-05-21 13:02:18 2024-05-21 13:03:54 2024-05-21 13:27:42 0:23:48 0:12:46 0:11:02 smithi install-fix centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} 2
fail 7717977 2024-05-21 07:59:02 2024-05-21 08:03:41 2024-05-21 08:33:44 0:30:03 0:18:14 0:11:49 smithi main centos 9.stream fs:mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/mirror} 1
Failure Reason:

Test failure: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring)

fail 7717954 2024-05-21 05:56:03 2024-05-21 06:03:03 2024-05-21 06:35:34 0:32:31 0:21:28 0:11:03 smithi main ubuntu 22.04 fs:mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} 1
Failure Reason:

Test failure: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring)

fail 7717904 2024-05-21 04:40:20 2024-05-21 04:42:47 2024-05-21 05:26:25 0:43:38 0:29:58 0:13:40 smithi install-fix ubuntu 22.04 rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} 4
Failure Reason:

Command failed on smithi076 with status 11: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3'

pass 7717868 2024-05-21 01:33:20 2024-05-21 03:22:31 2024-05-21 04:12:26 0:49:55 0:33:21 0:16:34 smithi main centos 9.stream fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/pjd}} 3
fail 7717488 2024-05-20 23:58:19 2024-05-21 01:38:42 2024-05-21 03:23:38 1:44:56 1:32:54 0:12:02 smithi install-fix centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} 3
Failure Reason:

"2024-05-21T02:35:25.717410+0000 mds.b (mds.0) 94 : cluster [WRN] Scrub error on inode 0x1000000e0c3 (/volumes/qa/sv_1/71b47411-eb4c-4dcc-a226-ba9543a1bf7d/client.0/tmp/t/linux-6.5.11/drivers/tty/vt) see mds.b log and `damage ls` output for details" in cluster log

pass 7717463 2024-05-20 23:58:12 2024-05-21 00:52:07 2024-05-21 01:40:17 0:48:10 0:33:55 0:14:15 smithi install-fix centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} 3
pass 7717448 2024-05-20 23:58:09 2024-05-21 00:29:48 2024-05-21 00:54:36 0:24:48 0:11:56 0:12:52 smithi install-fix centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} 2
pass 7717412 2024-05-20 23:07:57 2024-05-20 23:45:00 2024-05-21 00:31:09 0:46:09 0:34:13 0:11:56 smithi install-fix centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iozone}} 3
running 7716851 2024-05-20 20:51:38 2024-05-21 16:21:06 2024-05-21 16:42:09 0:21:08 smithi main ubuntu 22.04 rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} 2
dead 7716836 2024-05-20 20:51:22 2024-05-21 12:54:45 2024-05-21 12:56:09 0:01:24 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} 4
Failure Reason:

Error reimaging machines: Failed to power on smithi188

pass 7716775 2024-05-20 20:50:16 2024-05-21 12:16:06 2024-05-21 12:55:01 0:38:55 0:26:39 0:12:16 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-snaps} 4
dead 7716767 2024-05-20 20:50:08 2024-05-21 12:08:02 2024-05-21 12:09:16 0:01:14 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} 4
Failure Reason:

Error reimaging machines: Failed to power on smithi076

pass 7716725 2024-05-20 20:49:25 2024-05-21 11:29:54 2024-05-21 12:08:09 0:38:15 0:26:50 0:11:25 smithi main centos 9.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 4
pass 7716680 2024-05-20 20:48:51 2024-05-20 21:42:49 2024-05-20 22:07:07 0:24:18 0:13:43 0:10:35 smithi main centos 9.stream orch:cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_extra_daemon_features} 2