Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi170.front.sepia.ceph.com smithi True True 2024-07-27 06:28:37.146910 scheduled_vshankar@teuthology centos 9.stream x86_64 /home/teuthworker/archive/vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi/7818896
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7820926 2024-07-26 23:48:07 2024-07-27 01:56:12 2024-07-27 05:44:22 3:48:10 3:36:50 0:11:20 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} 3
Failure Reason:

"2024-07-27T02:33:15.759844+0000 mds.i (mds.0) 44 : cluster [WRN] Scrub error on inode 0x1000000513b (/volumes/qa/sv_0/4d8d3b82-6ac2-4a6b-b539-26366f26f06f/client.0/tmp/t/linux-6.5.11/arch/powerpc) see mds.i log and `damage ls` output for details" in cluster log

pass 7820906 2024-07-26 23:47:45 2024-07-27 01:36:10 2024-07-27 01:55:14 0:19:04 0:09:19 0:09:45 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/alternate-pool} 2
pass 7820846 2024-07-26 23:46:40 2024-07-27 00:50:00 2024-07-27 01:36:03 0:46:03 0:36:09 0:09:54 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} 3
pass 7820643 2024-07-26 21:34:46 2024-07-27 00:18:28 2024-07-27 00:50:14 0:31:46 0:20:34 0:11:12 smithi main centos 9.stream rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful_host thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=2-crush} 4
pass 7820582 2024-07-26 21:33:35 2024-07-26 23:45:12 2024-07-27 00:18:17 0:33:05 0:21:51 0:11:14 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} 4
pass 7820484 2024-07-26 21:31:39 2024-07-26 22:45:35 2024-07-26 23:46:18 1:00:43 0:50:18 0:10:25 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} 4
pass 7820431 2024-07-26 21:30:34 2024-07-26 22:18:20 2024-07-26 22:45:12 0:26:52 0:12:33 0:14:19 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache} 4
pass 7819694 2024-07-26 16:08:54 2024-07-26 16:46:54 2024-07-26 17:33:54 0:47:00 0:32:56 0:14:04 smithi main ubuntu 22.04 fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/snapshot}} 2
pass 7819646 2024-07-26 16:05:32 2024-07-27 06:04:42 2024-07-27 06:29:02 0:24:20 0:13:33 0:10:47 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} 2
pass 7819609 2024-07-26 16:05:02 2024-07-27 05:44:09 2024-07-27 06:04:39 0:20:30 0:10:51 0:09:39 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7819343 2024-07-26 13:25:45 2024-07-26 22:20:44 10657 smithi main ubuntu 22.04 rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/compare-mirror-image-alternate-primary-nbd} 2
pass 7819300 2024-07-26 13:25:23 2024-07-26 19:12:30 1138 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
pass 7819263 2024-07-26 13:25:03 2024-07-26 18:18:09 2024-07-26 18:44:30 0:26:21 0:16:00 0:10:21 smithi main ubuntu 22.04 rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} 2
dead 7819232 2024-07-26 13:24:49 2024-07-26 18:00:19 2024-07-26 18:18:54 0:18:35 0:07:05 0:11:30 smithi main ubuntu 22.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest} 2
Failure Reason:

['E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6728 (apt-get)']

pass 7819191 2024-07-26 13:24:34 2024-07-26 17:34:03 2024-07-26 18:01:08 0:27:05 0:15:09 0:11:56 smithi main ubuntu 22.04 rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} 2
pass 7819150 2024-07-26 13:24:19 2024-07-26 16:00:42 2024-07-26 16:49:34 0:48:52 0:35:04 0:13:48 smithi main ubuntu 22.04 rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-journaling} 2
pass 7819130 2024-07-26 13:24:11 2024-07-26 15:24:10 2024-07-26 16:03:03 0:38:53 0:29:10 0:09:43 smithi main centos 9.stream rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_workunits} 2
pass 7819116 2024-07-26 13:24:06 2024-07-26 15:00:52 2024-07-26 15:23:12 0:22:20 0:11:09 0:11:11 smithi main ubuntu 22.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/workunits} 2
pass 7819089 2024-07-26 13:23:49 2024-07-26 14:34:39 2024-07-26 15:00:52 0:26:13 0:14:54 0:11:19 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} 2
pass 7819032 2024-07-26 13:23:11 2024-07-26 13:29:29 2024-07-26 14:34:32 1:05:03 0:55:49 0:09:14 smithi main centos 9.stream rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_api_tests_copy_on_read} 2