Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi039.front.sepia.ceph.com smithi True True 2024-03-24 12:09:29.384653 jenkins-build@teuthology centos 9 x86_64 Locked to capture FOG image for Jenkins build 798
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7619057 2024-03-24 04:56:50 2024-03-24 10:18:30 2024-03-24 11:05:48 0:47:18 0:34:21 0:12:57 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} 3
Failure Reason:

error during scrub thrashing: rank damage found: {'backtrace'}

pass 7618519 2024-03-23 15:49:02 2024-03-23 16:06:35 2024-03-23 16:56:22 0:49:47 0:34:12 0:15:35 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} 4
pass 7618457 2024-03-23 10:00:48 2024-03-23 15:20:07 2024-03-23 16:07:02 0:46:55 0:33:25 0:13:30 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} 3
pass 7618414 2024-03-23 10:00:04 2024-03-23 14:39:53 2024-03-23 15:22:53 0:43:00 0:31:31 0:11:29 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} 3
pass 7618330 2024-03-23 09:58:42 2024-03-23 13:32:57 2024-03-23 14:40:56 1:07:59 0:55:52 0:12:07 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/exports} 2
pass 7617978 2024-03-22 21:10:48 2024-03-24 11:45:09 2024-03-24 12:09:22 0:24:13 0:12:50 0:11:23 smithi main centos 9.stream orch/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7617918 2024-03-22 21:09:49 2024-03-24 11:14:01 2024-03-24 11:40:15 0:26:14 0:14:59 0:11:15 smithi main centos 9.stream orch/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-03-24T11:34:31.007605+0000 mon.smithi039 (mon.0) 263 : cluster 3 [WRN] CEPHADM_DAEMON_PLACE_FAIL: Failed to place 1 daemon(s)" in cluster log

pass 7617534 2024-03-22 13:19:29 2024-03-23 17:53:52 2024-03-23 23:04:13 5:10:21 4:57:39 0:12:42 smithi main ubuntu 20.04 rbd/encryption/{cache/none clusters/{fixed-3 openstack} conf/{disable-pool-app} features/defaults msgr-failures/few objectstore/bluestore-bitmap pool/ec-data-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests_luks1} 3
pass 7617490 2024-03-22 13:18:57 2024-03-23 17:17:27 2024-03-23 17:54:35 0:37:08 0:23:11 0:13:57 smithi main centos 8.stream rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} features/readbalance msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/qemu_fsstress} 3
pass 7617452 2024-03-22 13:18:42 2024-03-23 16:56:29 2024-03-23 17:18:21 0:21:52 0:09:54 0:11:58 smithi main ubuntu 20.04 rbd/singleton/{all/mon-command-help conf/{disable-pool-app} objectstore/bluestore-comp-snappy openstack supported-random-distro$/{ubuntu_latest}} 1
dead 7616450 2024-03-22 09:53:53 2024-03-24 00:39:32 2024-03-24 13:00:56 12:21:24 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} 3
Failure Reason:

hit max job timeout

pass 7616417 2024-03-22 09:53:08 2024-03-23 23:37:28 2024-03-24 00:51:23 1:13:55 1:00:34 0:13:21 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 4
pass 7616387 2024-03-22 09:52:26 2024-03-23 23:04:19 2024-03-23 23:38:26 0:34:07 0:22:43 0:11:24 smithi main centos 9.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} 2
fail 7616252 2024-03-22 09:42:50 2024-03-22 09:57:11 2024-03-22 12:13:22 2:16:11 2:04:55 0:11:16 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/snap-schedule} 2
Failure Reason:

"2024-03-22T10:17:40.036071+0000 mon.a (mon.0) 395 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7616138 2024-03-22 05:28:19 2024-03-22 05:32:56 2024-03-22 06:00:43 0:27:47 0:14:57 0:12:50 smithi main ubuntu 22.04 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_fsstress}} 3
fail 7616117 2024-03-22 03:04:28 2024-03-22 06:00:05 2024-03-22 06:24:46 0:24:41 0:11:20 0:13:21 smithi main centos 9.stream orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 3
Failure Reason:

"2024-03-22T06:19:45.144399+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7616049 2024-03-22 03:02:44 2024-03-22 04:52:53 2024-03-22 05:25:47 0:32:54 0:15:03 0:17:51 smithi main centos 9.stream orch:cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-03-22T05:16:39.009082+0000 mon.a (mon.0) 172 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b" in cluster log

fail 7616001 2024-03-22 02:09:05 2024-03-22 06:34:42 2024-03-22 09:53:56 3:19:14 3:04:55 0:14:19 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

"2024-03-22T07:07:17.985710+0000 mon.a (mon.0) 155 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log

pass 7615951 2024-03-22 01:10:54 2024-03-22 04:18:51 2024-03-22 04:53:34 0:34:43 0:22:51 0:11:52 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-full} 2
pass 7615907 2024-03-22 01:10:05 2024-03-22 03:28:43 2024-03-22 04:18:46 0:50:03 0:32:27 0:17:36 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3