Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7384934 2023-08-30 23:48:19 2023-08-30 23:49:22 2023-08-30 23:58:43 0:09:21 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} 2
pass 7384935 2023-08-30 23:48:20 2023-08-30 23:49:22 2023-08-31 00:29:37 0:40:15 0:29:36 0:10:39 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7384936 2023-08-30 23:48:21 2023-08-30 23:49:22 2023-08-31 00:43:13 0:53:51 0:41:48 0:12:03 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7384937 2023-08-30 23:48:22 2023-08-30 23:49:23 2023-08-31 00:22:23 0:33:00 0:22:17 0:10:43 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} 2
pass 7384938 2023-08-30 23:48:22 2023-08-30 23:49:23 2023-08-31 00:55:41 1:06:18 0:54:53 0:11:25 smithi main rhel 8.6 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/lockdep} 2
fail 7384939 2023-08-30 23:48:23 2023-08-30 23:49:24 2023-08-30 23:57:36 0:08:12 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} 2
Failure Reason:

Host key for server 'smithi002.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIE5d9GeTemCrNuZN7QKYJ2XQ9oC8sVZOJH07pyOGOnFc', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCzYGIO1LR9yTfUtThx1uMKV+xxj6hYp75i0fIM0czDuNcrG888BIYu0pUf5rSmT+GEiPayF4e0xNYqEZzwSvSW4DnN4X4DztONfVmfkvyI7r7TlRLObvCy6us9g5EMLUn8jrCl4migA4DbZcrTyoELqTrS1RwLov6y2GrpDX9fJwCGmqQP8I51N0nr7hQebqsNEv7aYFVrHiK0+GMTuQ7F3ki7b00mFSXC/7DqQT/AGqOCe8h3neTHfqSEG5KKKIxMrXGR8CmHv1bw9ez1mnxL3kp4UYoB0Mi7m1TW4FkcTTes4kIpmg0FMPpuYFFA8KP2NaVuVfZcDvT/ixpSvVWdrb4HIFQO0WBlYhe2IWgwaNCEP10fJ1dFwTYBzFAteEjP1RQ46PTGDd1ZP1hV3ZRQOVbOfozDhCHxTkbdOKQLRi6vfawXKJLrMxFyJU6f4UvkrIoJk3MZfvpoaWbqaeTlGpFkTQr/hfGnm8+Ig3AS/LjAiu0dQOc1PJW9XilstjM='

pass 7384940 2023-08-30 23:48:24 2023-08-30 23:49:24 2023-08-31 00:18:21 0:28:57 0:15:45 0:13:12 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-workload/new_ops/{0-clients/kclient 1-client-sanity}}} 2
dead 7384941 2023-08-30 23:48:24 2023-08-30 23:49:24 2023-08-31 00:10:51 0:21:27 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} 3
Failure Reason:

Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds

pass 7384942 2023-08-30 23:48:25 2023-08-30 23:49:25 2023-08-31 00:19:49 0:30:24 0:22:00 0:08:24 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} 2
fail 7384943 2023-08-30 23:48:26 2023-08-30 23:49:25 2023-08-31 00:14:17 0:24:52 0:16:57 0:07:55 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} 2
Failure Reason:

Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

fail 7384944 2023-08-30 23:48:26 2023-08-30 23:49:25 2023-08-31 00:13:39 0:24:14 0:11:25 0:12:49 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}} 2
Failure Reason:

Command failed on smithi033 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7384945 2023-08-30 23:48:27 2023-08-30 23:49:26 2023-08-31 00:36:30 0:47:04 0:37:54 0:09:10 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} 3
fail 7384946 2023-08-30 23:48:28 2023-08-30 23:49:26 2023-08-30 23:59:09 0:09:43 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} 2
pass 7384947 2023-08-30 23:48:29 2023-08-30 23:49:27 2023-08-31 00:51:19 1:01:52 0:46:21 0:15:31 smithi main centos 8.stream fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
fail 7384948 2023-08-30 23:48:29 2023-08-30 23:49:27 2023-08-31 00:48:48 0:59:21 0:46:14 0:13:07 smithi main ubuntu 22.04 fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} 2
Failure Reason:

"2023-08-31T00:13:08.527147+0000 mon.a (mon.0) 156 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7384949 2023-08-30 23:48:30 2023-08-30 23:49:27 2023-08-31 00:11:31 0:22:04 0:11:19 0:10:45 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Command failed on smithi104 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7384950 2023-08-30 23:48:31 2023-08-30 23:49:28 2023-08-31 00:21:39 0:32:11 0:21:32 0:10:39 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} 2
fail 7384951 2023-08-30 23:48:32 2023-08-30 23:49:28 2023-08-30 23:59:16 0:09:48 smithi main centos 9.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
pass 7384952 2023-08-30 23:48:32 2023-08-30 23:49:39 2023-08-31 00:20:08 0:30:29 0:16:13 0:14:16 smithi main ubuntu 20.04 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
fail 7384953 2023-08-30 23:48:33 2023-08-30 23:49:39 2023-08-31 00:41:10 0:51:31 0:38:52 0:12:39 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

fail 7384954 2023-08-30 23:48:34 2023-08-30 23:49:39 2023-08-31 00:01:53 0:12:14 smithi main ubuntu 22.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} 2
Failure Reason:

Host key for server 'smithi094.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIKJkAzKlK+QMEXW33459ooFDJ/pMwY+7Byzn76Y0p9PZ', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCYIVpXBTj4XrS+WKvF1XmMxXLXYDDNu7lU/mF9shIxjusrVj1bpyZ6TbsaJQ+eajD0SiS1/nTuEGejWEwb+NZpZeTaO762fhJ9ZWQUhB/8XlRyUZ2Yf2aN6IpCx4haOa03BKEAY09KBnO28gm5e29c5aTzvfVSSgqjobVP+cP+t/x5Wk/jEwjhN2fppTqwQXx6s6DWFoBZpH7r77r8XJ1kiPVpt4odsOWHdMitbkprKazBJ0/Kemd2KapIJZMn6gA990MMIaBtZj8AV5PhrNCsRD2JKsqkjw9R7x34DH1wN/S5g6PLitOA6CybBTDhm5yIYFRnCtInsltmFRKrRCpTxoS5ynJD0XpmI1OEcY3jUqEDAeJjxXrU+yBBJ0YOD1je1wJOos2Ppi8EFn1+LmT0AiLBxJbwlvAenxxXwAgxce0N+34/zC10QMMxBNDl3DWSDTITxgBpisZ00ZTTAB6vPmwBKQgTWDofsK7fMI++7fBYVHNxBYmJdgSVBJ5ViWM='

fail 7384955 2023-08-30 23:48:34 2023-08-30 23:49:40 2023-08-31 00:01:13 0:11:33 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} 2
Failure Reason:

Host key for server 'smithi121.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIF6SWMm4oef0RfWxxJJbVsvAuzxJAXZNhT0FFO3+hLa9', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCjhSuzZ8gw+ScIO5NaHyeRW/93X+LsAuedAIzuf6B0zjl0Batd6MT6U0nP5cBvjeKvg2pcAEtxjXDOeTT+W5O9IMRh92fma5dTB41n8aZGLzKhgYfBkXgAkemNeBfwD7CRJlFVGcz+V+/w83QX+egARk36vsGwHY78xGeRmk0MyIINybKq6M70ctWOTsPZGfPFR6ux12BkF7i9CMwMRWrsuvQT1DiqIaD/jXIgtOwRspGJ62ty1I8qsbfzusohpTr0QAAf+TRMX2Qv/1HUBuEq1tKL8LqMF2petoXRX+OpzxLecBkAwzzPsn7FyIpo+bqT119y5S5oG2rqy2P8aP12OtSfgBdgn8Q3py25yushzJAQLZewwltrSiaRjzKrcoqtM1C42Pa4nf8eLYU1/jIrkZAcPSHtsi0E1NgO78oynqiWkIP+gwHXv2as5RXLhbn3OuVMtBnRiBh0aZbhmaiyNputnbpoNbB2fP6ZEo6VLtf76gklJhIVrStAR2dYEMk='

fail 7384956 2023-08-30 23:48:35 2023-08-30 23:49:40 2023-08-31 00:00:56 0:11:16 smithi main centos 9.stream fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/{0-client 1-tests/fscrypt-common}} 3
fail 7384957 2023-08-30 23:48:36 2023-08-30 23:49:41 2023-08-31 00:00:58 0:11:17 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 5
Failure Reason:

Host key for server 'smithi153.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIIuu4PFgNSFdcKtHBu5jSsGoKcQxJCL+750RMRu3BOfY', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQC/OQcvgk2WKVTJfEnFaseAjl+S+9CsktOaM3Ppy4SCvTV9XLfA5GK71I6CfyESCL8uFMqcllE627NFj/t8Lunv6fWZuwD/l+5FQ2FAWcf8CvSRZwS9QdeLn8PylNGld22MyL6w566Z7sic4+BCrFCELzA7Hxe6EDW7wHhZgYGno/PZER/BQpGhxoaSnsl79UjkgRtJjT/1slN/7mcKydPPSUnlGhxAsyH7+OwQ7ZrMxFtwGSZuXw3SVF2h4foybBQAIvkJBmE3Yfu1OpLN8fUzzNtsWwPxq4NySwMv26K4jO+EhWLGvKf59CQl1fXs1p9LzRV49PU3atGGTnf4TUbjVerZeHgwn5LBSJTCHzAkdlZEjyPKbQUAZnS/6vxu71Fd3J2hPzOpqDKJBvITWp6/9KyCVnK62WdL7K8sXX/VBEidTlU2Tt4IMIsv2jKUy9RanIhRo3XCqf0mHB1L+9xE9AcChscEhDuhXxeorjRA6qRuqKlJLOmlxZ6ggIQHbec='

dead 7384958 2023-08-30 23:48:37 2023-08-30 23:49:41 2023-08-31 12:02:17 12:12:36 smithi main rhel 8.6 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
Failure Reason:

hit max job timeout

pass 7384959 2023-08-30 23:48:37 2023-08-30 23:49:41 2023-08-31 00:43:46 0:54:05 0:45:20 0:08:45 smithi main rhel 8.6 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
pass 7384960 2023-08-30 23:48:38 2023-08-30 23:49:42 2023-08-31 02:16:24 2:26:42 2:18:16 0:08:26 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} 2
pass 7384961 2023-08-30 23:48:39 2023-08-30 23:49:42 2023-08-31 00:38:18 0:48:36 0:38:10 0:10:26 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} 3
pass 7384962 2023-08-30 23:48:40 2023-08-30 23:49:42 2023-08-31 00:16:46 0:27:04 0:19:25 0:07:39 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} 2
fail 7384963 2023-08-30 23:48:40 2023-08-30 23:49:43 2023-08-31 00:01:28 0:11:45 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} 2
fail 7384964 2023-08-30 23:48:41 2023-08-30 23:49:43 2023-08-31 00:14:45 0:25:02 0:12:10 0:12:52 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} 3
Failure Reason:

Command failed on smithi045 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7384965 2023-08-30 23:48:42 2023-08-30 23:49:43 2023-08-31 01:58:42 2:08:59 1:57:44 0:11:15 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} 2
pass 7384966 2023-08-30 23:48:42 2023-08-30 23:49:44 2023-08-31 00:46:36 0:56:52 0:43:24 0:13:28 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} 3
pass 7384967 2023-08-30 23:48:43 2023-08-30 23:49:44 2023-08-31 00:25:50 0:36:06 0:26:41 0:09:25 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
pass 7384968 2023-08-30 23:48:44 2023-08-30 23:49:45 2023-08-31 00:19:34 0:29:49 0:15:02 0:14:47 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} 2
pass 7384969 2023-08-30 23:48:45 2023-08-30 23:49:45 2023-08-31 00:47:46 0:58:01 0:46:40 0:11:21 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iogen}} 3
pass 7384970 2023-08-30 23:48:45 2023-08-30 23:49:45 2023-08-31 00:38:50 0:49:05 0:37:56 0:11:09 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} 2
pass 7384971 2023-08-30 23:48:46 2023-08-30 23:49:46 2023-08-31 00:32:14 0:42:28 0:29:33 0:12:55 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
dead 7384972 2023-08-30 23:48:47 2023-08-30 23:49:46 2023-08-31 12:00:03 12:10:17 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} 2
Failure Reason:

hit max job timeout

pass 7384973 2023-08-30 23:48:47 2023-08-30 23:49:47 2023-08-31 00:39:38 0:49:51 0:34:58 0:14:53 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} 3
fail 7384974 2023-08-30 23:48:48 2023-08-30 23:49:47 2023-08-30 23:59:19 0:09:32 smithi main centos 9.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7384975 2023-08-30 23:48:49 2023-08-30 23:49:47 2023-08-30 23:58:34 0:08:47 smithi main centos 9.stream fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} 2
pass 7384976 2023-08-30 23:48:49 2023-08-30 23:49:48 2023-08-31 00:17:09 0:27:21 0:19:19 0:08:02 smithi main rhel 8.6 fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
pass 7384977 2023-08-30 23:48:50 2023-08-30 23:49:48 2023-08-31 00:29:05 0:39:17 0:25:32 0:13:45 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}} 2
pass 7384978 2023-08-30 23:48:51 2023-08-30 23:49:48 2023-08-31 00:24:00 0:34:12 0:22:07 0:12:05 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
fail 7384979 2023-08-30 23:48:52 2023-08-30 23:49:49 2023-08-31 00:02:20 0:12:31 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} 2
Failure Reason:

Host key for server 'smithi148.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIK+zBEeNJJ/SdNzH0z6Ubyuq9monyCFznuWWuGKPRHMr', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7n5apf2frFYl3ex8FR4whyPjd+FsKSfTi6hkmO8UtwWau+0dW6G/TjFM5x+xV65P+yGCYUL1u32G+YZP/7tsMg3xGsAqmH+kFYY89SKqtr6b8UtV68shLIfqmishY/6HxwoMud+wI6Xjj+E51kC/VLdwQ3pGEaMCccC5Mw/qDjuIe6J81zVT1i77SZQm8YH1ZE9yeliEbQCllsBw18XbY8rRBcGK8fuqMuzkFyJxMupud0KAX/YDjJJ/CURRtYB9ZGzQaCqlZEPNhSS7iBPvh9qIbsunWJxQVCGFXrdqLZ36XlVIOWRyVJPYXgDp3ix/IB1AsFV4DMFMhzYv1TBmP/i4OFdgc2xeb62V96w8UrMT8R5jXIO7zBo0KFe+i2MrhwwtCCCAZ8NftJSiYrhJdl+HJj6UbBkDcjfYDjvTY5Ygo33i757EmAqkk+Tx62eXOQnIueg+wC30fIXILTgMpiXt7UavfcQzLBGUmp5O80bsWlKw+nQl532ecVcNsPE='

fail 7384980 2023-08-30 23:48:52 2023-08-30 23:49:49 2023-08-31 00:30:29 0:40:40 0:26:57 0:13:43 smithi main ubuntu 20.04 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind} 2
Failure Reason:

saw valgrind issues

pass 7384981 2023-08-30 23:48:53 2023-08-30 23:49:49 2023-08-31 00:38:15 0:48:26 0:38:21 0:10:05 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/pjd}} 3
fail 7384982 2023-08-30 23:48:54 2023-08-30 23:49:50 2023-08-31 00:15:16 0:25:26 0:12:03 0:13:23 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Command failed on smithi005 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7384983 2023-08-30 23:48:55 2023-08-30 23:49:50 2023-08-31 00:33:45 0:43:55 0:30:23 0:13:32 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} 2
pass 7384984 2023-08-30 23:48:55 2023-08-30 23:49:51 2023-08-31 00:26:59 0:37:08 0:21:14 0:15:54 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} 2
pass 7384985 2023-08-30 23:48:56 2023-08-30 23:51:21 2023-08-31 01:00:37 1:09:16 0:52:46 0:16:30 smithi main ubuntu 20.04 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
pass 7384986 2023-08-30 23:48:57 2023-08-30 23:56:22 2023-08-31 00:41:24 0:45:02 0:32:49 0:12:13 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/direct_io}} 3
pass 7384987 2023-08-30 23:48:58 2023-08-30 23:57:43 2023-08-31 00:43:41 0:45:58 0:38:21 0:07:37 smithi main rhel 8.6 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
fail 7384988 2023-08-30 23:48:58 2023-08-30 23:58:43 2023-08-31 00:28:34 0:29:51 0:19:48 0:10:03 smithi main ubuntu 22.04 fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} 2
Failure Reason:

"2023-08-31T00:20:07.255600+0000 mon.a (mon.0) 163 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7384989 2023-08-30 23:48:59 2023-08-30 23:58:54 2023-08-31 00:25:51 0:26:57 0:20:05 0:06:52 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} 2
pass 7384990 2023-08-30 23:49:00 2023-08-30 23:59:14 2023-08-31 00:24:39 0:25:25 0:15:46 0:09:39 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} 2
pass 7384991 2023-08-30 23:49:00 2023-08-30 23:59:24 2023-08-31 00:44:33 0:45:09 0:38:11 0:06:58 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} 3
fail 7384992 2023-08-30 23:49:01 2023-08-30 23:59:25 2023-08-31 00:07:56 0:08:31 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} 2
fail 7384993 2023-08-30 23:49:02 2023-08-31 00:01:05 2023-08-31 00:08:59 0:07:54 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 4
Failure Reason:

Host key for server 'smithi069.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIASF7lukUicJnm+DevQs9USkddNnJEfL78gom8OJIac8', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQC0i5iQmPuzivqVSbzzC4iOwNEpDXlz4yaF0PK78wEcEfT0t3zn6s2iCetClxes0MW1j0bJGUa7vIEbjxmsnVikZzjo8VwtPuFfV/Fl8tAu0p2Mz94IZ1GrLXcx9rMh1awaMJ03UpMFNZQEFSQ1zZg+FoaUew8LetXbEf/EhdKtOr6FbiN/OoZF18BM6DWliti8ua6UC/2d/gNjRnmpwN9tmANbOUz374ig6ssnmQ7r7lqKQkzCF2Fn5ut3xapXTHW2yy+QTY9dQ4K4V5gcGFpsWtt0ID7ThtXmTAkX1QxeXFrO6tzSCJ66QnZEUtpE3ZoQlDkbQ6yIOv0/1U1BiyeGdaNYO//z0NlGU+B49vSZoNKhOqhLQBhCJ94se9tdeJojIGhCr+pb/5mnqHlqax6FvW/xR+jZrQ5tt7gyvgYBqQuoZ9FrdZg01NiDDGO6xs2HuiynKHqPypccJGzwcSZIk1YIDdpvSjyx7c/irxWBc+phzWbeRHS0tih4aB7gdd8='

fail 7384994 2023-08-30 23:49:02 2023-08-31 00:01:06 2023-08-31 00:08:26 0:07:20 smithi main centos 9.stream fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
fail 7384995 2023-08-30 23:49:03 2023-08-31 00:01:07 2023-08-31 00:08:46 0:07:39 smithi main ubuntu 22.04 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

Host key for server 'smithi121.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAICdiAK6JZXk7ZoolVCqvuOo1YEpWgngZugJilPLv+aOq', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ4BS24r4adczxXAToz5dTNJbf0gAHu4V+I0KqZkClUAA8pTJzLD6MbF/+WiqNfUEL+rj654CWgfMIYCpKT2Eyo7q2HZ8mTbDSUnMOBvj8g37vpn6Y7xn18u/EjjRCj/jC6fEHrGelljaucHjrjBbDZrORaSSSRmcQH7Hf27vJdYBHacJYVduLpU0qA+WHGN5uw2ewriv8Fs2m21WUkG7l+IqfKaDKQCXulZfWW+59kSqNoZNwndjAwyURS71Dzainzggli06IG66wzQTiMfzsCrsDWtzlAEzBqEX/gLW5Zclg/avkLcrmk3wtW9ts379BOndpmcI6RHYvsFpO4FZG7CQgLuoDH6Ip6MRXsDOuS8faRYS2rE77BVPQ9j4RCvB6tp/X1Dzo5YEprQXiGcpjRhJlQf9uc1iVsn3dKde6edKBeKTe45uDjFv8Xb96omcp8DkdAWTJaV0zz61bGDtc9nmRYfgLmBIDoEBwk4PyWX2hNX0KdkqNKVofsgJZuo8='

pass 7384996 2023-08-30 23:49:04 2023-08-31 00:01:17 2023-08-31 00:27:23 0:26:06 0:17:55 0:08:11 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} 2
fail 7384997 2023-08-30 23:49:05 2023-08-31 00:01:38 2023-08-31 00:57:32 0:55:54 0:45:11 0:10:43 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/kernel_untar_build}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

dead 7384998 2023-08-30 23:49:05 2023-08-31 00:02:28 2023-08-31 12:24:36 12:22:08 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} 2
Failure Reason:

hit max job timeout

fail 7384999 2023-08-30 23:49:06 2023-08-31 00:09:20 2023-08-31 00:28:31 0:19:11 0:10:10 0:09:01 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Command failed on smithi121 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385000 2023-08-30 23:49:07 2023-08-31 00:09:20 2023-08-31 00:34:50 0:25:30 0:18:04 0:07:26 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} 2
fail 7385001 2023-08-30 23:49:07 2023-08-31 00:09:21 2023-08-31 00:16:01 0:06:40 smithi main centos 9.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
fail 7385002 2023-08-30 23:49:08 2023-08-31 00:09:21 2023-08-31 00:18:09 0:08:48 smithi main centos 9.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
dead 7385003 2023-08-30 23:49:09 2023-08-31 00:11:02 2023-08-31 00:24:08 0:13:06 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} 2
Failure Reason:

Error reimaging machines: Timeout opening channel.

pass 7385004 2023-08-30 23:49:09 2023-08-31 00:11:12 2023-08-31 01:07:28 0:56:16 0:47:44 0:08:32 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} 3
pass 7385005 2023-08-30 23:49:10 2023-08-31 00:14:23 2023-08-31 00:47:55 0:33:32 0:26:02 0:07:30 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} 2
pass 7385006 2023-08-30 23:49:11 2023-08-31 00:14:54 2023-08-31 00:37:39 0:22:45 0:17:22 0:05:23 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} 2
fail 7385007 2023-08-30 23:49:11 2023-08-31 00:14:54 2023-08-31 01:06:23 0:51:29 0:41:57 0:09:32 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385008 2023-08-30 23:49:12 2023-08-31 00:15:25 2023-08-31 00:52:50 0:37:25 0:27:54 0:09:31 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7385009 2023-08-30 23:49:13 2023-08-31 00:16:05 2023-08-31 00:49:56 0:33:51 0:23:46 0:10:05 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} 2
fail 7385010 2023-08-30 23:49:13 2023-08-31 00:16:56 2023-08-31 00:24:49 0:07:53 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} 2
Failure Reason:

Host key for server 'smithi181.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIPpbU+EjGATvx+Y3nImfBEVKCMT2lyZb/14aPp7qbYK+', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDYlJkL5VoypxsnXkuv8Y44wmcOHPxKhVQ1h24PXEWpO/1jNTkS2YUVW8ZCfydTWx4Jny9/gCnBfykwbTnP8fVmyKgm83pBGnQAgFTr4X0fwh2Zgd4LDgrMPa8Hi4QstYmz7Z0xwPRVQSGtMCNuvTdSaUP+e5co6HiUli3u7rjZwNOp5K7byohoStozJc+PB0zkotnsExQWJSlnQzG8sVGzXDazX73gH3Rc3mVE4CyQ96DX9alShZnO9mBOpX1cv7XRNciVEIHesxD4ydrg/tB5Bvk6qI76sbw2slhxM30fUt33UGkJrftrIvFNB1mzTb0v8o9FO6cKslQkZFL1T9P14U7jS0+66MafOtuVcJGOy6Cw2blED9lvzQEbGx3iklQfAms5wngl+85roOm0dGKO9Q5DdAzLnQYJpFo6p6gp8XKOW/jKejeSHfB1DBx1YinvKroJXk+3WEJtMGoku7m7NDyZ/Hdj09oryJ14/NgSQXqfWh+tL97CX+D8q7N0Mpk='

pass 7385011 2023-08-30 23:49:14 2023-08-31 00:17:16 2023-08-31 00:39:36 0:22:20 0:15:46 0:06:34 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} 2
pass 7385012 2023-08-30 23:49:15 2023-08-31 00:18:16 2023-08-31 01:19:07 1:00:51 0:53:25 0:07:26 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} 3
fail 7385013 2023-08-30 23:49:15 2023-08-31 00:19:37 2023-08-31 00:26:58 0:07:21 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} 2
pass 7385014 2023-08-30 23:49:16 2023-08-31 00:19:57 2023-08-31 00:52:06 0:32:09 0:21:23 0:10:46 smithi main ubuntu 22.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} 2
fail 7385015 2023-08-30 23:49:17 2023-08-31 00:20:18 2023-08-31 00:44:43 0:24:25 0:11:06 0:13:19 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} 3
Failure Reason:

Command failed on smithi003 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

fail 7385016 2023-08-30 23:49:18 2023-08-31 00:22:29 2023-08-31 01:25:43 1:03:14 0:50:24 0:12:50 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385017 2023-08-30 23:49:18 2023-08-31 00:24:09 2023-08-31 01:22:53 0:58:44 0:47:21 0:11:23 smithi main ubuntu 20.04 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/lockdep} 2
pass 7385018 2023-08-30 23:49:19 2023-08-31 00:24:30 2023-08-31 00:52:07 0:27:37 0:20:11 0:07:26 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree} 2
pass 7385019 2023-08-30 23:49:20 2023-08-31 00:24:40 2023-08-31 02:41:52 2:17:12 2:07:58 0:09:14 smithi main centos 8.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} 2
pass 7385020 2023-08-30 23:49:20 2023-08-31 00:24:50 2023-08-31 01:09:20 0:44:30 0:36:39 0:07:51 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} 3
pass 7385021 2023-08-30 23:49:21 2023-08-31 00:26:01 2023-08-31 00:58:10 0:32:09 0:20:42 0:11:27 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair} 2
fail 7385022 2023-08-30 23:49:22 2023-08-31 00:34:28 smithi main ubuntu 22.04 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
Failure Reason:

Host key for server 'smithi123.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIKz1jiQc6EZCnbHUQl8B6VTii4v8RDfnk7Ja/FAR10e7', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQC957S+uFxlyiC0sV/yd9sP5Cv1AI6q21PFtE+jE6AhWR/+vueG8HwvUzMESReFnTSc3eIQkxcc6ouYvE0t4mqOCOeX1C3hl8pNdsvaVFbSQTasPDJyuPRba6GGtMLnpNfln3K2/wkS5UiPKwKggo2is/5CZOj42x6yS1ysyyoMiZJBulgeVQa5mu97ffjcen1IeOyhFDC5URmJ8PD+QGE7zqYX5dS7dxsXt6d2vU1Y7yXZAbSdQR+ZcJJ03S+yxMkmYBdmoW+sNxnqW7JvrJVf+Ets2K4LyLmgnM0ot/GMDVe7t73Sx5OydKn/oNYOHCTR9M0x88Ulm+krB5PpjWNf/Lo8l9lpGcpJtxjxvPRQEk1s9qE/ex3CzXjOjcIJOhAikBdujECTFvVanBxr9IBWszycevzAKPjitMjLzcy+cwqdq5XPLx3fLcvHbL/whxtkpdvEYdX9vqZFuEoTU1ZJZMMe/k7Fl7HiM5U1bEiKNMXajTpti6OAQlBEcSxXJLk='

fail 7385023 2023-08-30 23:49:22 2023-08-31 00:27:02 2023-08-31 00:34:05 0:07:03 smithi main centos 9.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} 2
pass 7385024 2023-08-30 23:49:23 2023-08-31 00:27:02 2023-08-31 01:09:51 0:42:49 0:30:23 0:12:26 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
dead 7385025 2023-08-30 23:49:24 2023-08-31 00:27:33 2023-08-31 12:37:47 12:10:14 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
Failure Reason:

hit max job timeout

fail 7385026 2023-08-30 23:49:25 2023-08-31 00:28:43 2023-08-31 00:36:07 0:07:24 smithi main centos 9.stream fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} 1
pass 7385027 2023-08-30 23:49:25 2023-08-31 00:29:14 2023-08-31 01:25:27 0:56:13 0:46:12 0:10:01 smithi main centos 8.stream fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} 2
fail 7385028 2023-08-30 23:49:26 2023-08-31 00:59:27 1254 smithi main ubuntu 22.04 fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} 1
Failure Reason:

"2023-08-31T00:48:50.320440+0000 mon.a (mon.0) 73 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7385029 2023-08-30 23:49:27 2023-08-31 00:54:33 869 smithi main ubuntu 22.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} 2
Failure Reason:

"2023-08-31T00:51:45.083926+0000 mon.a (mon.0) 248 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7385030 2023-08-30 23:49:28 2023-08-31 00:30:35 2023-08-31 02:46:38 2:16:03 2:03:23 0:12:40 smithi main ubuntu 20.04 fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_20.04} tasks/mirror} 1
pass 7385031 2023-08-30 23:49:29 2023-08-31 01:01:23 1425 smithi main rhel 8.6 fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} 1
fail 7385032 2023-08-30 23:49:29 2023-08-31 00:32:16 2023-08-31 00:42:46 0:10:30 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 5
Failure Reason:

Host key for server 'smithi124.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAICQwoxzJ/ytLx/a809C6R75cqQo/ViL+ug3mYn+2HNA7', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCipxLkc3L/EOnzd2mg1FTdq6EuXIPtqYIv0MdryJauzyw3hNqEnBsfUtSI+skka/OkLXVpV8cyxMlmGxNVu76rB2vg3GwxUBVZ7fBlolgrlFCvVYJR8f0Tg2omFD5sfj4WG9buYsLwFRFnCacBuLviOLlAm5az7quK2Pl+H6q1UzJHB6Ov19S9jwv3oQrfztMgah59cxKMrCW71d5GyWoOFmAtK+OnQpl775SLLsYVRe8PJRy3JXF3NakeVJchGrOuLDI7Aa5aD3ietoCZr1L99cKM/qaOjvWyg7mO1v+AKfpt4xbtn2zM4obXg5a/fXx5hrS1RksePBtP/rvbjrIE73c4HIOCHrEiLKqiSlQ110HMQ5XpUzxMUjdj54LGwIx7RX7mOeTkKZ4lKtMI582qtouuAlyWYUhNr+RRjcq7UGJ+yDW8KBCNLW0frVepDfI9LQf5fJV30QB2/s/ah4CDW0A194avZEy7QJKzIU4+10rDjx7NY0ANCIn+gV8qfgU='

fail 7385033 2023-08-30 23:49:30 2023-08-31 00:34:37 2023-08-31 00:42:22 0:07:45 smithi main centos 9.stream fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
fail 7385034 2023-08-30 23:49:31 2023-08-31 00:34:57 2023-08-31 00:43:09 0:08:12 smithi main centos 9.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
pass 7385035 2023-08-30 23:49:31 2023-08-31 00:36:18 2023-08-31 01:32:34 0:56:16 0:44:54 0:11:22 smithi main ubuntu 20.04 fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} 2
fail 7385036 2023-08-30 23:49:32 2023-08-31 00:36:38 2023-08-31 02:11:38 1:35:00 1:22:52 0:12:08 smithi main ubuntu 22.04 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
Failure Reason:

"2023-08-31T00:58:13.062821+0000 mon.a (mon.0) 191 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7385037 2023-08-30 23:49:33 2023-08-31 00:37:49 2023-08-31 01:03:23 0:25:34 0:15:08 0:10:26 smithi main ubuntu 22.04 fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/fstop} 1
Failure Reason:

"2023-08-31T00:57:02.051256+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7385038 2023-08-30 23:49:33 2023-08-31 00:37:49 2023-08-31 02:05:08 1:27:19 1:17:27 0:09:52 smithi main centos 9.stream fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} 1
Failure Reason:

Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring)

pass 7385039 2023-08-30 23:49:34 2023-08-31 00:38:19 2023-08-31 01:49:32 1:11:13 1:00:29 0:10:44 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} 2
pass 7385040 2023-08-30 23:49:35 2023-08-31 00:38:20 2023-08-31 01:23:43 0:45:23 0:33:41 0:11:42 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} 3
pass 7385041 2023-08-30 23:49:35 2023-08-31 00:38:20 2023-08-31 01:01:58 0:23:38 0:13:29 0:10:09 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace} 2
fail 7385042 2023-08-30 23:49:36 2023-08-31 00:38:51 2023-08-31 00:58:59 0:20:08 0:10:18 0:09:50 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Command failed on smithi116 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385043 2023-08-30 23:49:37 2023-08-31 00:39:41 2023-08-31 01:23:14 0:43:33 0:36:35 0:06:58 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} 2
pass 7385044 2023-08-30 23:49:37 2023-08-31 00:39:42 2023-08-31 01:04:08 0:24:26 0:15:59 0:08:27 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush} 2
pass 7385045 2023-08-30 23:49:38 2023-08-31 00:41:12 2023-08-31 01:03:55 0:22:43 0:13:50 0:08:53 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_8 conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} 1
fail 7385046 2023-08-30 23:49:39 2023-08-31 01:25:32 2241 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385047 2023-08-30 23:49:40 2023-08-31 00:41:33 2023-08-31 01:16:20 0:34:47 0:22:54 0:11:53 smithi main ubuntu 22.04 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} 2
pass 7385048 2023-08-30 23:49:40 2023-08-31 00:42:24 2023-08-31 01:08:45 0:26:21 0:15:34 0:10:47 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}} 2
fail 7385049 2023-08-30 23:49:41 2023-08-31 00:42:54 2023-08-31 00:49:47 0:06:53 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} 2
fail 7385050 2023-08-30 23:49:42 2023-08-31 00:42:54 2023-08-31 00:50:03 0:07:09 smithi main centos 9.stream fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/{0-client 1-tests/fscrypt-iozone}} 3
pass 7385051 2023-08-30 23:49:43 2023-08-31 00:43:15 2023-08-31 01:24:21 0:41:06 0:30:27 0:10:39 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} 2
pass 7385052 2023-08-30 23:49:43 2023-08-31 00:43:15 2023-08-31 01:26:56 0:43:41 0:32:51 0:10:50 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} 3
pass 7385053 2023-08-30 23:49:44 2023-08-31 01:07:44 782 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead} 2
pass 7385054 2023-08-30 23:49:45 2023-08-31 01:19:46 1494 smithi main centos 8.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} 2
pass 7385055 2023-08-30 23:49:46 2023-08-31 00:44:37 2023-08-31 01:31:12 0:46:35 0:37:59 0:08:36 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} 3
pass 7385056 2023-08-30 23:49:46 2023-08-31 00:44:47 2023-08-31 01:45:49 1:01:02 0:52:31 0:08:31 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} 2
fail 7385057 2023-08-30 23:49:47 2023-08-31 00:53:58 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} 2
pass 7385058 2023-08-30 23:49:48 2023-08-31 00:46:38 2023-08-31 01:43:22 0:56:44 0:44:54 0:11:50 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} 3
pass 7385059 2023-08-30 23:49:49 2023-08-31 00:47:49 2023-08-31 01:13:01 0:25:12 0:19:02 0:06:10 smithi main rhel 8.6 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
pass 7385060 2023-08-30 23:49:49 2023-08-31 00:47:59 2023-08-31 01:14:28 0:26:29 0:15:43 0:10:46 smithi main centos 8.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7385061 2023-08-30 23:49:50 2023-08-31 01:09:06 610 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Command failed on smithi070 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385062 2023-08-30 23:49:51 2023-08-31 00:50:00 2023-08-31 01:33:17 0:43:17 0:32:50 0:10:27 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} 2
pass 7385063 2023-08-30 23:49:52 2023-08-31 00:50:11 2023-08-31 01:17:11 0:27:00 0:21:00 0:06:00 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} 2
pass 7385064 2023-08-30 23:49:52 2023-08-31 00:50:11 2023-08-31 01:35:31 0:45:20 0:37:30 0:07:50 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} 3
fail 7385065 2023-08-30 23:49:53 2023-08-31 00:52:12 2023-08-31 00:59:28 0:07:16 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} 2
fail 7385066 2023-08-30 23:49:54 2023-08-31 00:52:12 2023-08-31 01:29:11 0:36:59 0:25:34 0:11:25 smithi main ubuntu 20.04 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/valgrind} 2
Failure Reason:

saw valgrind issues

fail 7385067 2023-08-30 23:49:55 2023-08-31 00:52:53 2023-08-31 01:02:33 0:09:40 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} 4
Failure Reason:

Host key for server 'smithi131.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAICWsze6ilXExd0VoNPVX3x4znPgI/kFNLD1lZOYmr6hv', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDLgSbYoD4Kf+KwKfH0xAGesDLR+qV51xYrTobliO/PLrxCDOY4KfEO/X1ykxguwpTEu2rGYGrJzv9ksGJJyuzaV85jgdvGsLPezYn3ccW2pCgBwlzD+BGSgq3N6luWpeVUi9GksIILD1YoJ350SQVy/BkCZToPEzKbdMzmkD9Yf0DG1NhjdT3li2H5qfHrKSXkeBjw9FQu8gGRmzotA0cAVZHIJpQvhztX+InaLT05MmrjajhGnw7BvwUQu/qN1+Tajr/gSAUbnDFu9SIw2APzLTBQKtiTvfRA9qEb30IVGpJOTCAV48zmkf+bed5kbIvI9qjN/HGAU9EbFk0oDiHtJWdfdC7kmoYsFeZ0hHaq5Y5BUZWfp0Io3ujlc03O7RXctOGpyyqdCrLX9ED5083luKrUo8A7/TVcv8ntufJorVTsalai73hB6kE48z2ufD3YTgMzjTMtPMKCnmL3CrkzgFHlMoXfK6YCgugiQYz1Zrtvwi9xTfd0CN9dLNCNfpM='

dead 7385068 2023-08-30 23:49:55 2023-08-31 00:54:43 2023-08-31 13:04:43 12:10:00 smithi main rhel 8.6 fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
Failure Reason:

hit max job timeout

pass 7385069 2023-08-30 23:49:56 2023-08-31 00:55:44 2023-08-31 01:58:14 1:02:30 0:54:58 0:07:32 smithi main rhel 8.6 fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
fail 7385070 2023-08-30 23:49:57 2023-08-31 00:57:34 2023-08-31 01:16:58 0:19:24 0:09:37 0:09:47 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}} 2
Failure Reason:

Command failed on smithi078 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385071 2023-08-30 23:49:57 2023-08-31 00:57:35 2023-08-31 01:23:43 0:26:08 0:18:50 0:07:18 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} 2
pass 7385072 2023-08-30 23:49:58 2023-08-31 00:58:15 2023-08-31 01:45:00 0:46:45 0:35:20 0:11:25 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} 3
fail 7385073 2023-08-30 23:49:59 2023-08-31 00:59:06 2023-08-31 01:54:53 0:55:47 0:47:59 0:07:48 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} 2
Failure Reason:

Test failure: test_export_pin_many (tasks.cephfs.test_exports.TestExportPin)

pass 7385074 2023-08-30 23:49:59 2023-08-31 00:59:36 2023-08-31 01:56:19 0:56:43 0:48:21 0:08:22 smithi main rhel 8.6 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} 2
fail 7385075 2023-08-30 23:50:00 2023-08-31 01:00:47 2023-08-31 02:00:32 0:59:45 0:49:18 0:10:27 smithi main centos 9.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} 2
Failure Reason:

"2023-08-31T01:19:08.890905+0000 mon.a (mon.0) 218 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7385076 2023-08-30 23:50:01 2023-08-31 01:01:27 2023-08-31 01:39:20 0:37:53 0:28:25 0:09:28 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7385077 2023-08-30 23:50:02 2023-08-31 01:02:08 2023-08-31 01:31:48 0:29:40 0:18:38 0:11:02 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} 2
pass 7385078 2023-08-30 23:50:02 2023-08-31 01:02:39 2023-08-31 01:35:49 0:33:10 0:21:00 0:12:10 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} 2
pass 7385079 2023-08-30 23:50:03 2023-08-31 01:02:39 2023-08-31 01:46:53 0:44:14 0:37:43 0:06:31 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} 2
pass 7385080 2023-08-30 23:50:04 2023-08-31 01:04:00 2023-08-31 01:37:28 0:33:28 0:26:58 0:06:30 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} 3
fail 7385081 2023-08-30 23:50:05 2023-08-31 01:04:10 2023-08-31 01:27:23 0:23:13 0:11:12 0:12:01 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} 3
Failure Reason:

Command failed on smithi005 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

fail 7385082 2023-08-30 23:50:05 2023-08-31 01:06:31 2023-08-31 01:15:09 0:08:38 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} 2
Failure Reason:

Host key for server 'smithi064.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIJ+GkRneitmHvRBBaNUkmk4TGEkpkLhfrdvnYXIIQG0e', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDrw4akEp+66hkb1y+JPhBcqaTX+/8huLzLlNjpHW7ZP4hUG4OwWzh/AeRBWtRZ0r+Cpozqry0LV4hfJvST0bdJYRH6duA1wAvhOXe829o+NphmUFfs/NwTo+FxxWB65TV7GwWNWkIg/QnNMQ/wJ20BWO58Nk0iN7Ab+NPIoF19whZ8MQ/uxF17sXlEguEYl5o0JQoOUB0YiEZmL+9fz4L/dBVWC955bvMK7ENx3PQYBmUz7GPruk990br+r0GFA9cYYhRtgDChN8GPBuJrB9RehDcYDJu6OEZhbdKZTzOrK1Gu7KGEmTh0HW/Nvjo68LqEU3zLqFBA8VZssv7MMWq/3PTZMxXCOvT/hYvFyw5uunA+2AhGIoJFmay0GST6JibSO9RguZq5ZEp5pWg+2L5lao/4im67CMmxdeE0hxoTDKEl1saNEtdIjzEn8QugQmjdoao7haXtlXlgZ5BMYblL9qfIcjaA7FejJ7DUr65nRsW6yyIExbuuGGIrDHrXYdk='

fail 7385083 2023-08-30 23:50:06 2023-08-31 01:07:31 2023-08-31 01:14:31 0:07:00 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
pass 7385084 2023-08-30 23:50:07 2023-08-31 01:07:32 2023-08-31 02:18:05 1:10:33 1:00:08 0:10:25 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} 3
pass 7385085 2023-08-30 23:50:07 2023-08-31 01:08:52 2023-08-31 01:38:35 0:29:43 0:22:15 0:07:28 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} 2
fail 7385086 2023-08-30 23:50:08 2023-08-31 01:09:23 2023-08-31 01:16:56 0:07:33 smithi main centos 9.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} 2
pass 7385087 2023-08-30 23:50:09 2023-08-31 01:09:23 2023-08-31 01:34:42 0:25:19 0:13:22 0:11:57 smithi main ubuntu 20.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_20.04} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} 2
fail 7385088 2023-08-30 23:50:09 2023-08-31 01:09:54 2023-08-31 01:20:02 0:10:08 smithi main centos 9.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} 2
pass 7385089 2023-08-30 23:50:10 2023-08-31 01:13:05 2023-08-31 01:59:45 0:46:40 0:35:32 0:11:08 smithi main centos 9.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} 2
fail 7385090 2023-08-30 23:50:11 2023-08-31 01:14:35 2023-08-31 02:13:20 0:58:45 0:50:31 0:08:14 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/kernel_untar_build}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385091 2023-08-30 23:50:11 2023-08-31 01:15:16 2023-08-31 01:42:13 0:26:57 0:15:33 0:11:24 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} 2
fail 7385092 2023-08-30 23:50:12 2023-08-31 01:16:16 2023-08-31 01:38:42 0:22:26 0:10:18 0:12:08 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Command failed on smithi118 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

fail 7385093 2023-08-30 23:50:13 2023-08-31 01:17:07 2023-08-31 01:24:33 0:07:26 smithi main ubuntu 22.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} 2
Failure Reason:

Host key for server 'smithi078.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIEbldFqksch4m/nJ+VvDEy94zzTnWrzeigFDpOFhTS/s', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDmOt5lib5xxZC33IELM7gZ2MUp4CUzu05Gfg/ARg67h2NRCttOKuyeJe3I5Xc70htPxYJabhbI01nKsWIhSvUeJVSd0Wd49VFTtYtbNQ1u880FZRMwOOBKwWtkZ05dGXOCW+/6M9XEpicPwt5sVHhuJ7uvtPyrPotAs9vsJM671nqhU7O6KS259oOBkwygy8xMgs8iEv3Spxx4+q9tD/XHgBg0+dqi9S2HBH9zNKy9X5Z9mw+H+TPIZgnsoLZtwMljAYfbfvC1MyPCHj2w7NcOFsOXbkbHvy/zT/KeK9NRGn2bydsQ6bjBghYdB+/OJhfdeSSpky+JvJs239LQqjY/FvbC4qZpMOmMVTjdI4PqjO6X/f+na6ZlwbRpFa5Hzs4Hivjpo22GnzRN2ryR+RWLbkWPqnqjIJAQvtJ3tf/OApxQP1tkClfmAknEamrkRWl9FLXF4YIofJOXQEDSZJpoi6FcJdiXCvMYTt/NdOl4LbpO4xnBQZUP3fVqFH8Zn6M='

pass 7385094 2023-08-30 23:50:13 2023-08-31 01:17:07 2023-08-31 02:38:56 1:21:49 1:11:18 0:10:31 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} 3
fail 7385095 2023-08-30 23:50:14 2023-08-31 01:17:17 2023-08-31 01:27:09 0:09:52 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} 2
Failure Reason:

Host key for server 'smithi172.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIHLlidWdPi5QgdsFATE74d/shZehg+hdoUG4rihZy08V', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCiAQk41KOYt9sGcBqwKtgSA4Qtxp8MH/pF3U70NwpeirdRaw2QYdv5q+yAMaHjfzt9v6pw0baAJLlt3M0djV2SVq/Em0I9ArT3KU9s+a4qzh5XRCeQI7OGY3+DaOavmKJojLLUsRoGKLNFjjDn1odX9vITKTqgShnYdpPTsrZSkesPM48r2I7D05EgplUNBK/Viygu2YZV5kEQ6+aMlag4RayLdHVedketqpg3UZ9+CE4fV3zskXAJ8zot56u8b1CfvW0ChrlwZQpPQvHOdZle2v/tOLlzfNhySSyCLJVAHxdAsRL+Zi81h4MN2libtX7ekntBlS18IC1noQ4z5EI4AAAUVOSvAx6CQScfIOEzw0UpaiYcXcE1h1amghRYsJkVys9fHX90g6wWuUCk8g3AmL3OxfIjKKYJDZWc7rVgrcti/0tn/ssWyUItqNPQ1aX71C3uQSbqw+e2Gc6Z32vNyVlOgpPKGjiijEx+X6Bf0cOPIptidv4OoDc4o04Tc+k='

fail 7385096 2023-08-30 23:50:15 2023-08-31 01:19:08 2023-08-31 01:58:53 0:39:45 0:32:56 0:06:49 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385097 2023-08-30 23:50:16 2023-08-31 01:19:49 2023-08-31 01:47:14 0:27:25 0:20:16 0:07:09 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} 2
dead 7385098 2023-08-30 23:50:16 2023-08-31 01:20:09 2023-08-31 01:36:04 0:15:55 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} 2
Failure Reason:

Error reimaging machines: Timeout opening channel.

pass 7385099 2023-08-30 23:50:17 2023-08-31 01:23:45 2023-08-31 02:47:07 1:23:22 1:13:45 0:09:37 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}} 2
fail 7385100 2023-08-30 23:50:18 2023-08-31 01:23:46 2023-08-31 01:34:22 0:10:36 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} 5
Failure Reason:

Host key for server 'smithi002.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIF70PRBFXSogByc6g8LIQEVZ9VGffIinkemR5ScpYdGE', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDoVnoicqZDmQDXmFQ4VVb3kye6NP65R2SU27n5QuPrDoNHCq/FgwR+vm+/4lohaKSGAqWeMja3HQt886ppQdaUrbS3iSFcxDFOqZEoJPzzgzJ7RVOICOxMcuyOCrvnWSd8UYH89DWAN/0K3xD9hX0GdmMeaTsGyn8xBgEEmYBY+XtQhagtDFWkfk6FPCt8vFisi4FwahoKDKJ0nq57mrudys/0yc2sZgC+ub/TBnQr7hmIEx8zxGLfHIJ44n++sF4N2rr0wtw1vgbf0xY1QLiAlRW2s4VkZKWmKuN2D914v2QIx95M8QCm4/8hYDkXcpo27gMVSXukBu0Ql85THidgL02puGUMdPj9twtcve05511Pl6xYDN+h9ojhTiEwJxWOu9cNEog245lqV2wJLht4s4hVuhVueq1F6YEJyIVeCyEhpWy+3O5L6cICrNcKN1LKxJYA4bEBSSqirZK9bBLZTsLO0AtWnZmZZzeKCX9JR/4QJVnuVjT4/CtBFPF+fKc='

fail 7385101 2023-08-30 23:50:18 2023-08-31 01:25:36 2023-08-31 01:32:42 0:07:06 smithi main centos 9.stream fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} 2
pass 7385102 2023-08-30 23:50:19 2023-08-31 01:25:37 2023-08-31 02:22:57 0:57:20 0:47:13 0:10:07 smithi main centos 8.stream fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
pass 7385103 2023-08-30 23:50:20 2023-08-31 01:25:37 2023-08-31 03:27:04 2:01:27 1:55:22 0:06:05 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} 2
fail 7385104 2023-08-30 23:50:21 2023-08-31 01:25:48 2023-08-31 02:20:09 0:54:21 0:42:40 0:11:41 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} 3
Failure Reason:

Command failed (workunit test suites/dbench.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=833faa571491534dcb6bc6b25a578499081113d8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'

pass 7385105 2023-08-30 23:50:21 2023-08-31 01:28:46 2023-08-31 02:10:30 0:41:44 0:35:14 0:06:30 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} 2
fail 7385106 2023-08-30 23:50:22 2023-08-31 01:29:16 2023-08-31 01:38:54 0:09:38 smithi main ubuntu 22.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} 2
Failure Reason:

Host key for server 'smithi003.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIBPiiLO0LwsG0MZOXENLlgI0jmvccA9z+8FynLskog0q', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCz2+hvT8859eZl3KEqmxAOUKE7cq9jo6GQ39htbd4whTGEPPVHrcsLAN77SW0PK/rLj1oXHLBRdRk90/HsqzwEoLwDNSe+U6jAUhdzYO4uZEijjhCdIDEpbMFeSC5vZCUrD9p55Rm8lwVdOCpJ4QWhe2D1vNhJRiQI7S+oWqTiMkw2ZwXz8XSjZ2zmAZnGqMlX3l7QRAwI+5dxaRo23dLWgQOC1SOIMQLe32RscGe8guLvVnMmUs/Jxo6K8Lfupc9s5+ohAV67SfT1uqm7mPC2h5g1WoqXxt5Q2k+ACCJarZ87fCEZsJeTY2pgXfkz32jahqYN0dE4bqFE469wQVWIZenf1qhAUJ1Ws/GXilSYDQOCP3lIVMtdTCTzjPb9lH2cU5exRMVp0njJqh6VOgEdq3YECmZ/xCx61coQB9yHZUs4vCPcmAyNWqQfnf1xlYDHhWFo5tEy5fdNsxo72uhGNDcgbzCuXo0DPCu03IXIgN4N5RZS0/Szw/Csf9BQCvE='

fail 7385107 2023-08-30 23:50:23 2023-08-31 01:31:17 2023-08-31 01:54:12 0:22:55 0:10:30 0:12:25 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

Command failed on smithi066 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385108 2023-08-30 23:50:23 2023-08-31 01:31:57 2023-08-31 02:29:39 0:57:42 0:51:07 0:06:35 smithi main rhel 8.6 fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/lockdep} 2
fail 7385109 2023-08-30 23:50:24 2023-08-31 01:32:38 2023-08-31 01:39:40 0:07:02 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} 2
fail 7385110 2023-08-30 23:50:25 2023-08-31 01:32:48 2023-08-31 02:23:53 0:51:05 0:42:01 0:09:04 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} 3
Failure Reason:

The following counters failed to be set on mds daemons: {'mds.dir_split'}

pass 7385111 2023-08-30 23:50:25 2023-08-31 01:34:29 2023-08-31 02:11:39 0:37:10 0:26:27 0:10:43 smithi main centos 8.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} 2
pass 7385112 2023-08-30 23:50:26 2023-08-31 01:34:29 2023-08-31 02:23:53 0:49:24 0:38:23 0:11:01 smithi main rhel 8.6 fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} 2
pass 7385113 2023-08-30 23:50:27 2023-08-31 01:34:30 2023-08-31 02:25:10 0:50:40 0:39:55 0:10:45 smithi main centos 8.stream fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} 2
pass 7385114 2023-08-30 23:50:27 2023-08-31 01:34:50 2023-08-31 02:01:43 0:26:53 0:15:20 0:11:33 smithi main ubuntu 20.04 fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7385115 2023-08-30 23:50:28 2023-08-31 01:35:40 2023-08-31 01:42:58 0:07:18 smithi main centos 9.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
dead 7385116 2023-08-30 23:50:29 2023-08-31 01:35:51 2023-08-31 01:49:18 0:13:27 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} 2
Failure Reason:

Error reimaging machines: Timeout opening channel.

fail 7385117 2023-08-30 23:50:29 2023-08-31 01:36:21 2023-08-31 02:26:46 0:50:25 0:39:39 0:10:46 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} 3
Failure Reason:

error during scrub thrashing: rank damage found: {'backtrace'}

pass 7385118 2023-08-30 23:50:30 2023-08-31 01:37:32 2023-08-31 02:17:51 0:40:19 0:29:42 0:10:37 smithi main centos 8.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
pass 7385119 2023-08-30 23:50:31 2023-08-31 01:38:42 2023-08-31 03:00:16 1:21:34 1:11:14 0:10:20 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} 2
fail 7385120 2023-08-30 23:50:32 2023-08-31 01:38:43 2023-08-31 01:46:09 0:07:26 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} 2
pass 7385121 2023-08-30 23:50:32 2023-08-31 01:38:43 2023-08-31 02:12:42 0:33:59 0:26:00 0:07:59 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsstress}} 3
fail 7385122 2023-08-30 23:50:33 2023-08-31 01:39:24 2023-08-31 01:47:21 0:07:57 smithi main ubuntu 22.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} 2
Failure Reason:

Host key for server 'smithi191.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAII52dCSWGYFMy+fd2hVXyYpyCpAHCVUvMB+SCuzwWjkn', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQC/pPZ2NcX36hG965J7/r8bgRjr7LQ2XL10j/nnw91VIOnaW3/qxXKwBVgpUn6eNBDXlwd8YFYSccVODKHRwlyNbeaTr94vI04Y3erIrfyY0jHTbMIVxYyUaRB0fG0Ws0fEb8/8DjEx8y7OL2pJ+1ALb7qDTLglbDJUXmqW1/H6vMVn4GjEm6H9kvlBe+wZ83UWcV31I1YAV6RnKOBOBdBejSsdI49oQv/n1b3JqttiF0beZROmQ5eDF1imzVfRKmjnmaN9D14nciyentotB15v9mddvbKmCVXyXpVCl28fIt8K7izF4pD6bEdExLaB/Qg5vEwucRRpiIwOLZpRM/lF27mWMa1GUjHJXOUiYwPai9q5uaLdjQdK9ZpDG8ZJDj6C5SpI/L5jk80l8fwfjPWiLPp+nwF5fqFZPF1g8DBILycwUrJRc+gMc3O+MIPNCd74bNFAhkVLtpFHh5ddGaYAtMEUfBsI7oc3xBnTq8IEYLXoQHs1f4f4gfDYfAcyvy8='

fail 7385123 2023-08-30 23:50:34 2023-08-31 01:39:44 2023-08-31 02:03:23 0:23:39 0:14:47 0:08:52 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} 2
Failure Reason:

Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

fail 7385124 2023-08-30 23:50:34 2023-08-31 01:42:15 2023-08-31 02:04:35 0:22:20 0:11:21 0:10:59 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} 3
Failure Reason:

Command failed on smithi129 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

fail 7385125 2023-08-30 23:50:35 2023-08-31 01:43:06 2023-08-31 02:36:50 0:53:44 0:41:48 0:11:56 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} 3
Failure Reason:

error during scrub thrashing: rank damage found: {'backtrace'}

pass 7385126 2023-08-30 23:50:36 2023-08-31 01:43:26 2023-08-31 02:18:25 0:34:59 0:23:28 0:11:31 smithi main centos 8.stream fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} 2
pass 7385127 2023-08-30 23:50:36 2023-08-31 01:45:07 2023-08-31 02:06:55 0:21:48 0:14:47 0:07:01 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} 2
fail 7385128 2023-08-30 23:50:37 2023-08-31 01:45:47 2023-08-31 02:06:56 0:21:09 0:11:06 0:10:03 smithi main centos 8.stream fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}} 2
Failure Reason:

Command failed on smithi008 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

fail 7385129 2023-08-30 23:50:38 2023-08-31 01:45:57 2023-08-31 01:54:30 0:08:33 smithi main ubuntu 22.04 fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/{0-client 1-tests/fscrypt-dbench}} 3
Failure Reason:

Host key for server 'smithi089.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIEWjaAoxq79IWD2eGOaZxC55GsjjfwtOE7g4VvRs92A6', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQDEtxsA7dwdUVkebzX5PJ1n3mZ1oomXiRpOR72ClUMzAOqhKFgn2HtmS0oUOnFxsS4u0e0oZN6S8u68jskkfMhtf14r0l5J0EQOghcNDf+ODr/ZCm2wrPluKuCjBnFC7R+fu0706nztM6PNP5ckZ+DOYWj9TWc7+rGzW9zZ3yP8/KnTyqjtBL2nM8W+L/ndckWjLT+VmWVmyyOPsvELpNcRL61nVdpRWrWuH3ty1tjTL4WsKVjtCGWIvvJha9i5Arr7UoHe1OGAQwJ5tRNicL8tTFiyyG52bP8Uf8Q79/q/AKH8qOx2ncRqr3u1M9xguko6du9eeh0pyvAm6EBVSyvaN8j9bzhBI0MLABoutZz9cqdY8cFnk+0sWUJWFPpayyOtGDoVTV6B4lIR0XAK1c71wmFu2io9jhVRx7XsNcM3yyveCGSWNbX2BexgNPGWQtnQ6WaLsjhj+Hez35H48mYe+z17FTDccxMR+tAA0Y7Kzhc0qfi0hbj6n8aoRIEtJFM='

pass 7385130 2023-08-30 23:50:38 2023-08-31 01:46:18 2023-08-31 03:07:26 1:21:08 1:10:50 0:10:18 smithi main rhel 8.6 fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} 2
pass 7385131 2023-08-30 23:50:39 2023-08-31 01:46:18 2023-08-31 02:20:22 0:34:04 0:27:03 0:07:01 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} 3
pass 7385132 2023-08-30 23:50:40 2023-08-31 01:47:19 2023-08-31 02:10:06 0:22:47 0:16:49 0:05:58 smithi main rhel 8.6 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} 2
fail 7385133 2023-08-30 23:50:40 2023-08-31 01:47:29 2023-08-31 01:55:32 0:08:03 smithi main centos 9.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} 2
dead 7385134 2023-08-30 23:50:41 2023-08-31 01:48:40 2023-08-31 02:02:37 0:13:57 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} 3
Failure Reason:

Error reimaging machines: Timeout opening channel.

pass 7385135 2023-08-30 23:50:42 2023-08-31 01:49:40 2023-08-31 02:16:49 0:27:09 0:17:55 0:09:14 smithi main centos 8.stream fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} 2
pass 7385136 2023-08-30 23:50:43 2023-08-31 01:49:41 2023-08-31 02:15:14 0:25:33 0:15:25 0:10:08 smithi main centos 8.stream fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
pass 7385137 2023-08-30 23:50:43 2023-08-31 01:49:51 2023-08-31 02:19:51 0:30:00 0:19:54 0:10:06 smithi main centos 8.stream fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health} 2
fail 7385138 2023-08-30 23:50:44 2023-08-31 01:50:52 2023-08-31 02:00:47 0:09:55 smithi main ubuntu 22.04 fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{test}} 2
Failure Reason:

Host key for server 'smithi043.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGpioX7OQ5G1oLL00ynzHQpJGnNZKwYQeNe2aDrS84hn', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCtptwwulJ/CYe7w8lKqoHanvgs8I2qxZoS8/3IQq8n4w1ohGp4bMS1LZZ67FsuOFVszSGnXNbMJJbUCAfD0Jsrz+yWX49OBtDpjrnHc3AKvpD6GKcRo9rhG1DALBrEAv4M/nhSec1C+/rrZdguq6IAQ/6CWEwAY3w2Ktz5Cc9CUdSkBuu+VGXxceYO/PakviKigSv3W7LsUAOHcOu0NZpNMuGMsmMttHnnB4Jp4rXj6jvZQFovSalEz5p/iMMa15HO8ocfS0LJ1d0giUdX5XEmWRJGwapOvkc2sEd3FM1/MkhgcmloXZoEnlUbMTJegf6tDNzq1NQNuQv5iCN5XhMwmEa+/6++8hhc8WkCkjdZ+hxN6ULirM7oi9yShQBrVA4oGSfiO6Fuw5YUtPgTNFNbwj2IAxVZJXrv66mxUbKxM0b7BSlxCYoH6PB3aFuBtKBYC4N6iSzA2cj5k1L+UPMHi6um5yixP8Kn025jWSvI5o+BObINnwr3Q55jAZLwZts='

fail 7385139 2023-08-30 23:50:45 2023-08-31 01:52:52 2023-08-31 02:03:21 0:10:29 smithi main ubuntu 22.04 fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} 4
Failure Reason:

Host key for server 'smithi203.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIG/sKZobCZzFhsw55oZVxtleZi+LDr+/oaa4b9TY6+q3', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABgQCq94bKceR9yaplU7zmCtpvTPhDXV7niqens6kz35BD4WsZg840vnxzLBgQ93s3zaMq5IAOr3lRNOl6srwffEt8H4Hs32V1zXLV0D/VJbC0nSiE7x3m4kjYhYqsBqItN+4TRBy6AiaAeKkouXKcJHa2639qSOzycyQvVXL+wKcQjCLtxajm6dsjoWzka6sjZt2RH7fkTeXBTpTVeUGg/ztcZBcfMp7x4hTsL7E7djLu15kVXMmaECRNF7gCJUeCuvZPmUZpDqhx/zlKp15ueQKsaYx6OxX3KnP+I0sDpf00pO1sQ6x/V6el2V3WBhocNUUhNuwD2hjO4A512yRgFEfLLcYPSjvPxV4081+DUKWNIT0KpQ2SR8EhQDsmE/vtJNLvlQg8up9uZc4x/rqWw13NZvZbfBD1nU9bnmdK+T/JdayDWkkUqzyXVKL6TtiwEXQjmWvDBpEL+H/m+vQ3/QN8JfLpK/65VZOHJcsiyZ3VLDFB5WC9sm1PggM+iTrr6v0='

fail 7385140 2023-08-30 23:50:45 2023-08-31 01:55:03 2023-08-31 02:02:41 0:07:38 smithi main centos 9.stream fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} 2
fail 7385141 2023-08-30 23:50:46 2023-08-31 01:55:34 2023-08-31 02:03:44 0:08:10 smithi main centos 9.stream fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} 2
fail 7385142 2023-08-30 23:50:47 2023-08-31 01:56:24 2023-08-31 02:05:16 0:08:52 smithi main centos 9.stream fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} 2
pass 7385143 2023-08-30 23:50:47 2023-08-31 01:58:15 2023-08-31 02:23:57 0:25:42 0:18:42 0:07:00 smithi main rhel 8.6 fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} 2
fail 7385144 2023-08-30 23:50:48 2023-08-31 01:58:45 2023-08-31 02:20:19 0:21:34 0:11:06 0:10:28 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
Failure Reason:

Command failed on smithi012 with status 22: "sudo ceph --cluster ceph config set client 'client mount timeout' 600"

pass 7385145 2023-08-30 23:50:49 2023-08-31 01:58:56 2023-08-31 02:42:56 0:44:00 0:35:53 0:08:07 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} 3
pass 7385146 2023-08-30 23:50:50 2023-08-31 01:59:46 2023-08-31 03:57:44 1:57:58 1:49:55 0:08:03 smithi main rhel 8.6 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} 2
pass 7385147 2023-08-30 23:50:50 2023-08-31 02:00:37 2023-08-31 02:32:54 0:32:17 0:20:33 0:11:44 smithi main ubuntu 20.04 fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} 2
pass 7385148 2023-08-30 23:50:51 2023-08-31 02:00:57 2023-08-31 02:24:26 0:23:29 0:14:38 0:08:51 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_8 conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} 1
pass 7385149 2023-08-30 23:50:52 2023-08-31 02:01:48 2023-08-31 02:30:38 0:28:50 0:15:22 0:13:28 smithi main ubuntu 20.04 fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} 2
pass 7385150 2023-08-30 23:50:52 2023-08-31 02:02:48 2023-08-31 02:52:17 0:49:29 0:39:54 0:09:35 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} 3