| User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Queued | Pass | Fail | Dead |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| pdonnell | 2024-03-15 13:00:41 | 2024-03-16 06:43:14 | 2024-03-16 16:22:03 | 9:38:49 | fs | wip-batrick-testing-20240314.183243 | smithi | 3d78f5b | 1 | 136 | 100 | 11 |
| Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| pass | 7602192 |
|
2024-03-15 13:00:56 | 2024-03-15 13:43:10 | 1610 | smithi | main | ubuntu | 22.04 | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} | 2 | |||
| pass | 7602193 |
|
2024-03-15 13:00:57 | 2024-03-15 13:04:59 | 2024-03-15 14:18:39 | 1:13:40 | 1:01:49 | 0:11:51 | smithi | main | centos | 9.stream | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 |
| fail | 7602194 |
|
2024-03-15 13:00:58 | 2024-03-15 13:05:00 | 2024-03-15 13:56:50 | 0:51:50 | 0:40:31 | 0:11:19 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} | 3 |
Failure Reason:
"2024-03-15T13:27:59.546573+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602195 |
|
2024-03-15 13:00:59 | 2024-03-15 13:05:00 | 2024-03-15 13:37:22 | 0:32:22 | 0:21:15 | 0:11:07 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/journal-repair} | 2 |
| pass | 7602196 |
|
2024-03-15 13:01:00 | 2024-03-15 13:05:00 | 2024-03-15 14:22:41 | 1:17:41 | 0:28:56 | 0:48:45 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7602197 |
|
2024-03-15 13:01:02 | 2024-03-15 13:07:31 | 2024-03-15 13:56:58 | 0:49:27 | 0:37:11 | 0:12:16 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} | 3 |
Failure Reason:
Command failed (workunit test suites/dbench.sh) on smithi059 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' |
||||||||||||||
| pass | 7602198 |
|
2024-03-15 13:01:03 | 2024-03-15 13:09:52 | 2024-03-15 13:40:10 | 0:30:18 | 0:18:43 | 0:11:35 | smithi | main | ubuntu | 22.04 | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} | 2 |
| pass | 7602199 |
|
2024-03-15 13:01:04 | 2024-03-15 13:10:32 | 2024-03-15 13:39:53 | 0:29:21 | 0:16:28 | 0:12:53 | smithi | main | centos | 9.stream | fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn} tasks/{0-client 1-tests/fscrypt-pjd}} | 3 |
| pass | 7602200 |
|
2024-03-15 13:01:05 | 2024-03-15 13:12:53 | 2024-03-15 14:01:20 | 0:48:27 | 0:36:41 | 0:11:46 | smithi | main | centos | 9.stream | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} | 2 |
| fail | 7602201 |
|
2024-03-15 13:01:06 | 2024-03-15 13:14:04 | 2024-03-15 14:22:21 | 1:08:17 | 0:56:09 | 0:12:08 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} | 3 |
Failure Reason:
"2024-03-15T13:36:16.165620+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602202 |
|
2024-03-15 13:01:07 | 2024-03-15 13:15:04 | 2024-03-15 13:41:26 | 0:26:22 | 0:13:09 | 0:13:13 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/mds-flush} | 2 |
| pass | 7602203 |
|
2024-03-15 13:01:08 | 2024-03-15 13:16:15 | 2024-03-15 13:41:46 | 0:25:31 | 0:13:36 | 0:11:55 | smithi | main | centos | 9.stream | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} | 2 |
| pass | 7602204 |
|
2024-03-15 13:01:09 | 2024-03-15 13:16:45 | 2024-03-15 13:43:44 | 0:26:59 | 0:14:17 | 0:12:42 | smithi | main | centos | 9.stream | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} | 2 |
| pass | 7602205 |
|
2024-03-15 13:01:10 | 2024-03-15 13:18:46 | 2024-03-15 14:26:49 | 1:08:03 | 0:29:43 | 0:38:20 | smithi | main | centos | 8.stream | fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} | 2 |
| fail | 7602206 |
|
2024-03-15 13:01:11 | 2024-03-15 13:19:06 | 2024-03-15 14:01:08 | 0:42:02 | 0:31:35 | 0:10:27 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602207 |
|
2024-03-15 13:01:12 | 2024-03-15 13:19:47 | 2024-03-15 13:47:40 | 0:27:53 | 0:17:55 | 0:09:58 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} | 2 |
| pass | 7602208 |
|
2024-03-15 13:01:13 | 2024-03-15 13:19:47 | 2024-03-15 13:48:30 | 0:28:43 | 0:19:34 | 0:09:09 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/mds-full} | 2 |
| fail | 7602209 |
|
2024-03-15 13:01:14 | 2024-03-15 13:19:48 | 2024-03-15 14:03:14 | 0:43:26 | 0:30:48 | 0:12:38 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsstress}} | 3 |
Failure Reason:
"2024-03-15T13:43:45.919373+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602210 |
|
2024-03-15 13:01:15 | 2024-03-15 13:21:49 | 2024-03-15 14:08:42 | 0:46:53 | 0:33:17 | 0:13:36 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-03-15T13:57:09.215642+0000 mon.smithi103 (mon.0) 250 : cluster [WRN] Health check failed: Degraded data redundancy: 44/222 objects degraded (19.820%), 18 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7602211 |
|
2024-03-15 13:01:16 | 2024-03-15 13:23:19 | 2024-03-15 14:08:15 | 0:44:56 | 0:32:11 | 0:12:45 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} | 3 |
Failure Reason:
Command failed (workunit test suites/fsx.sh) on smithi077 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
| pass | 7602212 |
|
2024-03-15 13:01:17 | 2024-03-15 13:27:10 | 2024-03-15 14:03:12 | 0:36:02 | 0:16:57 | 0:19:05 | smithi | main | ubuntu | 22.04 | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} | 2 |
| pass | 7602213 |
|
2024-03-15 13:01:19 | 2024-03-15 13:37:32 | 2024-03-15 14:03:47 | 0:26:15 | 0:13:29 | 0:12:46 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/mds_creation_retry} | 2 |
| fail | 7602214 |
|
2024-03-15 13:01:20 | 2024-03-15 13:40:03 | 2024-03-15 14:20:42 | 0:40:39 | 0:28:29 | 0:12:10 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} | 3 |
Failure Reason:
"2024-03-15T14:03:02.369888+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602215 |
|
2024-03-15 13:01:21 | 2024-03-15 13:40:13 | 2024-03-15 14:28:06 | 0:47:53 | 0:33:29 | 0:14:24 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} | 3 |
Failure Reason:
"2024-03-15T14:01:56.322223+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602216 |
|
2024-03-15 13:01:22 | 2024-03-15 13:41:34 | 2024-03-15 14:19:23 | 0:37:49 | 0:27:32 | 0:10:17 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/metrics} | 2 |
| pass | 7602217 |
|
2024-03-15 13:01:23 | 2024-03-15 13:41:54 | 2024-03-15 14:04:44 | 0:22:50 | 0:10:46 | 0:12:04 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} | 2 |
| fail | 7602218 |
|
2024-03-15 13:01:24 | 2024-03-15 13:43:15 | 2024-03-15 14:32:24 | 0:49:09 | 0:21:55 | 0:27:14 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 |
Failure Reason:
Command failed on smithi001 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell 4.3a deep-scrub' |
||||||||||||||
| fail | 7602219 |
|
2024-03-15 13:01:25 | 2024-03-15 13:47:26 | 2024-03-15 14:43:33 | 0:56:07 | 0:44:33 | 0:11:34 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} | 3 |
Failure Reason:
"2024-03-15T14:08:44.461985+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602220 |
|
2024-03-15 13:01:26 | 2024-03-15 13:47:46 | 2024-03-15 14:45:39 | 0:57:53 | 0:46:16 | 0:11:37 | smithi | main | centos | 9.stream | fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} | 2 |
| pass | 7602221 |
|
2024-03-15 13:01:27 | 2024-03-15 13:48:37 | 2024-03-15 14:49:26 | 1:00:49 | 0:47:15 | 0:13:34 | smithi | main | ubuntu | 22.04 | fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} | 2 |
| pass | 7602222 |
|
2024-03-15 13:01:28 | 2024-03-15 13:50:47 | 2024-03-15 14:17:48 | 0:27:01 | 0:16:04 | 0:10:57 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/multimds_misc} | 2 |
| fail | 7602223 |
|
2024-03-15 13:01:29 | 2024-03-15 13:50:48 | 2024-03-15 14:45:34 | 0:54:46 | 0:31:58 | 0:22:48 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} | 3 |
Failure Reason:
"2024-03-15T14:28:24.880965+0000 mon.a (mon.0) 1101 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7602224 |
|
2024-03-15 13:01:30 | 2024-03-16 06:43:14 | 2024-03-16 07:09:06 | 0:25:52 | 0:11:35 | 0:14:17 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} | 2 |
| fail | 7602225 |
|
2024-03-15 13:01:31 | 2024-03-16 06:46:15 | 2024-03-16 07:35:39 | 0:49:24 | 0:38:21 | 0:11:03 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7602226 |
|
2024-03-15 13:01:32 | 2024-03-16 06:46:15 | 2024-03-16 07:31:49 | 0:45:34 | 0:34:19 | 0:11:15 | smithi | main | ubuntu | 22.04 | fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/valgrind} | 2 |
Failure Reason:
valgrind error: Leak_PossiblyLost calloc __trans_list_add |
||||||||||||||
| fail | 7602227 |
|
2024-03-15 13:01:33 | 2024-03-16 06:46:15 | 2024-03-16 07:33:03 | 0:46:48 | 0:30:29 | 0:16:19 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/pjd}} | 3 |
Failure Reason:
"2024-03-16T07:15:07.471316+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602228 |
|
2024-03-15 13:01:34 | 2024-03-16 06:51:47 | 2024-03-16 07:17:41 | 0:25:54 | 0:14:27 | 0:11:27 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/openfiletable} | 2 |
Failure Reason:
Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) |
||||||||||||||
| pass | 7602229 |
|
2024-03-15 13:01:35 | 2024-03-16 06:51:47 | 2024-03-16 07:21:21 | 0:29:34 | 0:17:44 | 0:11:50 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} | 2 |
| pass | 7602230 |
|
2024-03-15 13:01:36 | 2024-03-16 06:52:18 | 2024-03-16 07:21:59 | 0:29:41 | 0:16:51 | 0:12:50 | smithi | main | ubuntu | 22.04 | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} | 2 |
| pass | 7602231 |
|
2024-03-15 13:01:37 | 2024-03-16 06:53:28 | 2024-03-16 07:15:10 | 0:21:42 | 0:09:03 | 0:12:39 | smithi | main | centos | 9.stream | fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} | 1 |
| fail | 7602232 |
|
2024-03-15 13:01:39 | 2024-03-16 06:55:19 | 2024-03-16 07:50:51 | 0:55:32 | 0:44:15 | 0:11:17 | smithi | main | centos | 9.stream | fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host} | 2 |
Failure Reason:
Command failed (workunit test suites/dbench.sh) on smithi082 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh' |
||||||||||||||
| pass | 7602233 |
|
2024-03-15 13:01:40 | 2024-03-16 06:55:49 | 2024-03-16 07:26:39 | 0:30:50 | 0:19:59 | 0:10:51 | smithi | main | ubuntu | 22.04 | fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} | 1 |
| pass | 7602234 |
|
2024-03-15 13:01:41 | 2024-03-16 06:55:50 | 2024-03-16 07:15:45 | 0:19:55 | 0:09:44 | 0:10:11 | smithi | main | centos | 9.stream | fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} | 2 |
| fail | 7602235 |
|
2024-03-15 13:01:42 | 2024-03-16 06:56:40 | 2024-03-16 09:12:37 | 2:15:57 | 2:04:50 | 0:11:07 | smithi | main | ubuntu | 22.04 | fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} | 1 |
Failure Reason:
"2024-03-16T07:34:27.356476+0000 mon.a (mon.0) 2180 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7602236 |
|
2024-03-15 13:01:43 | 2024-03-16 06:56:40 | 2024-03-16 07:23:31 | 0:26:51 | 0:17:32 | 0:09:19 | smithi | main | centos | 9.stream | fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distro$/{centos_latest} workloads/cephfs-mirror-ha-workunit} | 1 |
| pass | 7602237 |
|
2024-03-15 13:01:44 | 2024-03-16 06:56:41 | 2024-03-16 08:12:34 | 1:15:53 | 0:59:54 | 0:15:59 | smithi | main | ubuntu | 22.04 | fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} | 5 |
| pass | 7602238 |
|
2024-03-15 13:01:45 | 2024-03-16 07:01:42 | 2024-03-16 07:49:51 | 0:48:09 | 0:37:49 | 0:10:20 | smithi | main | ubuntu | 22.04 | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} | 2 |
| fail | 7602239 |
|
2024-03-15 13:01:46 | 2024-03-16 07:01:42 | 2024-03-16 07:46:34 | 0:44:52 | 0:34:54 | 0:09:58 | smithi | main | centos | 9.stream | fs/nfs/{cluster/{1-node} conf/{client mds mon osd} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} | 1 |
Failure Reason:
"2024-03-16T07:21:37.883938+0000 mon.a (mon.0) 491 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7602240 |
|
2024-03-15 13:01:47 | 2024-03-16 07:01:43 | 2024-03-16 07:24:50 | 0:23:07 | 0:13:27 | 0:09:40 | smithi | main | centos | 9.stream | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} | 2 |
| dead | 7602241 |
|
2024-03-15 13:01:48 | 2024-03-16 07:01:43 | 2024-03-16 07:07:38 | 0:05:55 | smithi | main | centos | 9.stream | fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/centos_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} | 2 | ||
Failure Reason:
Error reimaging machines: Expected smithi152's OS to be centos 9 but found ubuntu 22.04 |
||||||||||||||
| fail | 7602242 |
|
2024-03-15 13:01:49 | 2024-03-16 07:01:44 | 2024-03-16 07:09:42 | 0:07:58 | smithi | main | ubuntu | 22.04 | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 | ||
Failure Reason:
[Errno None] Unable to connect to port 22 on 172.21.15.152 |
||||||||||||||
| pass | 7602243 |
|
2024-03-15 13:01:50 | 2024-03-16 07:01:44 | 2024-03-16 07:20:49 | 0:19:05 | 0:10:37 | 0:08:28 | smithi | main | centos | 9.stream | fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/fstop} | 1 |
| fail | 7602244 |
|
2024-03-15 13:01:51 | 2024-03-16 07:01:44 | 2024-03-16 07:44:03 | 0:42:19 | 0:31:27 | 0:10:52 | smithi | main | centos | 9.stream | fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/ignorelist_health tasks/mirror}} | 1 |
Failure Reason:
Test failure: test_cephfs_mirror_cancel_mirroring_and_readd (tasks.cephfs.test_mirroring.TestMirroring) |
||||||||||||||
| pass | 7602245 |
|
2024-03-15 13:01:52 | 2024-03-16 07:01:45 | 2024-03-16 09:03:01 | 2:01:16 | 1:51:00 | 0:10:16 | smithi | main | centos | 9.stream | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} | 2 |
| fail | 7602246 |
|
2024-03-15 13:01:53 | 2024-03-16 07:01:45 | 2024-03-16 07:46:04 | 0:44:19 | 0:32:31 | 0:11:48 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} | 3 |
Failure Reason:
"2024-03-16T07:21:48.115324+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602247 |
|
2024-03-15 13:01:54 | 2024-03-16 07:03:16 | 2024-03-16 08:14:59 | 1:11:43 | 0:59:28 | 0:12:15 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/misc}} | 3 |
Failure Reason:
"2024-03-16T07:26:35.071817+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602248 |
|
2024-03-15 13:01:55 | 2024-03-16 07:05:16 | 2024-03-16 07:28:33 | 0:23:17 | 0:12:11 | 0:11:06 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} | 2 |
Failure Reason:
Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) |
||||||||||||||
| pass | 7602249 |
|
2024-03-15 13:01:56 | 2024-03-16 07:05:27 | 2024-03-16 07:40:38 | 0:35:11 | 0:24:53 | 0:10:18 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| pass | 7602250 |
|
2024-03-15 13:01:57 | 2024-03-16 07:05:27 | 2024-03-16 07:30:38 | 0:25:11 | 0:12:49 | 0:12:22 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/pjd}} | 2 |
| fail | 7602251 |
|
2024-03-15 13:01:59 | 2024-03-16 07:05:58 | 2024-03-16 09:03:03 | 1:57:05 | 1:44:32 | 0:12:33 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/kernel_untar_build}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602252 |
|
2024-03-15 13:02:00 | 2024-03-16 07:07:48 | 2024-03-16 07:56:07 | 0:48:19 | 0:36:52 | 0:11:27 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} | 2 |
| fail | 7602253 |
|
2024-03-15 13:02:01 | 2024-03-16 07:09:09 | 2024-03-16 08:23:41 | 1:14:32 | 0:55:17 | 0:19:15 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/postgres}} | 3 |
Failure Reason:
"2024-03-16T07:38:39.125023+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602254 |
|
2024-03-15 13:02:02 | 2024-03-16 07:15:50 | 2024-03-16 07:36:47 | 0:20:57 | 0:11:44 | 0:09:13 | smithi | main | centos | 8.stream | fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} | 1 |
| pass | 7602255 |
|
2024-03-15 13:02:03 | 2024-03-16 07:15:51 | 2024-03-16 07:52:35 | 0:36:44 | 0:24:39 | 0:12:05 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} | 2 |
| fail | 7602256 |
|
2024-03-15 13:02:04 | 2024-03-16 07:17:11 | 2024-03-16 08:02:30 | 0:45:19 | 0:34:24 | 0:10:55 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} | 3 |
Failure Reason:
"2024-03-16T07:35:50.134302+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602257 |
|
2024-03-15 13:02:05 | 2024-03-16 07:17:22 | 2024-03-16 07:42:24 | 0:25:02 | 0:15:17 | 0:09:45 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quota} | 2 |
| fail | 7602258 |
|
2024-03-15 13:02:06 | 2024-03-16 07:17:22 | 2024-03-16 08:46:38 | 1:29:16 | 1:15:11 | 0:14:05 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} | 3 |
Failure Reason:
"2024-03-16T07:42:14.671676+0000 mon.a (mon.0) 214 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602259 |
|
2024-03-15 13:02:07 | 2024-03-16 07:21:23 | 2024-03-16 09:06:02 | 1:44:39 | 1:33:26 | 0:11:13 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/snaps}} | 2 |
| pass | 7602260 |
|
2024-03-15 13:02:08 | 2024-03-16 07:22:04 | 2024-03-16 07:48:39 | 0:26:35 | 0:12:59 | 0:13:36 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/recovery-fs} | 2 |
| fail | 7602261 |
|
2024-03-15 13:02:09 | 2024-03-16 07:24:54 | 2024-03-16 08:23:18 | 0:58:24 | 0:39:32 | 0:18:52 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} | 3 |
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on smithi027 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
| fail | 7602262 |
|
2024-03-15 13:02:10 | 2024-03-16 07:30:46 | 2024-03-16 08:17:40 | 0:46:54 | 0:33:35 | 0:13:19 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-03-16T08:06:13.052431+0000 mon.smithi003 (mon.0) 249 : cluster [WRN] Health check failed: Degraded data redundancy: 43/216 objects degraded (19.907%), 17 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7602263 |
|
2024-03-15 13:02:11 | 2024-03-16 07:32:36 | 2024-03-16 08:11:41 | 0:39:05 | 0:29:27 | 0:09:38 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/norstats}} | 3 |
Failure Reason:
"2024-03-16T07:52:40.957099+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602264 |
|
2024-03-15 13:02:12 | 2024-03-16 07:32:37 | 2024-03-16 07:55:25 | 0:22:48 | 0:13:21 | 0:09:27 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} | 2 |
| pass | 7602265 |
|
2024-03-15 13:02:13 | 2024-03-16 07:32:37 | 2024-03-16 08:06:03 | 0:33:26 | 0:23:07 | 0:10:19 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} | 2 |
| fail | 7602266 |
|
2024-03-15 13:02:14 | 2024-03-16 07:32:47 | 2024-03-16 08:13:41 | 0:40:54 | 0:32:20 | 0:08:34 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsstress}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602267 |
|
2024-03-15 13:02:15 | 2024-03-16 07:32:48 | 2024-03-16 08:04:05 | 0:31:17 | 0:19:56 | 0:11:21 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| pass | 7602268 |
|
2024-03-15 13:02:16 | 2024-03-16 07:32:48 | 2024-03-16 07:57:45 | 0:24:57 | 0:11:10 | 0:13:47 | smithi | main | centos | 9.stream | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| pass | 7602269 |
|
2024-03-15 13:02:18 | 2024-03-16 07:36:49 | 2024-03-16 08:06:11 | 0:29:22 | 0:15:34 | 0:13:48 | smithi | main | ubuntu | 22.04 | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| pass | 7602270 |
|
2024-03-15 13:02:19 | 2024-03-16 07:40:40 | 2024-03-16 08:49:53 | 1:09:13 | 0:57:55 | 0:11:18 | smithi | main | centos | 9.stream | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} | 2 |
| fail | 7602271 |
|
2024-03-15 13:02:20 | 2024-03-16 07:42:31 | 2024-03-16 08:28:46 | 0:46:15 | 0:29:48 | 0:16:27 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} | 3 |
Failure Reason:
Command failed (workunit test suites/fsx.sh) on smithi007 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
| pass | 7602272 |
|
2024-03-15 13:02:21 | 2024-03-16 07:48:02 | 2024-03-16 08:41:26 | 0:53:24 | 0:41:45 | 0:11:39 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} | 2 |
| pass | 7602273 |
|
2024-03-15 13:02:22 | 2024-03-16 07:48:02 | 2024-03-16 08:14:56 | 0:26:54 | 0:15:55 | 0:10:59 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/sessionmap} | 2 |
| fail | 7602274 |
|
2024-03-15 13:02:23 | 2024-03-16 07:48:03 | 2024-03-16 08:32:18 | 0:44:15 | 0:32:16 | 0:11:59 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}} | 3 |
Failure Reason:
"2024-03-16T08:09:02.237965+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602275 |
|
2024-03-15 13:02:24 | 2024-03-16 07:48:03 | 2024-03-16 08:27:17 | 0:39:14 | 0:29:16 | 0:09:58 | smithi | main | centos | 9.stream | fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} | 2 |
| pass | 7602276 |
|
2024-03-15 13:02:25 | 2024-03-16 07:48:24 | 2024-03-16 08:51:55 | 1:03:31 | 0:54:19 | 0:09:12 | smithi | main | ubuntu | 22.04 | fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} | 2 |
| fail | 7602277 |
|
2024-03-15 13:02:26 | 2024-03-16 07:48:24 | 2024-03-16 09:50:26 | 2:02:02 | 1:52:30 | 0:09:32 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/snap-schedule} | 2 |
Failure Reason:
Test failure: test_snap_schedule_subvol_and_group_arguments_01 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments) |
||||||||||||||
| fail | 7602278 |
|
2024-03-15 13:02:27 | 2024-03-16 07:48:24 | 2024-03-16 08:42:27 | 0:54:03 | 0:37:06 | 0:16:57 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} | 3 |
Failure Reason:
"2024-03-16T08:14:31.860902+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602279 |
|
2024-03-15 13:02:28 | 2024-03-16 07:52:45 | 2024-03-16 09:18:35 | 1:25:50 | 1:13:41 | 0:12:09 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} | 2 |
| pass | 7602280 |
|
2024-03-15 13:02:29 | 2024-03-16 07:55:26 | 2024-03-16 08:22:18 | 0:26:52 | 0:13:20 | 0:13:32 | smithi | main | ubuntu | 22.04 | fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} | 4 |
| pass | 7602281 |
|
2024-03-15 13:02:30 | 2024-03-16 07:57:47 | 2024-03-16 09:19:09 | 1:21:22 | 1:07:49 | 0:13:33 | smithi | main | centos | 9.stream | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} | 2 |
| pass | 7602282 |
|
2024-03-15 13:02:31 | 2024-03-16 08:01:48 | 2024-03-16 08:58:51 | 0:57:03 | 0:47:33 | 0:09:30 | smithi | main | centos | 9.stream | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 |
| fail | 7602283 |
|
2024-03-15 13:02:32 | 2024-03-16 08:01:48 | 2024-03-16 08:52:55 | 0:51:07 | 0:38:10 | 0:12:57 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| dead | 7602284 |
|
2024-03-15 13:02:33 | 2024-03-16 08:03:29 | 2024-03-16 08:04:33 | 0:01:04 | smithi | main | ubuntu | 22.04 | fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep} | 2 | ||
Failure Reason:
Error reimaging machines: Failed to power on smithi089 |
||||||||||||||
| dead | 7602285 |
|
2024-03-15 13:02:34 | 2024-03-16 08:03:29 | 2024-03-16 08:04:34 | 0:01:05 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iogen}} | 3 | ||
Failure Reason:
Error reimaging machines: Failed to power on smithi121 |
||||||||||||||
| pass | 7602286 |
|
2024-03-15 13:02:35 | 2024-03-16 08:03:30 | 2024-03-16 08:22:41 | 0:19:11 | 0:09:29 | 0:09:42 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/snap_schedule_snapdir} | 2 |
| pass | 7602287 |
|
2024-03-15 13:02:37 | 2024-03-16 08:03:50 | 2024-03-16 08:33:28 | 0:29:38 | 0:20:22 | 0:09:16 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} | 2 |
| fail | 7602288 |
|
2024-03-15 13:02:38 | 2024-03-16 08:03:51 | 2024-03-16 08:45:34 | 0:41:43 | 0:29:14 | 0:12:29 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iozone}} | 3 |
Failure Reason:
"2024-03-16T08:26:06.586358+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602289 |
|
2024-03-15 13:02:39 | 2024-03-16 08:04:41 | 2024-03-16 08:51:38 | 0:46:57 | 0:34:04 | 0:12:53 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/pjd}} | 3 |
Failure Reason:
"2024-03-16T08:27:48.940242+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602290 |
|
2024-03-15 13:02:40 | 2024-03-16 08:06:12 | 2024-03-16 08:44:00 | 0:37:48 | 0:26:50 | 0:10:58 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/snapshots} | 2 |
| pass | 7602291 |
|
2024-03-15 13:02:41 | 2024-03-16 08:06:12 | 2024-03-16 08:33:18 | 0:27:06 | 0:17:34 | 0:09:32 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} | 2 |
| fail | 7602292 |
|
2024-03-15 13:02:42 | 2024-03-16 08:06:13 | 2024-03-16 08:44:57 | 0:38:44 | 0:22:08 | 0:16:36 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 |
Failure Reason:
Command failed on smithi023 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell 4.29 deep-scrub' |
||||||||||||||
| fail | 7602293 |
|
2024-03-15 13:02:43 | 2024-03-16 08:12:44 | 2024-03-16 08:57:46 | 0:45:02 | 0:36:09 | 0:08:53 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/direct_io}} | 3 |
Failure Reason:
"2024-03-16T08:33:09.430629+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602294 |
|
2024-03-15 13:02:44 | 2024-03-16 08:12:44 | 2024-03-16 08:56:46 | 0:44:02 | 0:32:10 | 0:11:52 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/strays} | 2 |
Failure Reason:
"2024-03-16T08:48:38.740668+0000 mon.a (mon.0) 1939 : cluster [WRN] Health check failed: Degraded data redundancy: 88/6748 objects degraded (1.304%), 2 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7602295 |
|
2024-03-15 13:02:45 | 2024-03-16 08:15:05 | 2024-03-16 09:39:58 | 1:24:53 | 1:10:52 | 0:14:01 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602296 |
|
2024-03-15 13:02:46 | 2024-03-16 08:18:56 | 2024-03-16 08:41:49 | 0:22:53 | 0:11:43 | 0:11:10 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} | 2 |
| pass | 7602297 |
|
2024-03-15 13:02:47 | 2024-03-16 08:18:56 | 2024-03-16 08:42:27 | 0:23:31 | 0:13:43 | 0:09:48 | smithi | main | centos | 9.stream | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} | 2 |
| pass | 7602298 |
|
2024-03-15 13:02:48 | 2024-03-16 08:18:57 | 2024-03-16 08:40:29 | 0:21:32 | 0:12:56 | 0:08:36 | smithi | main | ubuntu | 22.04 | fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} | 2 |
| pass | 7602299 |
|
2024-03-15 13:02:49 | 2024-03-16 08:18:57 | 2024-03-16 08:46:30 | 0:27:33 | 0:17:10 | 0:10:23 | smithi | main | ubuntu | 22.04 | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} | 2 |
| fail | 7602300 |
|
2024-03-15 13:02:51 | 2024-03-16 08:18:58 | 2024-03-16 10:24:24 | 2:05:26 | 1:50:40 | 0:14:46 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} | 3 |
Failure Reason:
"2024-03-16T08:43:58.194500+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602301 |
|
2024-03-15 13:02:52 | 2024-03-16 08:22:29 | 2024-03-16 08:47:53 | 0:25:24 | 0:15:40 | 0:09:44 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/test_journal_migration} | 2 |
| pass | 7602302 |
|
2024-03-15 13:02:53 | 2024-03-16 08:22:29 | 2024-03-16 10:47:14 | 2:24:45 | 2:08:34 | 0:16:11 | smithi | main | ubuntu | 22.04 | fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn} tasks/{0-client 1-tests/fscrypt-ffsb}} | 3 |
| pass | 7602303 |
|
2024-03-15 13:02:54 | 2024-03-16 08:27:20 | 2024-03-16 08:56:56 | 0:29:36 | 0:13:45 | 0:15:51 | smithi | main | ubuntu | 22.04 | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} | 2 |
| fail | 7602304 |
|
2024-03-15 13:02:55 | 2024-03-16 08:33:21 | 2024-03-16 09:22:40 | 0:49:19 | 0:38:48 | 0:10:31 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} | 3 |
Failure Reason:
"2024-03-16T08:51:21.344251+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602305 |
|
2024-03-15 13:02:56 | 2024-03-16 08:33:32 | 2024-03-16 09:17:07 | 0:43:35 | 0:31:45 | 0:11:50 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-03-16T09:10:00.000172+0000 mon.smithi007 (mon.0) 552 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; Degraded data redundancy: 45/354 objects degraded (12.712%), 10 pgs degraded" in cluster log |
||||||||||||||
| pass | 7602306 |
|
2024-03-15 13:02:57 | 2024-03-16 08:34:22 | 2024-03-16 09:03:27 | 0:29:05 | 0:18:49 | 0:10:16 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/iozone}} | 2 |
| fail | 7602307 |
|
2024-03-15 13:02:58 | 2024-03-16 08:34:23 | 2024-03-16 09:34:14 | 0:59:51 | 0:47:44 | 0:12:07 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} | 3 |
Failure Reason:
"2024-03-16T08:56:22.841040+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602308 |
|
2024-03-15 13:02:59 | 2024-03-16 08:34:23 | 2024-03-16 09:01:01 | 0:26:38 | 0:14:23 | 0:12:15 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/truncate_delay} | 2 |
| pass | 7602309 |
|
2024-03-15 13:03:00 | 2024-03-16 08:34:23 | 2024-03-16 09:12:53 | 0:38:30 | 0:25:02 | 0:13:28 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7602310 |
|
2024-03-15 13:03:01 | 2024-03-16 08:34:24 | 2024-03-16 09:53:10 | 1:18:46 | 0:59:20 | 0:19:26 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602311 |
|
2024-03-15 13:03:03 | 2024-03-16 08:41:35 | 2024-03-16 09:04:53 | 0:23:18 | 0:12:39 | 0:10:39 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} | 2 |
| pass | 7602312 |
|
2024-03-15 13:03:04 | 2024-03-16 08:41:56 | 2024-03-16 09:08:49 | 0:26:53 | 0:15:20 | 0:11:33 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/workunit/dir-max-entries} | 2 |
| fail | 7602313 |
|
2024-03-15 13:03:05 | 2024-03-16 08:42:36 | 2024-03-16 09:43:14 | 1:00:38 | 0:48:28 | 0:12:10 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} | 3 |
Failure Reason:
Command failed (workunit test suites/ffsb.sh) on smithi079 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' |
||||||||||||||
| fail | 7602314 |
|
2024-03-15 13:03:06 | 2024-03-16 08:44:07 | 2024-03-16 09:23:04 | 0:38:57 | 0:25:29 | 0:13:28 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602315 |
|
2024-03-15 13:03:07 | 2024-03-16 08:47:58 | 2024-03-16 09:15:02 | 0:27:04 | 0:15:27 | 0:11:37 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/pjd}} | 2 |
| pass | 7602316 |
|
2024-03-15 13:03:08 | 2024-03-16 08:49:38 | 2024-03-16 09:20:24 | 0:30:46 | 0:19:29 | 0:11:17 | smithi | main | ubuntu | 22.04 | fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} | 5 |
| pass | 7602317 |
|
2024-03-15 13:03:09 | 2024-03-16 08:49:39 | 2024-03-16 09:27:11 | 0:37:32 | 0:26:34 | 0:10:58 | smithi | main | centos | 9.stream | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} | 2 |
| pass | 7602318 |
|
2024-03-15 13:03:10 | 2024-03-16 08:49:39 | 2024-03-16 09:41:19 | 0:51:40 | 0:40:45 | 0:10:55 | smithi | main | ubuntu | 22.04 | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 |
| fail | 7602319 |
|
2024-03-15 13:03:11 | 2024-03-16 08:49:40 | 2024-03-16 16:22:03 | 7:32:23 | 7:17:11 | 0:15:12 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} | 2 |
Failure Reason:
Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) |
||||||||||||||
| pass | 7602320 |
|
2024-03-15 13:03:12 | 2024-03-16 08:49:40 | 2024-03-16 09:53:38 | 1:03:58 | 0:52:56 | 0:11:02 | smithi | main | ubuntu | 22.04 | fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} | 2 |
| pass | 7602321 |
|
2024-03-15 13:03:14 | 2024-03-16 08:50:00 | 2024-03-16 09:17:37 | 0:27:37 | 0:16:54 | 0:10:43 | smithi | main | ubuntu | 22.04 | fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} | 2 |
| fail | 7602322 |
|
2024-03-15 13:03:15 | 2024-03-16 08:52:01 | 2024-03-16 09:46:19 | 0:54:18 | 0:38:28 | 0:15:50 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} | 3 |
Failure Reason:
"2024-03-16T09:17:57.016977+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602323 |
|
2024-03-15 13:03:16 | 2024-03-16 08:57:02 | 2024-03-16 09:45:58 | 0:48:56 | 0:37:19 | 0:11:37 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7602324 |
|
2024-03-15 13:03:17 | 2024-03-16 08:58:53 | 2024-03-16 09:42:26 | 0:43:33 | 0:25:56 | 0:17:37 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} | 3 |
Failure Reason:
Command failed (workunit test suites/fsx.sh) on smithi005 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
| pass | 7602325 |
|
2024-03-15 13:03:18 | 2024-03-16 09:03:04 | 2024-03-16 09:28:06 | 0:25:02 | 0:10:49 | 0:14:13 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} | 2 |
| pass | 7602326 |
|
2024-03-15 13:03:19 | 2024-03-16 09:03:34 | 2024-03-16 09:30:43 | 0:27:09 | 0:15:25 | 0:11:44 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/workunit/quota} | 2 |
| dead | 7602327 |
|
2024-03-15 13:03:20 | 2024-03-16 09:04:55 | 2024-03-16 09:05:59 | 0:01:04 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 | ||
Failure Reason:
Error reimaging machines: Failed to power on smithi192 |
||||||||||||||
| fail | 7602328 |
|
2024-03-15 13:03:21 | 2024-03-16 09:04:55 | 2024-03-16 10:04:27 | 0:59:32 | 0:49:37 | 0:09:55 | smithi | main | ubuntu | 22.04 | fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} | 2 |
Failure Reason:
"2024-03-16T09:32:59.506725+0000 mon.a (mon.0) 245 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| dead | 7602329 |
|
2024-03-15 13:03:22 | 2024-03-16 09:04:56 | 2024-03-16 09:10:27 | 0:05:31 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}} | 3 | ||
Failure Reason:
Error reimaging machines: Expected smithi146's OS to be centos 9 but found ubuntu 22.04 |
||||||||||||||
| dead | 7602330 |
|
2024-03-15 13:03:23 | 2024-03-16 09:04:56 | 2024-03-16 09:15:46 | 0:10:50 | smithi | main | ubuntu | 22.04 | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 | ||
Failure Reason:
Error reimaging machines: Expected smithi164's OS to be ubuntu 22.04 but found centos 9 |
||||||||||||||
| pass | 7602331 |
|
2024-03-15 13:03:24 | 2024-03-16 09:04:56 | 2024-03-16 09:25:56 | 0:21:00 | 0:11:58 | 0:09:02 | smithi | main | centos | 9.stream | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| pass | 7602332 |
|
2024-03-15 13:03:25 | 2024-03-16 09:04:57 | 2024-03-16 09:25:48 | 0:20:51 | 0:10:34 | 0:10:17 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/acls} | 2 |
| pass | 7602333 |
|
2024-03-15 13:03:26 | 2024-03-16 09:04:57 | 2024-03-16 10:30:05 | 1:25:08 | 1:14:06 | 0:11:02 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} | 2 |
| fail | 7602334 |
|
2024-03-15 13:03:27 | 2024-03-16 09:04:58 | 2024-03-16 09:51:58 | 0:47:00 | 0:33:10 | 0:13:50 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} | 3 |
Failure Reason:
"2024-03-16T09:26:28.424790+0000 mon.a (mon.0) 208 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602335 |
|
2024-03-15 13:03:28 | 2024-03-16 09:06:08 | 2024-03-16 09:38:00 | 0:31:52 | 0:21:27 | 0:10:25 | smithi | main | ubuntu | 22.04 | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} | 2 |
| fail | 7602336 |
|
2024-03-15 13:03:30 | 2024-03-16 09:06:19 | 2024-03-16 10:12:22 | 1:06:03 | 0:51:44 | 0:14:19 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iogen}} | 3 |
Failure Reason:
"2024-03-16T09:31:36.671722+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| dead | 7602337 |
|
2024-03-15 13:03:31 | 2024-03-16 09:10:40 | 2024-03-16 09:17:47 | 0:07:07 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin} | 2 | ||
Failure Reason:
SSH connection to smithi164 was lost: 'sudo yum install -y kernel' |
||||||||||||||
| pass | 7602338 |
|
2024-03-15 13:03:32 | 2024-03-16 09:10:40 | 2024-03-16 09:52:04 | 0:41:24 | 0:27:14 | 0:14:10 | smithi | main | ubuntu | 22.04 | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} | 2 |
| fail | 7602339 |
|
2024-03-15 13:03:33 | 2024-03-16 09:13:01 | 2024-03-16 09:50:16 | 0:37:15 | 0:23:53 | 0:13:22 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iozone}} | 3 |
Failure Reason:
"2024-03-16T09:35:50.655146+0000 mon.a (mon.0) 487 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7602340 |
|
2024-03-15 13:03:34 | 2024-03-16 09:15:11 | 2024-03-16 10:03:00 | 0:47:49 | 0:34:57 | 0:12:52 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/pjd}} | 3 |
Failure Reason:
"2024-03-16T09:37:51.572708+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602341 |
|
2024-03-15 13:03:35 | 2024-03-16 09:17:42 | 2024-03-16 09:42:18 | 0:24:36 | 0:14:00 | 0:10:36 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/alternate-pool} | 2 |
| pass | 7602342 |
|
2024-03-15 13:03:36 | 2024-03-16 09:18:43 | 2024-03-16 10:04:34 | 0:45:51 | 0:35:08 | 0:10:43 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} | 2 |
| fail | 7602343 |
|
2024-03-15 13:03:37 | 2024-03-16 09:19:13 | 2024-03-16 09:52:00 | 0:32:47 | 0:22:12 | 0:10:35 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 |
Failure Reason:
Command failed on smithi007 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell 4.11 deep-scrub' |
||||||||||||||
| pass | 7602344 |
|
2024-03-15 13:03:38 | 2024-03-16 09:20:24 | 2024-03-16 09:54:20 | 0:33:56 | 0:23:17 | 0:10:39 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} | 3 |
| pass | 7602345 |
|
2024-03-15 13:03:39 | 2024-03-16 09:20:24 | 2024-03-16 09:47:57 | 0:27:33 | 0:17:33 | 0:10:00 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/asok_dump_tree} | 2 |
| fail | 7602346 |
|
2024-03-15 13:03:40 | 2024-03-16 09:20:25 | 2024-03-16 12:03:34 | 2:43:09 | 2:26:21 | 0:16:48 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} | 3 |
Failure Reason:
"2024-03-16T09:46:57.888901+0000 mon.a (mon.0) 213 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602347 |
|
2024-03-15 13:03:41 | 2024-03-16 09:25:56 | 2024-03-16 10:07:12 | 0:41:16 | 0:31:25 | 0:09:51 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-03-16T09:57:45.153066+0000 mon.smithi172 (mon.0) 245 : cluster [WRN] Health check failed: Degraded data redundancy: 67/345 objects degraded (19.420%), 19 pgs degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| pass | 7602348 |
|
2024-03-15 13:03:42 | 2024-03-16 09:25:56 | 2024-03-16 09:50:55 | 0:24:59 | 0:13:01 | 0:11:58 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} | 2 |
| fail | 7602349 |
|
2024-03-15 13:03:43 | 2024-03-16 09:27:17 | 2024-03-16 11:29:25 | 2:02:08 | 1:46:39 | 0:15:29 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/kernel_untar_build}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602350 |
|
2024-03-15 13:03:45 | 2024-03-16 09:30:48 | 2024-03-16 09:51:44 | 0:20:56 | 0:10:08 | 0:10:48 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/auto-repair} | 2 |
| pass | 7602351 |
|
2024-03-15 13:03:46 | 2024-03-16 09:32:59 | 2024-03-16 11:16:45 | 1:43:46 | 1:30:32 | 0:13:14 | smithi | main | centos | 8.stream | fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/kernel_untar_build}} | 2 |
| fail | 7602352 |
|
2024-03-15 13:03:47 | 2024-03-16 09:35:59 | 2024-03-16 10:47:59 | 1:12:00 | 1:00:03 | 0:11:57 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} | 3 |
Failure Reason:
"2024-03-16T09:59:20.780305+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602353 |
|
2024-03-15 13:03:48 | 2024-03-16 09:36:00 | 2024-03-16 10:02:58 | 0:26:58 | 0:17:53 | 0:09:05 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} | 2 |
| pass | 7602354 |
|
2024-03-15 13:03:49 | 2024-03-16 09:36:00 | 2024-03-16 10:01:29 | 0:25:29 | 0:15:06 | 0:10:23 | smithi | main | ubuntu | 22.04 | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| pass | 7602355 |
|
2024-03-15 13:03:50 | 2024-03-16 09:36:01 | 2024-03-16 10:05:07 | 0:29:06 | 0:17:42 | 0:11:24 | smithi | main | centos | 9.stream | fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health} | 2 |
| pass | 7602356 |
|
2024-03-15 13:03:51 | 2024-03-16 09:38:01 | 2024-03-16 10:08:28 | 0:30:27 | 0:18:18 | 0:12:09 | smithi | main | centos | 9.stream | fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} | 2 |
| pass | 7602357 |
|
2024-03-15 13:03:52 | 2024-03-16 09:41:22 | 2024-03-16 10:58:24 | 1:17:02 | 0:59:11 | 0:17:51 | smithi | main | ubuntu | 22.04 | fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} | 4 |
| pass | 7602358 |
|
2024-03-15 13:03:53 | 2024-03-16 09:48:04 | 2024-03-16 11:11:38 | 1:23:34 | 1:09:40 | 0:13:54 | smithi | main | centos | 9.stream | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} | 2 |
| pass | 7602359 |
|
2024-03-15 13:03:54 | 2024-03-16 09:51:04 | 2024-03-16 10:14:03 | 0:22:59 | 0:10:59 | 0:12:00 | smithi | main | centos | 9.stream | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| fail | 7602360 |
|
2024-03-15 13:03:55 | 2024-03-16 09:51:15 | 2024-03-16 10:13:19 | 0:22:04 | 0:12:25 | 0:09:39 | smithi | main | ubuntu | 22.04 | fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} | 2 |
Failure Reason:
Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) |
||||||||||||||
| pass | 7602361 |
|
2024-03-15 13:03:56 | 2024-03-16 09:51:15 | 2024-03-16 11:06:28 | 1:15:13 | 1:06:10 | 0:09:03 | smithi | main | centos | 9.stream | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 |
| fail | 7602362 |
|
2024-03-15 13:03:57 | 2024-03-16 09:51:15 | 2024-03-16 10:36:22 | 0:45:07 | 0:35:33 | 0:09:34 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} | 3 |
Failure Reason:
"2024-03-16T10:08:56.601034+0000 mon.a (mon.0) 208 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602363 |
|
2024-03-15 13:03:58 | 2024-03-16 09:51:16 | 2024-03-16 10:15:33 | 0:24:17 | 0:12:47 | 0:11:30 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/backtrace} | 2 |
| pass | 7602364 |
|
2024-03-15 13:03:59 | 2024-03-16 09:51:16 | 2024-03-16 10:24:15 | 0:32:59 | 0:24:31 | 0:08:28 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7602365 |
|
2024-03-15 13:04:00 | 2024-03-16 09:51:17 | 2024-03-16 10:52:55 | 1:01:38 | 0:50:08 | 0:11:30 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} | 3 |
Failure Reason:
Command failed (workunit test suites/dbench.sh) on smithi029 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' |
||||||||||||||
| pass | 7602366 |
|
2024-03-15 13:04:01 | 2024-03-16 09:51:17 | 2024-03-16 10:12:24 | 0:21:07 | 0:10:49 | 0:10:18 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} | 2 |
| pass | 7602367 |
|
2024-03-15 13:04:02 | 2024-03-16 09:51:18 | 2024-03-16 10:40:55 | 0:49:37 | 0:39:06 | 0:10:31 | smithi | main | centos | 9.stream | fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} | 2 |
| pass | 7602368 |
|
2024-03-15 13:04:03 | 2024-03-16 09:51:28 | 2024-03-16 10:18:55 | 0:27:27 | 0:18:39 | 0:08:48 | smithi | main | centos | 9.stream | fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} | 2 |
| pass | 7602369 |
|
2024-03-15 13:04:04 | 2024-03-16 09:51:48 | 2024-03-16 10:17:16 | 0:25:28 | 0:13:17 | 0:12:11 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/cap-flush} | 2 |
| pass | 7602370 |
|
2024-03-15 13:04:05 | 2024-03-16 09:52:09 | 2024-03-16 10:34:50 | 0:42:41 | 0:31:20 | 0:11:21 | smithi | main | centos | 9.stream | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} | 2 |
| pass | 7602371 |
|
2024-03-15 13:04:07 | 2024-03-16 09:53:39 | 2024-03-16 10:48:39 | 0:55:00 | 0:43:32 | 0:11:28 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} | 3 |
| pass | 7602372 |
|
2024-03-15 13:04:08 | 2024-03-16 09:54:30 | 2024-03-16 10:15:40 | 0:21:10 | 0:11:15 | 0:09:55 | smithi | main | centos | 8.stream | fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} | 1 |
| fail | 7602373 |
|
2024-03-15 13:04:09 | 2024-03-16 09:54:30 | 2024-03-16 10:52:43 | 0:58:13 | 0:35:43 | 0:22:30 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} | 3 |
Failure Reason:
"2024-03-16T10:27:34.415497+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602374 |
|
2024-03-15 13:04:10 | 2024-03-16 10:15:43 | 2024-03-16 11:36:12 | 1:20:29 | 1:09:23 | 0:11:06 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} | 2 |
| pass | 7602375 |
|
2024-03-15 13:04:11 | 2024-03-16 10:15:43 | 2024-03-16 10:48:54 | 0:33:11 | 0:17:52 | 0:15:19 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/client-limits} | 2 |
| fail | 7602376 |
|
2024-03-15 13:04:12 | 2024-03-16 10:17:24 | 2024-03-16 11:06:13 | 0:48:49 | 0:36:48 | 0:12:01 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7602377 |
|
2024-03-15 13:04:13 | 2024-03-16 10:19:04 | 2024-03-16 10:59:53 | 0:40:49 | 0:26:01 | 0:14:48 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602378 |
|
2024-03-15 13:04:14 | 2024-03-16 10:21:55 | 2024-03-16 10:52:08 | 0:30:13 | 0:19:08 | 0:11:05 | smithi | main | ubuntu | 22.04 | fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep} | 2 |
| pass | 7602379 |
|
2024-03-15 13:04:15 | 2024-03-16 10:21:56 | 2024-03-16 10:51:14 | 0:29:18 | 0:16:48 | 0:12:30 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/iozone}} | 2 |
| fail | 7602380 |
|
2024-03-15 13:04:16 | 2024-03-16 10:21:56 | 2024-03-16 11:10:03 | 0:48:07 | 0:35:11 | 0:12:56 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} | 3 |
Failure Reason:
Command failed (workunit test suites/fsx.sh) on smithi079 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
| pass | 7602381 |
|
2024-03-15 13:04:17 | 2024-03-16 10:24:37 | 2024-03-16 10:55:03 | 0:30:26 | 0:13:31 | 0:16:55 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/client-readahead} | 2 |
| pass | 7602382 |
|
2024-03-15 13:04:18 | 2024-03-16 10:30:29 | 2024-03-16 10:56:34 | 0:26:05 | 0:11:59 | 0:14:06 | smithi | main | centos | 8.stream | fs/upgrade/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} | 2 |
| fail | 7602383 |
|
2024-03-15 13:04:19 | 2024-03-16 10:34:51 | 2024-03-16 11:14:28 | 0:39:37 | 0:23:28 | 0:16:09 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} | 3 |
Failure Reason:
"2024-03-16T10:58:01.243454+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602384 |
|
2024-03-15 13:04:20 | 2024-03-16 10:36:47 | 2024-03-16 11:41:57 | 1:05:10 | 0:53:48 | 0:11:22 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/client-recovery} | 2 |
| pass | 7602385 |
|
2024-03-15 13:04:21 | 2024-03-16 10:38:35 | 2024-03-16 11:05:47 | 0:27:12 | 0:18:00 | 0:09:12 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} | 2 |
| fail | 7602386 |
|
2024-03-15 13:04:22 | 2024-03-16 10:38:36 | 2024-03-16 11:28:17 | 0:49:41 | 0:37:21 | 0:12:20 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} | 3 |
Failure Reason:
"2024-03-16T11:01:40.697366+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602387 |
|
2024-03-15 13:04:23 | 2024-03-16 10:41:03 | 2024-03-16 11:18:52 | 0:37:49 | 0:20:27 | 0:17:22 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7602388 |
|
2024-03-15 13:04:24 | 2024-03-16 10:47:36 | 2024-03-16 11:47:56 | 1:00:20 | 0:48:02 | 0:12:18 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} | 3 |
Failure Reason:
"2024-03-16T11:10:07.731862+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602389 |
|
2024-03-15 13:04:25 | 2024-03-16 10:47:48 | 2024-03-16 11:17:15 | 0:29:27 | 0:15:56 | 0:13:31 | smithi | main | centos | 9.stream | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} | 2 |
| pass | 7602390 |
|
2024-03-15 13:04:26 | 2024-03-16 10:49:02 | 2024-03-16 11:12:11 | 0:23:09 | 0:13:43 | 0:09:26 | smithi | main | centos | 9.stream | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} | 2 |
| pass | 7602391 |
|
2024-03-15 13:04:28 | 2024-03-16 10:49:02 | 2024-03-16 11:21:48 | 0:32:46 | 0:19:51 | 0:12:55 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/damage} | 2 |
| pass | 7602392 |
|
2024-03-15 13:04:29 | 2024-03-16 10:49:16 | 2024-03-16 11:15:48 | 0:26:32 | 0:16:04 | 0:10:28 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} | 2 |
| fail | 7602393 |
|
2024-03-15 13:04:30 | 2024-03-16 10:49:59 | 2024-03-16 11:39:01 | 0:49:02 | 0:35:58 | 0:13:04 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} | 3 |
Failure Reason:
"2024-03-16T11:12:48.436905+0000 mon.a (mon.0) 212 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602394 |
|
2024-03-15 13:04:31 | 2024-03-16 10:51:26 | 2024-03-16 11:30:46 | 0:39:20 | 0:24:00 | 0:15:20 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} | 3 |
Failure Reason:
error during scrub thrashing: rank damage found: {'backtrace'} |
||||||||||||||
| pass | 7602395 |
|
2024-03-15 13:04:32 | 2024-03-16 10:55:13 | 2024-03-16 11:51:33 | 0:56:20 | 0:43:25 | 0:12:55 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/data-scan} | 2 |
| fail | 7602396 |
|
2024-03-15 13:04:33 | 2024-03-16 10:56:31 | 2024-03-16 11:23:20 | 0:26:49 | 0:13:34 | 0:13:15 | smithi | main | ubuntu | 22.04 | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} | 2 |
| fail | 7602397 |
|
2024-03-15 13:04:34 | 2024-03-16 10:56:50 | 2024-03-16 11:44:05 | 0:47:15 | 0:37:54 | 0:09:21 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7602398 |
|
2024-03-15 13:04:35 | 2024-03-16 10:57:08 | 2024-03-16 12:43:03 | 1:45:55 | 1:36:13 | 0:09:42 | smithi | main | ubuntu | 22.04 | fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn} tasks/{0-client 1-tests/fscrypt-common}} | 3 |
Failure Reason:
Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt) |
||||||||||||||
| pass | 7602399 |
|
2024-03-15 13:04:36 | 2024-03-16 10:57:17 | 2024-03-16 11:21:58 | 0:24:41 | 0:13:17 | 0:11:24 | smithi | main | ubuntu | 22.04 | fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} | 5 |
| pass | 7602400 |
|
2024-03-15 13:04:37 | 2024-03-16 10:57:17 | 2024-03-16 11:29:28 | 0:32:11 | 0:21:05 | 0:11:06 | smithi | main | centos | 9.stream | fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} | 2 |
| pass | 7602401 |
|
2024-03-15 13:04:38 | 2024-03-16 10:58:28 | 2024-03-16 11:41:35 | 0:43:07 | 0:33:36 | 0:09:31 | smithi | main | centos | 9.stream | fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} | 2 |
| pass | 7602402 |
|
2024-03-15 13:04:39 | 2024-03-16 10:58:28 | 2024-03-16 12:59:03 | 2:00:35 | 1:50:13 | 0:10:22 | smithi | main | ubuntu | 22.04 | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} | 2 |
| fail | 7602403 |
|
2024-03-15 13:04:40 | 2024-03-16 10:59:19 | 2024-03-16 11:42:50 | 0:43:31 | 0:27:00 | 0:16:31 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} | 3 |
Failure Reason:
"2024-03-16T11:25:37.167221+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602404 |
|
2024-03-15 13:04:41 | 2024-03-16 11:05:50 | 2024-03-16 12:01:47 | 0:55:57 | 0:45:36 | 0:10:21 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/exports} | 2 |
| fail | 7602405 |
|
2024-03-15 13:04:42 | 2024-03-16 11:06:30 | 2024-03-16 13:44:01 | 2:37:31 | 2:20:40 | 0:16:51 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} | 3 |
Failure Reason:
"2024-03-16T11:32:43.155565+0000 mon.a (mon.0) 490 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| dead | 7602406 |
|
2024-03-15 13:04:43 | 2024-03-16 11:12:11 | 2024-03-16 11:17:04 | 0:04:53 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} | 2 | ||
Failure Reason:
Error reimaging machines: Expected smithi097's OS to be centos 9 but found centos 8 |
||||||||||||||
| fail | 7602407 |
|
2024-03-15 13:04:44 | 2024-03-16 11:12:12 | 2024-03-16 12:36:57 | 1:24:45 | 1:14:35 | 0:10:10 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/kernel_untar_build}} | 3 |
Failure Reason:
"2024-03-16T11:32:45.325908+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602408 |
|
2024-03-15 13:04:45 | 2024-03-16 11:12:12 | 2024-03-16 11:41:38 | 0:29:26 | 0:18:51 | 0:10:35 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} | 2 |
Failure Reason:
"2024-03-16T11:34:54.708032+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log |
||||||||||||||
| dead | 7602409 |
|
2024-03-15 13:04:47 | 2024-03-16 11:12:13 | 2024-03-16 11:18:36 | 0:06:23 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 | ||
Failure Reason:
Error reimaging machines: Failed to reconnect to smithi097 |
||||||||||||||
| fail | 7602410 |
|
2024-03-15 13:04:48 | 2024-03-16 11:12:13 | 2024-03-16 12:10:57 | 0:58:44 | 0:43:21 | 0:15:23 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/postgres}} | 3 |
Failure Reason:
"2024-03-16T11:36:41.841613+0000 mon.a (mon.0) 208 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602411 |
|
2024-03-15 13:04:49 | 2024-03-16 11:16:54 | 2024-03-16 12:09:27 | 0:52:33 | 0:41:45 | 0:10:48 | smithi | main | centos | 9.stream | fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} | 2 |
| dead | 7602412 |
|
2024-03-15 13:04:50 | 2024-03-16 11:17:25 | 2024-03-16 11:23:33 | 0:06:08 | smithi | main | ubuntu | 22.04 | fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} | 2 | ||
Failure Reason:
Error reimaging machines: Expected smithi097's OS to be ubuntu 22.04 but found centos 9 |
||||||||||||||
| fail | 7602413 |
|
2024-03-15 13:04:51 | 2024-03-16 11:17:25 | 2024-03-16 11:25:36 | 0:08:11 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} | 2 | ||
Failure Reason:
Failed to reconnect to smithi097 |
||||||||||||||
| fail | 7602414 |
|
2024-03-15 13:04:52 | 2024-03-16 11:18:46 | 2024-03-16 12:11:58 | 0:53:12 | 0:41:43 | 0:11:29 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} | 3 |
Failure Reason:
"2024-03-16T11:39:55.882670+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| fail | 7602415 |
|
2024-03-15 13:04:53 | 2024-03-16 11:18:56 | 2024-03-16 11:49:47 | 0:30:51 | 0:20:01 | 0:10:50 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/fragment} | 2 |
Failure Reason:
"2024-03-16T11:41:10.507001+0000 mon.a (mon.0) 687 : cluster [WRN] Health check failed: Degraded data redundancy: 18/4260 objects degraded (0.423%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7602416 |
|
2024-03-15 13:04:54 | 2024-03-16 11:18:56 | 2024-03-16 15:14:46 | 3:55:50 | 3:37:34 | 0:18:16 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
Command failed (workunit test suites/fsstress.sh) on smithi142 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh' |
||||||||||||||
| fail | 7602417 |
|
2024-03-15 13:04:55 | 2024-03-16 11:21:57 | 2024-03-16 12:25:35 | 1:03:38 | 0:52:18 | 0:11:20 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} | 3 |
Failure Reason:
Command failed (workunit test suites/dbench.sh) on smithi029 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' |
||||||||||||||
| pass | 7602418 |
|
2024-03-15 13:04:56 | 2024-03-16 11:22:08 | 2024-03-16 12:23:51 | 1:01:43 | 0:50:51 | 0:10:52 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} | 2 |
| dead | 7602419 |
|
2024-03-15 13:04:57 | 2024-03-16 11:22:08 | 2024-03-16 12:24:01 | 1:01:53 | smithi | main | centos | 9.stream | fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 | ||
Failure Reason:
Error reimaging machines: reached maximum tries (241) after waiting for 3600 seconds |
||||||||||||||
| pass | 7602420 |
|
2024-03-15 13:04:58 | 2024-03-16 11:23:49 | 2024-03-16 11:51:25 | 0:27:36 | 0:13:42 | 0:13:54 | smithi | main | ubuntu | 22.04 | fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} | 2 |
| pass | 7602421 |
|
2024-03-15 13:04:59 | 2024-03-16 11:27:40 | 2024-03-16 11:51:12 | 0:23:32 | 0:14:55 | 0:08:37 | smithi | main | ubuntu | 22.04 | fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} | 2 |
| pass | 7602422 |
|
2024-03-15 13:05:00 | 2024-03-16 11:27:40 | 2024-03-16 11:57:24 | 0:29:44 | 0:18:14 | 0:11:30 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/journal-repair} | 2 |
| fail | 7602423 |
|
2024-03-15 13:05:01 | 2024-03-16 11:27:41 | 2024-03-16 12:07:08 | 0:39:27 | 0:23:53 | 0:15:34 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} | 3 |
Failure Reason:
No module named 'tasks.fs' |
||||||||||||||
| fail | 7602424 |
|
2024-03-15 13:05:02 | 2024-03-16 11:33:52 | 2024-03-16 12:05:57 | 0:32:05 | 0:18:37 | 0:13:28 | smithi | main | centos | 9.stream | fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} | 2 |
Failure Reason:
"2024-03-16T12:03:54.780107+0000 mon.a (mon.0) 226 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7602425 |
|
2024-03-15 13:05:03 | 2024-03-16 11:36:13 | 2024-03-16 12:05:37 | 0:29:24 | 0:12:18 | 0:17:06 | smithi | main | ubuntu | 22.04 | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} | 2 |
Failure Reason:
No module named 'tasks.ceph_fuse' |
||||||||||||||
| fail | 7602426 |
|
2024-03-15 13:05:04 | 2024-03-16 11:41:44 | 2024-03-16 12:23:30 | 0:41:46 | 0:31:17 | 0:10:29 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} | 3 |
Failure Reason:
"2024-03-16T12:00:22.981495+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602427 |
|
2024-03-15 13:05:06 | 2024-03-16 11:42:04 | 2024-03-16 12:02:06 | 0:20:02 | 0:08:57 | 0:11:05 | smithi | main | centos | 9.stream | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/mds-flush} | 2 |
| pass | 7602428 |
|
2024-03-15 13:05:07 | 2024-03-16 11:42:55 | 2024-03-16 12:18:28 | 0:35:33 | 0:26:21 | 0:09:12 | smithi | main | centos | 8.stream | fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7602429 |
|
2024-03-15 13:05:08 | 2024-03-16 11:42:55 | 2024-03-16 12:25:18 | 0:42:23 | 0:30:39 | 0:11:44 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsstress}} | 3 |
Failure Reason:
"2024-03-16T12:04:39.395160+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602430 |
|
2024-03-15 13:05:09 | 2024-03-16 11:42:56 | 2024-03-16 12:11:48 | 0:28:52 | 0:19:23 | 0:09:29 | smithi | main | centos | 9.stream | fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} | 2 |
| pass | 7602431 |
|
2024-03-15 13:05:10 | 2024-03-16 11:42:56 | 2024-03-16 12:50:06 | 1:07:10 | 0:58:26 | 0:08:44 | smithi | main | centos | 9.stream | fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} | 2 |
| fail | 7602432 |
|
2024-03-15 13:05:11 | 2024-03-16 11:42:56 | 2024-03-16 12:23:52 | 0:40:56 | 0:31:36 | 0:09:20 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} | 3 |
Failure Reason:
Command failed (workunit test suites/fsx.sh) on smithi032 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' |
||||||||||||||
| fail | 7602433 |
|
2024-03-15 13:05:12 | 2024-03-16 11:42:57 | 2024-03-16 12:04:40 | 0:21:43 | 0:10:34 | 0:11:09 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/mds-full} | 2 |
Failure Reason:
No module named 'tasks.ceph' |
||||||||||||||
| fail | 7602434 |
|
2024-03-15 13:05:13 | 2024-03-16 11:42:57 | 2024-03-16 12:28:54 | 0:45:57 | 0:27:52 | 0:18:05 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} | 3 |
Failure Reason:
"2024-03-16T12:11:50.277153+0000 mon.a (mon.0) 209 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log |
||||||||||||||
| pass | 7602435 |
|
2024-03-15 13:05:14 | 2024-03-16 11:51:29 | 2024-03-16 12:16:25 | 0:24:56 | 0:13:05 | 0:11:51 | smithi | main | ubuntu | 22.04 | fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/mds_creation_retry} | 2 |
| fail | 7602436 |
|
2024-03-15 13:05:15 | 2024-03-16 11:51:39 | 2024-03-16 12:15:17 | 0:23:38 | 0:06:26 | 0:17:12 | smithi | main | centos | 9.stream | fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} | 2 |
Failure Reason:
No module named 'tasks' |
||||||||||||||
| fail | 7602437 |
|
2024-03-15 13:05:16 | 2024-03-16 11:57:30 | 2024-03-16 12:20:54 | 0:23:24 | 0:06:00 | 0:17:24 | smithi | main | centos | 9.stream | fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} | 3 |
Failure Reason:
No module named 'tasks' |
||||||||||||||
| fail | 7602438 |
|
2024-03-15 13:05:17 | 2024-03-16 11:58:11 | 2024-03-16 12:19:02 | 0:20:51 | 0:08:30 | 0:12:21 | smithi | main | centos | 9.stream | fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
No module named 'tasks' |
||||||||||||||
| queued | 2024-03-15 13:00:56 | 2024-03-15 13:00:56 | — | |||||||||||
Pulpito