Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 7879998 2024-08-29 14:16:25 2024-08-29 15:33:15 2024-08-29 15:54:14 0:20:59 0:14:41 0:06:18 smithi main centos 9.stream upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
fail 7879999 2024-08-29 14:16:26 2024-08-29 15:33:15 2024-08-29 18:58:19 3:25:04 3:15:06 0:09:58 smithi main ubuntu 22.04 upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T15:55:59.096556+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7880000 2024-08-29 14:16:27 2024-08-29 15:34:05 2024-08-29 16:29:16 0:55:11 0:49:05 0:06:06 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7880001 2024-08-29 14:16:29 2024-08-29 15:34:16 2024-08-29 18:44:40 3:10:24 2:53:37 0:16:47 smithi main ubuntu 22.04 upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724947923.859747 mon.a (mon.0) 694 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880002 2024-08-29 14:16:30 2024-08-29 15:40:07 2024-08-29 19:38:02 3:57:55 3:48:35 0:09:20 smithi main ubuntu 22.04 upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T16:11:26.263759+0000 mon.a (mon.0) 698 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880003 2024-08-29 14:16:31 2024-08-29 15:40:17 2024-08-29 16:09:56 0:29:39 0:23:15 0:06:24 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/yes}} 3
fail 7880004 2024-08-29 14:16:33 2024-08-29 15:40:18 2024-08-29 19:17:11 3:36:53 3:25:57 0:10:56 smithi main ubuntu 22.04 upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724947963.6360176 mon.a (mon.0) 721 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880005 2024-08-29 14:16:34 2024-08-29 15:40:58 2024-08-29 16:17:49 0:36:51 0:29:44 0:07:07 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7880006 2024-08-29 14:16:35 2024-08-29 15:40:59 2024-08-29 18:27:49 2:46:50 2:39:22 0:07:28 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

Command failed (workunit test rbd/test_librbd.sh) on smithi162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh'

fail 7880007 2024-08-29 14:16:37 2024-08-29 15:40:59 2024-08-29 18:17:50 2:36:51 2:29:28 0:07:23 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724948134.4049656 mon.a (mon.0) 504 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880008 2024-08-29 14:16:38 2024-08-29 15:42:30 2024-08-29 16:39:19 0:56:49 0:46:55 0:09:54 smithi main ubuntu 22.04 upgrade/telemetry-upgrade/reef-x/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks} 2
Failure Reason:

"2024-08-29T16:30:00.000256+0000 mon.a (mon.0) 519 : cluster [WRN] pg 2.b is active+undersized+degraded, acting [7,4]" in cluster log

pass 7880009 2024-08-29 14:16:39 2024-08-29 15:43:20 2024-08-29 16:24:38 0:41:18 0:35:08 0:06:10 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/no}} 3
fail 7880010 2024-08-29 14:16:40 2024-08-29 15:43:21 2024-08-29 16:03:13 0:19:52 0:10:41 0:09:11 smithi main ubuntu 22.04 upgrade/quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_latest} 1
Failure Reason:

Command failed on smithi057 with status 1: "sudo egrep 'FileStore has been deprecated and is no longer supported' /var/log/ceph/ceph-osd.1.log"

fail 7880011 2024-08-29 14:16:42 2024-08-29 15:43:21 2024-08-29 18:13:50 2:30:29 2:19:04 0:11:25 smithi main ubuntu 22.04 upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-08-29T16:14:12.465087+0000 mon.a (mon.0) 677 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880012 2024-08-29 14:16:43 2024-08-29 15:44:11 2024-08-29 16:16:57 0:32:46 0:27:24 0:05:22 smithi main centos 9.stream upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} 1
fail 7880013 2024-08-29 14:16:45 2024-08-29 15:44:32 2024-08-29 17:06:17 1:21:45 1:13:34 0:08:11 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7880014 2024-08-29 14:16:46 2024-08-29 15:45:02 2024-08-29 17:53:06 2:08:04 1:55:22 0:12:42 smithi main centos 9.stream upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"1724948968.7304428 mon.a (mon.0) 646 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880015 2024-08-29 14:16:47 2024-08-29 15:45:43 2024-08-29 19:39:44 3:54:01 3:32:52 0:21:09 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T16:31:06.949599+0000 mon.a (mon.0) 668 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880016 2024-08-29 14:16:48 2024-08-29 15:45:43 2024-08-29 16:45:07 0:59:24 0:49:37 0:09:47 smithi main centos 9.stream upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/kernel_untar_build}} 2
pass 7880017 2024-08-29 14:16:50 2024-08-29 15:45:53 2024-08-29 16:31:00 0:45:07 0:24:04 0:21:03 smithi main centos 9.stream upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
fail 7880018 2024-08-29 14:16:51 2024-08-29 15:47:54 2024-08-29 19:29:24 3:41:30 3:18:42 0:22:48 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724948658.5789602 mon.a (mon.0) 368 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880019 2024-08-29 14:16:52 2024-08-29 15:48:05 2024-08-29 19:35:47 3:47:42 3:35:35 0:12:07 smithi main ubuntu 22.04 upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T16:11:44.213768+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

dead 7880020 2024-08-29 14:16:54 2024-08-29 15:49:25 2024-08-30 00:03:22 8:13:57 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

hit max job timeout

fail 7880021 2024-08-29 14:16:55 2024-08-29 15:53:46 2024-08-29 18:42:15 2:48:29 2:38:55 0:09:34 smithi main ubuntu 22.04 upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724948498.9735422 mon.a (mon.0) 511 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7880022 2024-08-29 14:16:56 2024-08-29 15:54:27 2024-08-29 16:33:23 0:38:56 0:19:31 0:19:25 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/yes}} 3
fail 7880023 2024-08-29 14:16:58 2024-08-29 15:56:22 2024-08-29 19:16:28 3:20:06 3:02:49 0:17:17 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T16:31:07.385227+0000 mon.a (mon.0) 670 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880024 2024-08-29 14:16:59 2024-08-29 15:57:33 2024-08-29 19:07:22 3:09:49 2:54:57 0:14:52 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724949001.1463401 mon.a (mon.0) 518 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880025 2024-08-29 14:17:00 2024-08-29 15:59:54 2024-08-29 16:59:10 0:59:16 0:43:40 0:15:36 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7880026 2024-08-29 14:17:02 2024-08-29 16:01:45 2024-08-29 18:29:27 2:27:42 2:17:23 0:10:19 smithi main ubuntu 22.04 upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-08-29T16:29:46.930773+0000 mon.a (mon.0) 578 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 7880028 2024-08-29 14:17:03 2024-08-29 16:01:46 2024-08-29 17:52:35 1:50:49 1:39:31 0:11:18 smithi main centos 9.stream upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"1724948960.3842163 mon.a (mon.0) 653 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880030 2024-08-29 14:17:04 2024-08-29 16:03:27 2024-08-29 16:36:39 0:33:12 0:21:02 0:12:10 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/no}} 3
pass 7880032 2024-08-29 14:17:06 2024-08-29 16:06:59 2024-08-29 16:28:57 0:21:58 0:11:56 0:10:02 smithi main centos 9.stream upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/reef 1-upgrade}} 1
fail 7880034 2024-08-29 14:17:07 2024-08-29 16:07:40 2024-08-29 19:33:13 3:25:33 3:15:56 0:09:37 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T16:33:52.923133+0000 mon.a (mon.0) 699 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880036 2024-08-29 14:17:08 2024-08-29 16:09:21 2024-08-29 19:08:03 2:58:42 2:47:00 0:11:42 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724949686.4335005 mon.a (mon.0) 661 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880038 2024-08-29 14:17:10 2024-08-29 16:10:12 2024-08-29 16:54:14 0:44:02 0:35:44 0:08:18 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7880040 2024-08-29 14:17:11 2024-08-29 16:17:15 2024-08-29 19:56:42 3:39:27 3:29:04 0:10:23 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T16:40:20.610984+0000 mon.a (mon.0) 695 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880042 2024-08-29 14:17:12 2024-08-29 16:18:06 2024-08-29 16:46:31 0:28:25 0:17:56 0:10:29 smithi main centos 9.stream upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
fail 7880045 2024-08-29 14:17:14 2024-08-29 16:24:49 2024-08-29 19:41:48 3:16:59 3:07:13 0:09:46 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724949825.0498574 mon.a (mon.0) 695 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880047 2024-08-29 14:17:15 2024-08-29 16:28:51 2024-08-29 16:46:25 0:17:34 0:10:04 0:07:30 smithi main centos 9.stream upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/iozone}} 2
fail 7880049 2024-08-29 14:17:16 2024-08-29 16:29:33 2024-08-29 19:47:25 3:17:52 3:08:48 0:09:04 smithi main ubuntu 22.04 upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T16:51:20.179488+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

fail 7880051 2024-08-29 14:17:18 2024-08-29 16:30:34 2024-08-29 17:14:21 0:43:47 0:35:17 0:08:30 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7880053 2024-08-29 14:17:19 2024-08-29 16:31:55 2024-08-29 19:13:16 2:41:21 2:31:00 0:10:21 smithi main ubuntu 22.04 upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724950800.899855 mon.a (mon.0) 511 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7880055 2024-08-29 14:17:20 2024-08-29 16:53:12 710 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/yes}} 3
fail 7880057 2024-08-29 14:17:22 2024-08-29 16:34:18 2024-08-29 19:42:33 3:08:15 3:01:02 0:07:13 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T16:49:38.171123+0000 mon.a (mon.0) 510 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880059 2024-08-29 14:17:23 2024-08-29 16:35:29 2024-08-29 19:18:46 2:43:17 2:36:02 0:07:15 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724950206.4964392 mon.a (mon.0) 516 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880061 2024-08-29 14:17:24 2024-08-29 16:37:00 2024-08-29 17:11:52 0:34:52 0:27:36 0:07:16 smithi main centos 9.stream upgrade/telemetry-upgrade/quincy-x/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks} 2
Failure Reason:

"2024-08-29T17:06:00.917069+0000 mon.a (mon.0) 568 : cluster [WRN] Health check failed: Telemetry requires re-opt-in (TELEMETRY_CHANGED)" in cluster log

pass 7880063 2024-08-29 14:17:26 2024-08-29 16:37:01 2024-08-29 17:02:56 0:25:55 0:16:22 0:09:33 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/no}} 3
fail 7880065 2024-08-29 14:17:27 2024-08-29 16:39:33 2024-08-29 16:59:18 0:19:45 0:10:11 0:09:34 smithi main ubuntu 22.04 upgrade/quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_latest} 1
Failure Reason:

"2024-08-29T16:58:13.204859+0000 mon.a (mon.0) 38 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7880067 2024-08-29 14:17:28 2024-08-29 16:39:53 2024-08-29 18:38:02 1:58:09 1:51:07 0:07:02 smithi main centos 9.stream upgrade/reef-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-08-29T16:56:01.782815+0000 mon.a (mon.0) 664 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880069 2024-08-29 14:17:30 2024-08-29 16:41:05 2024-08-29 17:24:58 0:43:53 0:35:07 0:08:46 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 7880071 2024-08-29 14:17:31 2024-08-29 16:45:16 2024-08-29 16:57:59 0:12:43 0:06:06 0:06:37 smithi main centos 9.stream upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} 1
fail 7880073 2024-08-29 14:17:32 2024-08-29 16:46:07 2024-08-29 18:25:39 1:39:32 1:31:38 0:07:54 smithi main centos 9.stream upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"1724950817.1766622 mon.a (mon.0) 515 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880075 2024-08-29 14:17:34 2024-08-29 16:49:39 2024-08-29 20:12:36 3:22:57 3:12:38 0:10:19 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T17:09:17.728938+0000 mon.a (mon.0) 668 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7880077 2024-08-29 14:17:35 2024-08-29 16:53:32 2024-08-29 17:25:57 0:32:25 0:20:06 0:12:19 smithi main centos 9.stream upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/blogbench}} 2
pass 7880079 2024-08-29 14:17:36 2024-08-29 16:54:33 2024-08-29 17:17:39 0:23:06 0:15:17 0:07:49 smithi main centos 9.stream upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
fail 7880081 2024-08-29 14:17:38 2024-08-29 16:56:35 2024-08-29 19:48:28 2:51:53 2:44:00 0:07:53 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724951505.7766347 mon.a (mon.0) 518 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7880083 2024-08-29 14:17:39 2024-08-29 16:57:16 2024-08-29 17:36:17 0:39:01 0:30:32 0:08:29 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
fail 7880085 2024-08-29 14:17:41 2024-08-29 16:59:27 2024-08-29 20:37:15 3:37:48 3:28:08 0:09:40 smithi main ubuntu 22.04 upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"2024-08-29T17:23:39.335715+0000 mon.a (mon.0) 375 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880087 2024-08-29 14:17:42 2024-08-29 17:00:28 2024-08-29 17:31:57 0:31:29 0:24:39 0:06:50 smithi main centos 9.stream upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/dbench}} 2
Failure Reason:

Command failed (workunit test suites/dbench.sh) on smithi164 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1ef864504b8875c83ee6c2c5fedc13315bebf7f5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'

fail 7880089 2024-08-29 14:17:43 2024-08-29 17:01:39 2024-08-29 20:09:14 3:07:35 2:56:12 0:11:23 smithi main ubuntu 22.04 upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} 2
Failure Reason:

"1724952666.6735291 mon.a (mon.0) 511 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

pass 7880091 2024-08-29 14:17:45 2024-08-29 17:03:41 2024-08-29 17:26:21 0:22:40 0:13:34 0:09:06 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/yes}} 3
fail 7880093 2024-08-29 14:17:46 2024-08-29 17:05:02 2024-08-29 19:56:42 2:51:40 2:44:23 0:07:17 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"2024-08-29T17:21:10.876142+0000 mon.a (mon.0) 665 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880095 2024-08-29 14:17:47 2024-08-29 17:05:33 2024-08-29 19:39:49 2:34:16 2:25:38 0:08:38 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} 2
Failure Reason:

"1724952140.9049945 mon.a (mon.0) 667 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880097 2024-08-29 14:17:49 2024-08-29 17:06:54 2024-08-29 17:50:10 0:43:16 0:35:31 0:07:45 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

fail 7880099 2024-08-29 14:17:50 2024-08-29 17:10:06 2024-08-29 19:41:20 2:31:14 2:20:13 0:11:01 smithi main ubuntu 22.04 upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-08-29T17:43:12.124731+0000 mon.a (mon.0) 684 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880101 2024-08-29 14:17:52 2024-08-29 17:10:27 2024-08-29 19:09:01 1:58:34 1:47:17 0:11:17 smithi main ubuntu 22.04 upgrade/quincy-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"1724952754.0433066 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log

pass 7880103 2024-08-29 14:17:53 2024-08-29 17:12:08 2024-08-29 17:35:48 0:23:40 0:16:46 0:06:54 smithi main centos 9.stream upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/no}} 3
pass 7880105 2024-08-29 14:17:54 2024-08-29 17:14:39 2024-08-29 17:27:39 0:13:00 0:06:01 0:06:59 smithi main centos 9.stream upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/reef 1-upgrade}} 1
fail 7880107 2024-08-29 14:17:56 2024-08-29 17:15:00 2024-08-29 20:46:13 3:31:13 3:25:15 0:05:58 smithi main centos 9.stream upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

"2024-08-29T17:30:56.209227+0000 mon.a (mon.0) 678 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7880109 2024-08-29 14:17:57 2024-08-29 17:15:01 2024-08-29 20:27:06 3:12:05 3:04:16 0:07:49 smithi main centos 9.stream upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} 2
Failure Reason:

"1724952647.2415454 mon.a (mon.0) 507 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7880111 2024-08-29 14:17:58 2024-08-29 17:16:53 2024-08-29 17:54:03 0:37:10 0:29:12 0:07:58 smithi main centos 9.stream upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

"2024-08-29T17:40:00.000190+0000 mon.smithi037 (mon.0) 276 : cluster [WRN] osd.0 (root=default,host=smithi037) is down" in cluster log