| User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail |
|---|---|---|---|---|---|---|---|---|---|---|
| sangadi | 2024-09-09 06:29:19 | 2024-09-12 13:22:07 | 2024-09-12 19:04:28 | 5:42:21 | upgrade | squid-release | smithi | f03a108 | 44 | 194 |
| Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| pass | 7895963 |
|
2024-09-09 06:29:33 | 2024-09-11 15:43:45 | 2024-09-11 16:12:59 | 0:29:14 | 0:19:42 | 0:09:32 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7895964 |
|
2024-09-09 06:29:34 | 2024-09-11 15:43:45 | 2024-09-11 16:03:19 | 0:19:34 | 0:10:16 | 0:09:18 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_latest} | 1 |
Failure Reason:
"2024-09-11T16:02:21.668382+0000 mon.a (mon.0) 38 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7895965 |
|
2024-09-09 06:29:35 | 2024-09-11 15:43:45 | 2024-09-11 17:48:59 | 2:05:14 | 1:54:20 | 0:10:54 | smithi | main | centos | 9.stream | upgrade/reef-x/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-09-11T16:04:25.859919+0000 mon.a (mon.0) 662 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7895966 |
|
2024-09-09 06:29:37 | 2024-09-11 15:43:46 | 2024-09-11 16:24:16 | 0:40:30 | 0:28:50 | 0:11:40 | smithi | main | centos | 9.stream | upgrade/telemetry-upgrade/quincy-x/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks} | 2 |
| fail | 7895967 |
|
2024-09-09 06:29:38 | 2024-09-11 15:44:56 | 2024-09-11 16:30:28 | 0:45:32 | 0:35:23 | 0:10:09 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7895968 |
|
2024-09-09 06:29:39 | 2024-09-11 15:45:37 | 2024-09-11 17:42:25 | 1:56:48 | 1:45:59 | 0:10:49 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1726070864.1138535 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895969 |
|
2024-09-09 06:29:41 | 2024-09-11 15:46:38 | 2024-09-11 19:25:18 | 3:38:40 | 3:27:26 | 0:11:14 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:06:37.633625+0000 mon.a (mon.0) 372 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7895970 |
|
2024-09-09 06:29:42 | 2024-09-11 15:47:38 | 2024-09-11 16:06:55 | 0:19:17 | 0:08:21 | 0:10:56 | smithi | main | centos | 9.stream | upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} | 1 |
| pass | 7895971 |
|
2024-09-09 06:29:44 | 2024-09-11 15:48:09 | 2024-09-11 16:07:16 | 0:19:07 | 0:08:33 | 0:10:34 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} | 2 |
| fail | 7895972 |
|
2024-09-09 06:29:45 | 2024-09-11 15:48:19 | 2024-09-11 19:05:45 | 3:17:26 | 3:05:34 | 0:11:52 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726071037.9089353 mon.a (mon.0) 516 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7895973 |
|
2024-09-09 06:29:46 | 2024-09-11 15:50:50 | 2024-09-11 18:40:11 | 2:49:21 | 2:39:24 | 0:09:57 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:12:21.336635+0000 mon.a (mon.0) 690 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7895974 |
|
2024-09-09 06:29:48 | 2024-09-11 15:50:51 | 2024-09-11 16:36:11 | 0:45:20 | 0:32:03 | 0:13:17 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7895975 |
|
2024-09-09 06:29:49 | 2024-09-11 15:55:22 | 2024-09-11 18:40:31 | 2:45:09 | 2:36:01 | 0:09:08 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726071339.017343 mon.a (mon.0) 544 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7895976 |
|
2024-09-09 06:29:50 | 2024-09-11 15:55:32 | 2024-09-11 19:39:24 | 3:43:52 | 3:33:05 | 0:10:47 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:18:02.191162+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895977 |
|
2024-09-09 06:29:52 | 2024-09-11 15:55:33 | 2024-09-11 16:42:50 | 0:47:17 | 0:36:52 | 0:10:25 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7895978 |
|
2024-09-09 06:29:53 | 2024-09-11 15:55:33 | 2024-09-11 18:54:10 | 2:58:37 | 2:46:01 | 0:12:36 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726071624.7137048 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895979 |
|
2024-09-09 06:29:54 | 2024-09-11 15:58:04 | 2024-09-11 19:36:08 | 3:38:04 | 3:27:39 | 0:10:25 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:20:14.249317+0000 mon.a (mon.0) 689 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7895980 |
|
2024-09-09 06:29:56 | 2024-09-11 15:58:04 | 2024-09-11 16:41:52 | 0:43:48 | 0:32:03 | 0:11:45 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7895981 |
|
2024-09-09 06:29:57 | 2024-09-11 15:58:45 | 2024-09-11 18:52:12 | 2:53:27 | 2:44:32 | 0:08:55 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726071360.6181238 mon.a (mon.0) 172 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895982 |
|
2024-09-09 06:29:58 | 2024-09-11 15:58:55 | 2024-09-11 19:21:23 | 3:22:28 | 3:11:00 | 0:11:28 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:20:09.737878+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7895983 |
|
2024-09-09 06:30:00 | 2024-09-11 15:59:26 | 2024-09-11 16:45:48 | 0:46:22 | 0:36:11 | 0:10:11 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7895984 |
|
2024-09-09 06:30:01 | 2024-09-11 16:00:56 | 2024-09-11 18:56:00 | 2:55:04 | 2:45:40 | 0:09:24 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726071476.672924 mon.a (mon.0) 168 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895985 |
|
2024-09-09 06:30:02 | 2024-09-11 16:00:57 | 2024-09-11 19:30:03 | 3:29:06 | 3:17:58 | 0:11:08 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:24:14.917449+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7895986 |
|
2024-09-09 06:30:04 | 2024-09-11 16:02:07 | 2024-09-11 16:41:09 | 0:39:02 | 0:30:25 | 0:08:37 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/kernel_untar_build}} | 2 |
| pass | 7895987 |
|
2024-09-09 06:30:05 | 2024-09-11 16:02:08 | 2024-09-11 16:45:40 | 0:43:32 | 0:32:26 | 0:11:06 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7895988 |
|
2024-09-09 06:30:06 | 2024-09-11 16:02:49 | 2024-09-11 18:56:37 | 2:53:48 | 2:44:04 | 0:09:44 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726071848.901941 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895989 |
|
2024-09-09 06:30:08 | 2024-09-11 16:02:50 | 2024-09-11 20:07:12 | 4:04:22 | 3:54:24 | 0:09:58 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:26:27.991470+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895990 |
|
2024-09-09 06:30:09 | 2024-09-11 16:03:20 | 2024-09-11 16:51:45 | 0:48:25 | 0:36:16 | 0:12:09 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7895991 |
|
2024-09-09 06:30:10 | 2024-09-11 16:04:21 | 2024-09-11 19:30:08 | 3:25:47 | 3:16:38 | 0:09:09 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726071983.7037127 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7895992 |
|
2024-09-09 06:30:12 | 2024-09-11 16:04:21 | 2024-09-11 18:54:05 | 2:49:44 | 2:39:50 | 0:09:54 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:25:39.834213+0000 mon.a (mon.0) 692 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7895993 |
|
2024-09-09 06:30:13 | 2024-09-11 16:04:52 | 2024-09-11 16:46:13 | 0:41:21 | 0:32:25 | 0:08:56 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7895994 |
|
2024-09-09 06:30:14 | 2024-09-11 16:05:02 | 2024-09-11 18:48:13 | 2:43:11 | 2:31:31 | 0:11:40 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726072165.5090165 mon.a (mon.0) 547 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7895995 |
|
2024-09-09 06:30:16 | 2024-09-11 16:07:03 | 2024-09-11 19:20:08 | 3:13:05 | 3:04:19 | 0:08:46 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:28:20.130951+0000 mon.a (mon.0) 674 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7895996 |
|
2024-09-09 06:30:17 | 2024-09-11 16:07:03 | 2024-09-11 16:48:19 | 0:41:16 | 0:31:45 | 0:09:31 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T16:40:00.000117+0000 mon.smithi115 (mon.0) 673 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is degraded" in cluster log |
||||||||||||||
| fail | 7895997 |
|
2024-09-09 06:30:19 | 2024-09-11 16:07:04 | 2024-09-11 18:52:36 | 2:45:32 | 2:36:15 | 0:09:17 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726072132.7627108 mon.a (mon.0) 501 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7895998 |
|
2024-09-09 06:30:20 | 2024-09-11 16:07:14 | 2024-09-11 19:57:16 | 3:50:02 | 3:39:27 | 0:10:35 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:28:51.234505+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7895999 |
|
2024-09-09 06:30:21 | 2024-09-11 16:07:15 | 2024-09-11 16:30:15 | 0:23:00 | 0:12:34 | 0:10:26 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/iozone}} | 2 |
| fail | 7896000 |
|
2024-09-09 06:30:23 | 2024-09-11 16:07:15 | 2024-09-11 19:03:14 | 2:55:59 | 2:46:17 | 0:09:42 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726072143.7984383 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896001 |
|
2024-09-09 06:30:24 | 2024-09-11 16:07:26 | 2024-09-11 19:36:21 | 3:28:55 | 3:12:05 | 0:16:50 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:34:16.757496+0000 mon.a (mon.0) 527 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896002 |
|
2024-09-09 06:30:25 | 2024-09-11 16:12:27 | 2024-09-11 17:30:31 | 1:18:04 | 1:07:02 | 0:11:02 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896003 |
|
2024-09-09 06:30:27 | 2024-09-11 16:13:17 | 2024-09-11 19:15:24 | 3:02:07 | 2:51:18 | 0:10:49 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726072472.4248455 mon.a (mon.0) 503 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896004 |
|
2024-09-09 06:30:28 | 2024-09-11 16:13:48 | 2024-09-11 19:17:17 | 3:03:29 | 2:52:33 | 0:10:56 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:34:30.580741+0000 mon.a (mon.0) 528 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896005 |
|
2024-09-09 06:30:29 | 2024-09-11 16:13:48 | 2024-09-11 16:55:40 | 0:41:52 | 0:30:15 | 0:11:37 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896006 |
|
2024-09-09 06:30:31 | 2024-09-11 16:14:38 | 2024-09-11 19:00:06 | 2:45:28 | 2:35:03 | 0:10:25 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726072334.667959 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896007 |
|
2024-09-09 06:30:32 | 2024-09-11 16:14:49 | 2024-09-11 16:44:32 | 0:29:43 | 0:17:31 | 0:12:12 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7896008 |
|
2024-09-09 06:30:33 | 2024-09-11 16:17:20 | 2024-09-11 20:06:43 | 3:49:23 | 3:38:30 | 0:10:53 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:39:43.014883+0000 mon.a (mon.0) 671 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896009 |
|
2024-09-09 06:30:34 | 2024-09-11 16:17:20 | 2024-09-11 19:34:58 | 3:17:38 | 3:06:58 | 0:10:40 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726072533.6754222 mon.a (mon.0) 168 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896010 |
|
2024-09-09 06:30:36 | 2024-09-11 16:17:20 | 2024-09-11 16:46:45 | 0:29:25 | 0:19:14 | 0:10:11 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T16:40:00.000164+0000 mon.smithi026 (mon.0) 662 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log |
||||||||||||||
| fail | 7896011 |
|
2024-09-09 06:30:37 | 2024-09-11 16:17:31 | 2024-09-11 19:31:29 | 3:13:58 | 2:58:27 | 0:15:31 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:45:23.636005+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896012 |
|
2024-09-09 06:30:38 | 2024-09-11 16:23:42 | 2024-09-11 19:18:39 | 2:54:57 | 2:42:58 | 0:11:59 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726073220.6393032 mon.a (mon.0) 180 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896013 |
|
2024-09-09 06:30:39 | 2024-09-11 16:24:33 | 2024-09-11 17:07:14 | 0:42:41 | 0:30:16 | 0:12:25 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"1726073400.0001488 mon.smithi130 (mon.0) 694 : cluster [WRN] Health detail: HEALTH_WARN insufficient standby MDS daemons available" in cluster log |
||||||||||||||
| fail | 7896014 |
|
2024-09-09 06:30:41 | 2024-09-11 16:28:24 | 2024-09-11 19:37:43 | 3:09:19 | 2:57:07 | 0:12:12 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T16:50:51.606095+0000 mon.a (mon.0) 672 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896015 |
|
2024-09-09 06:30:42 | 2024-09-11 16:30:24 | 2024-09-11 17:31:42 | 1:01:18 | 0:50:54 | 0:10:24 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/dbench}} | 2 |
| fail | 7896016 |
|
2024-09-09 06:30:43 | 2024-09-11 16:30:45 | 2024-09-11 19:30:13 | 2:59:28 | 2:48:44 | 0:10:44 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726073349.5183196 mon.a (mon.0) 354 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896017 |
|
2024-09-09 06:30:45 | 2024-09-11 16:30:55 | 2024-09-11 16:59:53 | 0:28:58 | 0:19:31 | 0:09:27 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7896018 |
|
2024-09-09 06:30:46 | 2024-09-11 16:31:06 | 2024-09-11 20:11:48 | 3:40:42 | 3:26:53 | 0:13:49 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T16:55:59.053459+0000 mon.a (mon.0) 524 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896019 |
|
2024-09-09 06:30:47 | 2024-09-11 16:36:27 | 2024-09-11 19:42:46 | 3:06:19 | 2:51:30 | 0:14:49 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726074039.8649666 mon.a (mon.0) 500 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896020 |
|
2024-09-09 06:30:49 | 2024-09-11 16:41:28 | 2024-09-11 17:23:09 | 0:41:41 | 0:31:19 | 0:10:22 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T17:10:00.000136+0000 mon.smithi136 (mon.0) 257 : cluster [WRN] osd.0 (root=default,host=smithi136) is down" in cluster log |
||||||||||||||
| fail | 7896021 |
|
2024-09-09 06:30:50 | 2024-09-11 16:42:08 | 2024-09-11 20:21:06 | 3:38:58 | 3:28:32 | 0:10:26 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T17:05:21.645894+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896022 |
|
2024-09-09 06:30:51 | 2024-09-11 16:43:09 | 2024-09-11 19:48:56 | 3:05:47 | 2:53:48 | 0:11:59 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726074447.0091257 mon.a (mon.0) 178 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896023 |
|
2024-09-09 06:30:53 | 2024-09-11 16:44:49 | 2024-09-11 17:29:50 | 0:45:01 | 0:36:06 | 0:08:55 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896024 |
|
2024-09-09 06:30:54 | 2024-09-11 16:44:50 | 2024-09-11 19:54:50 | 3:10:00 | 2:58:55 | 0:11:05 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T17:06:52.005261+0000 mon.a (mon.0) 685 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896025 |
|
2024-09-09 06:30:56 | 2024-09-11 16:45:50 | 2024-09-11 19:29:07 | 2:43:17 | 2:33:10 | 0:10:07 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726074166.4045138 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896026 |
|
2024-09-09 06:30:57 | 2024-09-11 16:46:01 | 2024-09-11 20:15:54 | 3:29:53 | 3:19:01 | 0:10:52 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T17:07:37.325808+0000 mon.a (mon.0) 674 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896027 |
|
2024-09-09 06:30:58 | 2024-09-11 16:46:31 | 2024-09-11 17:34:13 | 0:47:42 | 0:37:15 | 0:10:27 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896028 |
|
2024-09-09 06:31:00 | 2024-09-11 16:47:02 | 2024-09-11 20:19:59 | 3:32:57 | 3:22:01 | 0:10:56 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726074341.0211823 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896029 |
|
2024-09-09 06:31:01 | 2024-09-11 16:48:32 | 2024-09-11 19:53:30 | 3:04:58 | 2:50:01 | 0:14:57 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T17:14:08.309095+0000 mon.a (mon.0) 648 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896030 |
|
2024-09-09 06:31:02 | 2024-09-11 16:52:03 | 2024-09-11 17:36:54 | 0:44:51 | 0:31:19 | 0:13:32 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T17:30:00.000253+0000 mon.smithi050 (mon.0) 697 : cluster [ERR] fs cephfs is degraded" in cluster log |
||||||||||||||
| fail | 7896031 |
|
2024-09-09 06:31:04 | 2024-09-11 16:55:54 | 2024-09-11 19:33:29 | 2:37:35 | 2:22:16 | 0:15:19 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726075219.7534387 mon.a (mon.0) 510 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896032 |
|
2024-09-09 06:31:05 | 2024-09-11 17:00:05 | 2024-09-11 20:45:23 | 3:45:18 | 3:28:08 | 0:17:10 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T17:38:24.975767+0000 mon.a (mon.0) 703 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896033 |
|
2024-09-09 06:31:06 | 2024-09-11 17:07:27 | 2024-09-11 18:10:11 | 1:02:44 | 0:36:48 | 0:25:56 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896034 |
|
2024-09-09 06:31:08 | 2024-09-11 17:23:19 | 2024-09-11 20:33:45 | 3:10:26 | 2:55:19 | 0:15:07 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726077418.1580656 mon.a (mon.0) 506 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896035 |
|
2024-09-09 06:31:09 | 2024-09-11 17:30:00 | 2024-09-11 20:42:09 | 3:12:09 | 3:01:17 | 0:10:52 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
Command failed (workunit test rbd/test_librbd.sh) on smithi087 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' |
||||||||||||||
| pass | 7896036 |
|
2024-09-09 06:31:11 | 2024-09-11 17:30:41 | 2024-09-11 17:54:55 | 0:24:14 | 0:13:16 | 0:10:58 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/fsstress}} | 2 |
| pass | 7896037 |
|
2024-09-09 06:31:12 | 2024-09-11 17:31:51 | 2024-09-11 18:15:09 | 0:43:18 | 0:30:57 | 0:12:21 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896038 |
|
2024-09-09 06:31:13 | 2024-09-11 17:34:22 | 2024-09-11 20:40:10 | 3:05:48 | 2:54:10 | 0:11:38 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726077442.7074978 mon.a (mon.0) 558 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896039 |
|
2024-09-09 06:31:15 | 2024-09-11 17:36:43 | 2024-09-11 20:45:40 | 3:08:57 | 2:57:32 | 0:11:25 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T17:59:55.239391+0000 mon.a (mon.0) 660 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896040 |
|
2024-09-09 06:31:16 | 2024-09-11 17:37:03 | 2024-09-11 18:30:23 | 0:53:20 | 0:37:15 | 0:16:05 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896041 |
|
2024-09-09 06:31:17 | 2024-09-11 17:42:35 | 2024-09-11 21:00:29 | 3:17:54 | 3:00:42 | 0:17:12 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726078199.6091907 mon.a (mon.0) 634 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896042 |
|
2024-09-09 06:31:18 | 2024-09-11 17:49:16 | 2024-09-11 21:11:46 | 3:22:30 | 3:06:17 | 0:16:13 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T18:19:44.907419+0000 mon.a (mon.0) 369 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896043 |
|
2024-09-09 06:31:20 | 2024-09-11 17:53:47 | 2024-09-11 18:36:07 | 0:42:20 | 0:31:02 | 0:11:18 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896044 |
|
2024-09-09 06:31:21 | 2024-09-11 17:55:07 | 2024-09-11 21:10:30 | 3:15:23 | 2:51:03 | 0:24:20 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726079528.0356066 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896045 |
|
2024-09-09 06:31:26 | 2024-09-11 18:10:30 | 2024-09-11 22:09:19 | 3:58:49 | 3:44:20 | 0:14:29 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T18:36:58.578206+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896046 |
|
2024-09-09 06:31:27 | 2024-09-11 18:15:33 | 2024-09-11 19:02:05 | 0:46:32 | 0:15:04 | 0:31:28 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/yes}} | 3 |
| fail | 7896047 |
|
2024-09-09 06:31:28 | 2024-09-11 18:40:28 | 2024-09-11 22:04:31 | 3:24:03 | 3:13:51 | 0:10:12 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726081482.8484728 mon.a (mon.0) 359 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896048 |
|
2024-09-09 06:31:30 | 2024-09-11 18:40:48 | 2024-09-11 19:05:54 | 0:25:06 | 0:08:20 | 0:16:46 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/new_ops/{newops}}} | 2 |
| fail | 7896049 |
|
2024-09-09 06:31:31 | 2024-09-11 18:48:31 | 2024-09-11 21:45:45 | 2:57:14 | 2:41:38 | 0:15:36 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T19:15:19.482696+0000 mon.a (mon.0) 672 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896050 |
|
2024-09-09 06:31:32 | 2024-09-11 18:52:33 | 2024-09-11 19:40:11 | 0:47:38 | 0:38:34 | 0:09:04 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896051 |
|
2024-09-09 06:31:34 | 2024-09-11 18:52:53 | 2024-09-11 21:29:53 | 2:37:00 | 2:25:02 | 0:11:58 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726081884.5605054 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896052 |
|
2024-09-09 06:31:35 | 2024-09-11 18:54:24 | 2024-09-11 19:23:21 | 0:28:57 | 0:19:38 | 0:09:19 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} | 2 |
| fail | 7896053 |
|
2024-09-09 06:31:36 | 2024-09-11 18:54:24 | 2024-09-11 22:03:28 | 3:09:04 | 2:57:22 | 0:11:42 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T19:17:04.284492+0000 mon.a (mon.0) 688 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896054 |
|
2024-09-09 06:31:38 | 2024-09-11 18:56:15 | 2024-09-11 19:37:55 | 0:41:40 | 0:31:58 | 0:09:42 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896055 |
|
2024-09-09 06:31:39 | 2024-09-11 18:56:56 | 2024-09-11 22:32:10 | 3:35:14 | 3:22:13 | 0:13:01 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726082443.4755738 mon.a (mon.0) 502 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896056 |
|
2024-09-09 06:31:40 | 2024-09-11 19:00:27 | 2024-09-11 22:56:18 | 3:55:51 | 3:44:42 | 0:11:09 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T19:23:59.052643+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896057 |
|
2024-09-09 06:31:42 | 2024-09-11 19:02:28 | 2024-09-11 19:44:29 | 0:42:01 | 0:31:11 | 0:10:50 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896058 |
|
2024-09-09 06:31:43 | 2024-09-11 19:03:28 | 2024-09-11 22:03:33 | 3:00:05 | 2:48:22 | 0:11:43 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726082866.2319355 mon.a (mon.0) 180 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896059 |
|
2024-09-09 06:31:44 | 2024-09-11 19:05:59 | 2024-09-11 22:29:22 | 3:23:23 | 3:14:02 | 0:09:21 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T19:25:32.921850+0000 mon.a (mon.0) 514 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896060 |
|
2024-09-09 06:31:46 | 2024-09-11 19:06:10 | 2024-09-11 22:07:19 | 3:01:09 | 2:40:43 | 0:20:26 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726083275.97189 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896061 |
|
2024-09-09 06:31:47 | 2024-09-11 19:15:42 | 2024-09-11 20:18:47 | 1:03:05 | 0:51:02 | 0:12:03 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896062 |
|
2024-09-09 06:31:49 | 2024-09-11 19:17:33 | 2024-09-11 22:26:23 | 3:08:50 | 2:56:38 | 0:12:12 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T19:40:34.468758+0000 mon.a (mon.0) 648 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896063 |
|
2024-09-09 06:31:50 | 2024-09-11 19:18:54 | 2024-09-11 22:07:34 | 2:48:40 | 2:38:00 | 0:10:40 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726083453.7985337 mon.a (mon.0) 170 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896064 |
|
2024-09-09 06:31:52 | 2024-09-11 19:20:25 | 2024-09-11 20:01:44 | 0:41:19 | 0:30:55 | 0:10:24 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T19:50:00.000281+0000 mon.smithi028 (mon.0) 363 : cluster [WRN] osd.2 (root=default,host=smithi028) is down" in cluster log |
||||||||||||||
| fail | 7896065 |
|
2024-09-09 06:31:53 | 2024-09-11 19:20:35 | 2024-09-11 22:55:44 | 3:35:09 | 3:23:53 | 0:11:16 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T19:44:31.618400+0000 mon.a (mon.0) 677 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896066 |
|
2024-09-09 06:31:54 | 2024-09-11 19:21:36 | 2024-09-11 22:43:00 | 3:21:24 | 3:10:02 | 0:11:22 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726083805.3973744 mon.a (mon.0) 545 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896067 |
|
2024-09-09 06:31:56 | 2024-09-11 19:23:37 | 2024-09-11 20:26:28 | 1:02:51 | 0:50:30 | 0:12:21 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896068 |
|
2024-09-09 06:31:57 | 2024-09-11 19:25:28 | 2024-09-11 23:25:26 | 3:59:58 | 3:46:12 | 0:13:46 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T19:50:48.811002+0000 mon.a (mon.0) 178 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896069 |
|
2024-09-09 06:31:58 | 2024-09-11 19:29:20 | 2024-09-11 22:18:32 | 2:49:12 | 2:37:39 | 0:11:33 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726084374.781541 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896070 |
|
2024-09-09 06:32:00 | 2024-09-11 19:30:20 | 2024-09-11 20:11:20 | 0:41:00 | 0:31:41 | 0:09:19 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896071 |
|
2024-09-09 06:32:01 | 2024-09-11 19:30:21 | 2024-09-11 22:41:48 | 3:11:27 | 3:02:19 | 0:09:08 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T19:51:03.095710+0000 mon.a (mon.0) 688 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896072 |
|
2024-09-09 06:32:02 | 2024-09-11 19:30:31 | 2024-09-11 22:28:49 | 2:58:18 | 2:47:22 | 0:10:56 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T20:31:29.443394+0000 mon.a (mon.0) 1882 : cluster [ERR] Health check failed: Possible data damage: 1 pg recovery_unfound (PG_DAMAGED)" in cluster log |
||||||||||||||
| fail | 7896073 |
|
2024-09-09 06:32:04 | 2024-09-11 19:31:42 | 2024-09-11 20:20:57 | 0:49:15 | 0:36:59 | 0:12:16 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896074 |
|
2024-09-09 06:32:05 | 2024-09-11 19:33:43 | 2024-09-11 23:00:42 | 3:26:59 | 3:14:17 | 0:12:42 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T19:56:45.519086+0000 mon.a (mon.0) 694 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896075 |
|
2024-09-09 06:32:06 | 2024-09-11 19:35:14 | 2024-09-11 22:36:15 | 3:01:01 | 2:49:30 | 0:11:31 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726084800.0000908 mon.a (mon.0) 744 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds" in cluster log |
||||||||||||||
| pass | 7896076 |
|
2024-09-09 06:32:08 | 2024-09-11 19:36:25 | 2024-09-11 20:17:44 | 0:41:19 | 0:32:08 | 0:09:11 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896077 |
|
2024-09-09 06:32:09 | 2024-09-11 19:36:36 | 2024-09-11 23:05:50 | 3:29:14 | 3:19:07 | 0:10:07 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T19:59:21.579287+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896078 |
|
2024-09-09 06:32:10 | 2024-09-11 19:37:56 | 2024-09-11 22:38:37 | 3:00:41 | 2:50:01 | 0:10:40 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726084845.931249 mon.a (mon.0) 179 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896079 |
|
2024-09-09 06:32:11 | 2024-09-11 19:38:07 | 2024-09-11 20:24:35 | 0:46:28 | 0:36:14 | 0:10:14 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896080 |
|
2024-09-09 06:32:13 | 2024-09-11 19:39:38 | 2024-09-11 23:00:07 | 3:20:29 | 3:10:07 | 0:10:22 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T20:01:18.833736+0000 mon.a (mon.0) 669 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896081 |
|
2024-09-09 06:32:14 | 2024-09-11 19:39:48 | 2024-09-11 21:09:14 | 1:29:26 | 1:19:23 | 0:10:03 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/kernel_untar_build}} | 2 |
| fail | 7896082 |
|
2024-09-09 06:32:16 | 2024-09-11 19:39:59 | 2024-09-11 22:19:48 | 2:39:49 | 2:30:03 | 0:09:46 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726084807.989998 mon.a (mon.0) 518 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896083 |
|
2024-09-09 06:32:17 | 2024-09-11 19:40:29 | 2024-09-11 20:13:48 | 0:33:19 | 0:18:54 | 0:14:25 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7896084 |
|
2024-09-09 06:32:18 | 2024-09-11 19:44:50 | 2024-09-11 21:51:24 | 2:06:34 | 1:53:53 | 0:12:41 | smithi | main | centos | 9.stream | upgrade/reef-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-09-11T20:07:36.164021+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896085 |
|
2024-09-09 06:32:20 | 2024-09-11 19:48:22 | 2024-09-11 20:46:17 | 0:57:55 | 0:46:37 | 0:11:18 | smithi | main | ubuntu | 22.04 | upgrade/telemetry-upgrade/reef-x/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks} | 2 |
Failure Reason:
"2024-09-11T20:20:00.000232+0000 mon.a (mon.0) 695 : cluster [WRN] application not enabled on pool 'cephfs_metadata'" in cluster log |
||||||||||||||
| fail | 7896086 |
|
2024-09-09 06:32:21 | 2024-09-11 19:48:22 | 2024-09-11 21:34:37 | 1:46:15 | 1:35:03 | 0:11:12 | smithi | main | centos | 9.stream | upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1726085392.8303697 mon.a (mon.0) 539 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896087 |
|
2024-09-09 06:32:22 | 2024-09-11 19:49:13 | 2024-09-11 20:37:11 | 0:47:58 | 0:37:10 | 0:10:48 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896088 |
|
2024-09-09 06:32:24 | 2024-09-11 19:50:04 | 2024-09-11 23:15:25 | 3:25:21 | 3:10:57 | 0:14:24 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T20:14:12.736190+0000 mon.a (mon.0) 527 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896089 |
|
2024-09-09 06:32:25 | 2024-09-11 19:53:45 | 2024-09-11 20:10:49 | 0:17:04 | 0:07:57 | 0:09:07 | smithi | main | centos | 9.stream | upgrade/cephfs/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/reef 1-upgrade}} | 1 |
| fail | 7896090 |
|
2024-09-09 06:32:27 | 2024-09-11 19:53:45 | 2024-09-11 22:54:10 | 3:00:25 | 2:48:36 | 0:11:49 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726085712.0132957 mon.a (mon.0) 545 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896091 |
|
2024-09-09 06:32:28 | 2024-09-11 19:57:31 | 2024-09-11 20:21:16 | 0:23:45 | 0:08:53 | 0:14:52 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} | 2 |
| fail | 7896092 |
|
2024-09-09 06:32:29 | 2024-09-11 20:02:02 | 2024-09-11 23:22:24 | 3:20:22 | 3:05:37 | 0:14:45 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T20:27:14.548071+0000 mon.a (mon.0) 526 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896093 |
|
2024-09-09 06:32:31 | 2024-09-11 20:07:04 | 2024-09-11 23:04:49 | 2:57:45 | 2:45:55 | 0:11:50 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726086585.4933116 mon.a (mon.0) 681 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896094 |
|
2024-09-09 06:32:32 | 2024-09-11 20:07:34 | 2024-09-11 20:54:34 | 0:47:00 | 0:32:11 | 0:14:49 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896095 |
|
2024-09-09 06:32:34 | 2024-09-11 20:11:36 | 2024-09-11 23:19:54 | 3:08:18 | 2:58:51 | 0:09:27 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T20:36:04.895020+0000 mon.a (mon.0) 367 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896096 |
|
2024-09-09 06:32:35 | 2024-09-11 20:12:06 | 2024-09-11 23:03:45 | 2:51:39 | 2:39:36 | 0:12:03 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726086921.1000884 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896097 |
|
2024-09-09 06:32:36 | 2024-09-11 20:13:57 | 2024-09-11 20:59:12 | 0:45:15 | 0:36:54 | 0:08:21 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896098 |
|
2024-09-09 06:32:38 | 2024-09-11 20:13:58 | 2024-09-12 00:01:30 | 3:47:32 | 3:35:06 | 0:12:26 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T20:35:29.960294+0000 mon.a (mon.0) 523 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7896099 |
|
2024-09-09 06:32:39 | 2024-09-11 20:16:09 | 2024-09-11 23:39:27 | 3:23:18 | 3:10:47 | 0:12:31 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726087000.3751545 mon.a (mon.0) 369 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896100 |
|
2024-09-09 06:32:40 | 2024-09-11 20:18:00 | 2024-09-11 21:00:15 | 0:42:15 | 0:31:27 | 0:10:48 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896101 |
|
2024-09-09 06:32:42 | 2024-09-11 20:19:00 | 2024-09-11 23:11:38 | 2:52:38 | 2:41:49 | 0:10:49 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T20:41:20.117716+0000 mon.a (mon.0) 670 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896102 |
|
2024-09-09 06:32:43 | 2024-09-11 20:20:11 | 2024-09-11 22:54:33 | 2:34:22 | 2:23:52 | 0:10:30 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726087136.7088747 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896103 |
|
2024-09-09 06:32:44 | 2024-09-11 20:21:12 | 2024-09-11 21:22:37 | 1:01:25 | 0:51:15 | 0:10:10 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896104 |
|
2024-09-09 06:32:46 | 2024-09-11 20:21:23 | 2024-09-12 00:13:39 | 3:52:16 | 3:40:51 | 0:11:25 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T20:43:45.402646+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896105 |
|
2024-09-09 06:32:47 | 2024-09-11 20:21:33 | 2024-09-11 23:34:44 | 3:13:11 | 3:00:21 | 0:12:50 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726088132.7028508 mon.a (mon.0) 711 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896106 |
|
2024-09-09 06:32:49 | 2024-09-11 20:24:54 | 2024-09-11 21:07:36 | 0:42:42 | 0:31:12 | 0:11:30 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896107 |
|
2024-09-09 06:32:50 | 2024-09-11 20:26:45 | 2024-09-12 00:15:54 | 3:49:09 | 3:32:39 | 0:16:30 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T20:55:55.992919+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896108 |
|
2024-09-09 06:32:51 | 2024-09-11 20:34:02 | 2024-09-11 23:39:15 | 3:05:13 | 2:51:36 | 0:13:37 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726088322.0686703 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896109 |
|
2024-09-09 06:32:53 | 2024-09-11 20:37:23 | 2024-09-11 21:38:25 | 1:01:02 | 0:51:08 | 0:09:54 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896110 |
|
2024-09-09 06:32:54 | 2024-09-11 20:37:33 | 2024-09-12 00:01:49 | 3:24:16 | 3:12:13 | 0:12:03 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T20:59:58.591985+0000 mon.a (mon.0) 525 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7896111 |
|
2024-09-09 06:32:56 | 2024-09-11 20:42:32 | 2024-09-11 23:42:45 | 3:00:13 | 2:48:41 | 0:11:32 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726088676.4326398 mon.a (mon.0) 511 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896112 |
|
2024-09-09 06:32:57 | 2024-09-11 20:45:33 | 2024-09-11 21:26:59 | 0:41:26 | 0:31:29 | 0:09:57 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896113 |
|
2024-09-09 06:32:58 | 2024-09-11 20:45:54 | 2024-09-11 23:48:02 | 3:02:08 | 2:50:27 | 0:11:41 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T21:07:35.700171+0000 mon.a (mon.0) 529 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7896114 |
|
2024-09-09 06:33:00 | 2024-09-11 20:46:35 | 2024-09-11 23:34:13 | 2:47:38 | 2:28:35 | 0:19:03 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726089215.6816838 mon.a (mon.0) 373 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896115 |
|
2024-09-09 06:33:01 | 2024-09-11 20:54:56 | 2024-09-11 21:40:33 | 0:45:37 | 0:31:20 | 0:14:17 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-11T21:30:00.000125+0000 mon.smithi110 (mon.0) 324 : cluster [WRN] pg 2.4 is active+undersized+degraded, acting [0,4]" in cluster log |
||||||||||||||
| fail | 7896116 |
|
2024-09-09 06:33:03 | 2024-09-11 20:59:27 | 2024-09-12 01:24:38 | 4:25:11 | 4:13:11 | 0:12:00 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T21:22:54.530712+0000 mon.a (mon.0) 178 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896117 |
|
2024-09-09 06:33:04 | 2024-09-11 21:00:28 | 2024-09-12 00:12:33 | 3:12:05 | 3:03:15 | 0:08:50 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726090067.4978495 mon.a (mon.0) 546 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896118 |
|
2024-09-09 06:33:05 | 2024-09-11 21:00:48 | 2024-09-11 21:29:47 | 0:28:59 | 0:12:53 | 0:16:06 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/iozone}} | 2 |
| fail | 7896119 |
|
2024-09-09 06:33:07 | 2024-09-11 21:06:30 | 2024-09-12 00:10:58 | 3:04:28 | 2:53:56 | 0:10:32 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-11T21:30:00.000211+0000 mon.a (mon.0) 736 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is offline; 1 filesystem is online with fewer MDS than max_mds" in cluster log |
||||||||||||||
| fail | 7896120 |
|
2024-09-09 06:33:08 | 2024-09-11 21:07:50 | 2024-09-11 21:54:36 | 0:46:46 | 0:36:40 | 0:10:06 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896121 |
|
2024-09-09 06:33:09 | 2024-09-11 21:09:31 | 2024-09-11 23:48:09 | 2:38:38 | 2:26:57 | 0:11:41 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726090246.4549417 mon.a (mon.0) 544 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896122 |
|
2024-09-09 06:33:11 | 2024-09-11 21:10:51 | 2024-09-12 00:47:11 | 3:36:20 | 3:24:57 | 0:11:23 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-11T21:31:06.082935+0000 mon.a (mon.0) 358 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896123 |
|
2024-09-09 06:33:12 | 2024-09-11 21:12:02 | 2024-09-11 22:04:04 | 0:52:02 | 0:31:30 | 0:20:32 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896124 |
|
2024-09-09 06:33:13 | 2024-09-12 13:22:07 | 2024-09-12 16:47:55 | 3:25:48 | 3:15:22 | 0:10:26 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726148495.6902962 mon.a (mon.0) 369 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7896125 |
|
2024-09-09 06:33:15 | 2024-09-12 13:22:28 | 2024-09-12 13:48:06 | 0:25:38 | 0:16:22 | 0:09:16 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7896126 |
|
2024-09-09 06:33:16 | 2024-09-12 13:22:58 | 2024-09-12 16:58:50 | 3:35:52 | 3:25:36 | 0:10:16 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T13:40:56.789511+0000 mon.a (mon.0) 170 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896127 |
|
2024-09-09 06:33:18 | 2024-09-12 13:22:58 | 2024-09-12 16:15:01 | 2:52:03 | 2:40:29 | 0:11:34 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726148740.1984255 mon.a (mon.0) 653 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896128 |
|
2024-09-09 06:33:19 | 2024-09-12 13:23:39 | 2024-09-12 14:10:22 | 0:46:43 | 0:37:11 | 0:09:32 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896129 |
|
2024-09-09 06:33:20 | 2024-09-12 13:23:39 | 2024-09-12 16:57:36 | 3:33:57 | 3:24:05 | 0:09:52 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T13:45:18.900737+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896130 |
|
2024-09-09 06:33:22 | 2024-09-12 13:23:40 | 2024-09-12 16:29:55 | 3:06:15 | 2:56:43 | 0:09:32 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726148754.4272008 mon.a (mon.0) 181 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896131 |
|
2024-09-09 06:33:23 | 2024-09-12 13:24:10 | 2024-09-12 13:55:12 | 0:31:02 | 0:22:33 | 0:08:29 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/blogbench}} | 2 |
| pass | 7896132 |
|
2024-09-09 06:33:25 | 2024-09-12 13:24:11 | 2024-09-12 14:05:35 | 0:41:24 | 0:30:24 | 0:11:00 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896133 |
|
2024-09-09 06:33:26 | 2024-09-12 13:24:11 | 2024-09-12 16:31:07 | 3:06:56 | 2:57:05 | 0:09:51 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T13:44:39.644067+0000 mon.a (mon.0) 512 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896134 |
|
2024-09-09 06:33:27 | 2024-09-12 13:24:11 | 2024-09-12 15:58:48 | 2:34:37 | 2:24:05 | 0:10:32 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726148546.4333227 mon.a (mon.0) 170 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896135 |
|
2024-09-09 06:33:29 | 2024-09-12 13:25:32 | 2024-09-12 14:26:40 | 1:01:08 | 0:51:36 | 0:09:32 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/dbench}} | 2 |
| fail | 7896136 |
|
2024-09-09 06:33:30 | 2024-09-12 13:25:32 | 2024-09-12 14:13:29 | 0:47:57 | 0:37:18 | 0:10:39 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896137 |
|
2024-09-09 06:33:32 | 2024-09-12 13:26:33 | 2024-09-12 17:05:45 | 3:39:12 | 3:30:31 | 0:08:41 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T13:43:47.383882+0000 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896138 |
|
2024-09-09 06:33:33 | 2024-09-12 13:26:33 | 2024-09-12 16:43:48 | 3:17:15 | 3:08:50 | 0:08:25 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726148862.5746548 mon.a (mon.0) 653 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896139 |
|
2024-09-09 06:33:34 | 2024-09-12 13:26:34 | 2024-09-12 14:08:02 | 0:41:28 | 0:31:56 | 0:09:32 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"1726148797.6805663 mon.smithi112 (mon.0) 335 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log |
||||||||||||||
| fail | 7896140 |
|
2024-09-09 06:33:36 | 2024-09-12 13:27:04 | 2024-09-12 16:42:20 | 3:15:16 | 3:01:04 | 0:14:12 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T13:52:57.009186+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896141 |
|
2024-09-09 06:33:37 | 2024-09-12 13:29:55 | 2024-09-12 16:35:58 | 3:06:03 | 2:55:45 | 0:10:18 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} | 2 |
Failure Reason:
"1726149136.9458225 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896142 |
|
2024-09-09 06:33:39 | 2024-09-12 13:29:55 | 2024-09-12 14:19:57 | 0:50:02 | 0:37:15 | 0:12:47 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896143 |
|
2024-09-09 06:33:40 | 2024-09-12 13:32:26 | 2024-09-12 16:58:50 | 3:26:24 | 3:13:38 | 0:12:46 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T13:55:26.046866+0000 mon.a (mon.0) 513 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7896144 |
|
2024-09-09 06:33:41 | 2024-09-12 13:35:47 | 2024-09-12 16:37:11 | 3:01:24 | 2:50:09 | 0:11:15 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} | 2 |
Failure Reason:
"1726149216.9394453 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896145 |
|
2024-09-09 06:33:43 | 2024-09-12 13:35:48 | 2024-09-12 16:55:02 | 3:19:14 | 3:06:57 | 0:12:17 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T13:56:49.980764+0000 mon.a (mon.0) 523 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896146 |
|
2024-09-09 06:33:44 | 2024-09-12 13:37:48 | 2024-09-12 14:26:25 | 0:48:37 | 0:37:12 | 0:11:25 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896147 |
|
2024-09-09 06:33:46 | 2024-09-12 13:39:29 | 2024-09-12 16:37:26 | 2:57:57 | 2:46:51 | 0:11:06 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726149411.036823 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896148 |
|
2024-09-09 06:33:47 | 2024-09-12 13:39:29 | 2024-09-12 17:00:10 | 3:20:41 | 3:09:46 | 0:10:55 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:01:52.044500+0000 mon.a (mon.0) 677 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896149 |
|
2024-09-09 06:33:49 | 2024-09-12 13:41:00 | 2024-09-12 16:32:37 | 2:51:37 | 2:40:32 | 0:11:05 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726149628.2614303 mon.a (mon.0) 357 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896150 |
|
2024-09-09 06:33:50 | 2024-09-12 13:41:21 | 2024-09-12 14:23:00 | 0:41:39 | 0:31:17 | 0:10:22 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-12T14:10:00.000150+0000 mon.smithi090 (mon.0) 258 : cluster [WRN] osd.0 (root=default,host=smithi090) is down" in cluster log |
||||||||||||||
| fail | 7896151 |
|
2024-09-09 06:33:52 | 2024-09-12 13:41:21 | 2024-09-12 16:53:42 | 3:12:21 | 3:03:18 | 0:09:03 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:03:22.806975+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896152 |
|
2024-09-09 06:33:53 | 2024-09-12 13:41:52 | 2024-09-12 16:25:44 | 2:43:52 | 2:34:39 | 0:09:13 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726150003.084472 mon.a (mon.0) 364 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896153 |
|
2024-09-09 06:33:54 | 2024-09-12 13:41:52 | 2024-09-12 14:29:08 | 0:47:16 | 0:37:30 | 0:09:46 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896154 |
|
2024-09-09 06:33:56 | 2024-09-12 13:41:53 | 2024-09-12 17:57:58 | 4:16:05 | 4:05:21 | 0:10:44 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:02:35.509654+0000 mon.a (mon.0) 525 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896155 |
|
2024-09-09 06:33:57 | 2024-09-12 13:42:13 | 2024-09-12 14:07:02 | 0:24:49 | 0:13:24 | 0:11:25 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/fsstress}} | 2 |
| fail | 7896156 |
|
2024-09-09 06:33:58 | 2024-09-12 13:44:04 | 2024-09-12 16:51:16 | 3:07:12 | 2:58:04 | 0:09:08 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726149835.2021267 mon.a (mon.0) 589 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896157 |
|
2024-09-09 06:34:00 | 2024-09-12 13:44:04 | 2024-09-12 14:25:36 | 0:41:32 | 0:30:50 | 0:10:42 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896158 |
|
2024-09-09 06:34:01 | 2024-09-12 13:44:35 | 2024-09-12 17:00:12 | 3:15:37 | 3:05:29 | 0:10:08 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:03:40.684488+0000 mon.a (mon.0) 366 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896159 |
|
2024-09-09 06:34:02 | 2024-09-12 13:44:55 | 2024-09-12 16:14:06 | 2:29:11 | 2:19:38 | 0:09:33 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726149758.8606594 mon.a (mon.0) 170 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896160 |
|
2024-09-09 06:34:04 | 2024-09-12 13:44:55 | 2024-09-12 14:45:58 | 1:01:03 | 0:52:16 | 0:08:47 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896161 |
|
2024-09-09 06:34:05 | 2024-09-12 13:44:56 | 2024-09-12 17:42:43 | 3:57:47 | 3:48:49 | 0:08:58 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:06:47.213428+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896162 |
|
2024-09-09 06:34:07 | 2024-09-12 13:44:56 | 2024-09-12 17:06:18 | 3:21:22 | 3:11:12 | 0:10:10 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726150369.4874036 mon.a (mon.0) 506 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| pass | 7896163 |
|
2024-09-09 06:34:08 | 2024-09-12 13:46:37 | 2024-09-12 14:28:37 | 0:42:00 | 0:30:42 | 0:11:18 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896164 |
|
2024-09-09 06:34:09 | 2024-09-12 13:47:17 | 2024-09-12 18:01:15 | 4:13:58 | 4:01:48 | 0:12:10 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:12:54.935109+0000 mon.a (mon.0) 174 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896165 |
|
2024-09-09 06:34:11 | 2024-09-12 13:47:18 | 2024-09-12 19:04:28 | 5:17:10 | 5:06:46 | 0:10:24 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
Command failed (workunit test rbd/test_librbd.sh) on smithi077 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' |
||||||||||||||
| pass | 7896166 |
|
2024-09-09 06:34:12 | 2024-09-12 13:48:18 | 2024-09-12 14:11:33 | 0:23:15 | 0:13:55 | 0:09:20 | smithi | main | centos | 9.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/yes}} | 3 |
| fail | 7896167 |
|
2024-09-09 06:34:13 | 2024-09-12 13:48:29 | 2024-09-12 17:37:37 | 3:49:08 | 3:36:49 | 0:12:19 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:11:58.558507+0000 mon.a (mon.0) 688 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896168 |
|
2024-09-09 06:34:15 | 2024-09-12 13:50:09 | 2024-09-12 16:51:33 | 3:01:24 | 2:51:46 | 0:09:38 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726150087.609797 mon.a (mon.0) 171 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896169 |
|
2024-09-09 06:34:16 | 2024-09-12 13:50:10 | 2024-09-12 14:51:22 | 1:01:12 | 0:51:36 | 0:09:36 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| pass | 7896170 |
|
2024-09-09 06:34:18 | 2024-09-12 13:50:20 | 2024-09-12 14:21:19 | 0:30:59 | 0:19:26 | 0:11:33 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} | 2 |
| fail | 7896171 |
|
2024-09-09 06:34:19 | 2024-09-12 13:52:21 | 2024-09-12 17:05:37 | 3:13:16 | 3:03:03 | 0:10:13 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:13:33.840258+0000 mon.a (mon.0) 693 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896172 |
|
2024-09-09 06:34:20 | 2024-09-12 13:52:21 | 2024-09-12 16:36:25 | 2:44:04 | 2:32:15 | 0:11:49 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726150582.016843 mon.a (mon.0) 658 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896173 |
|
2024-09-09 06:34:22 | 2024-09-12 13:55:22 | 2024-09-12 17:42:40 | 3:47:18 | 3:34:41 | 0:12:37 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
Command failed (workunit test suites/fsstress.sh) on smithi137 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f03a10869dbcdcf78fc7c60470b0f6dfddc7d42e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh' |
||||||||||||||
| fail | 7896174 |
|
2024-09-09 06:34:23 | 2024-09-12 13:55:23 | 2024-09-12 18:15:27 | 4:20:04 | 4:11:00 | 0:09:04 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:17:15.716930+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896175 |
|
2024-09-09 06:34:24 | 2024-09-12 13:55:23 | 2024-09-12 17:15:58 | 3:20:35 | 3:10:31 | 0:10:04 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726150694.2584243 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896176 |
|
2024-09-09 06:34:26 | 2024-09-12 13:56:04 | 2024-09-12 14:37:59 | 0:41:55 | 0:32:06 | 0:09:49 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-12T14:30:00.000238+0000 mon.smithi154 (mon.0) 510 : cluster [WRN] osd.5 (root=default,host=smithi173) is down" in cluster log |
||||||||||||||
| fail | 7896177 |
|
2024-09-09 06:34:27 | 2024-09-12 13:56:44 | 2024-09-12 16:51:56 | 2:55:12 | 2:46:29 | 0:08:43 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:17:29.167191+0000 mon.a (mon.0) 694 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896178 |
|
2024-09-09 06:34:28 | 2024-09-12 13:56:44 | 2024-09-12 16:38:53 | 2:42:09 | 2:25:26 | 0:16:43 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726151050.532066 mon.a (mon.0) 504 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896179 |
|
2024-09-09 06:34:30 | 2024-09-12 14:02:36 | 2024-09-12 14:51:07 | 0:48:31 | 0:36:32 | 0:11:59 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896180 |
|
2024-09-09 06:34:31 | 2024-09-12 14:04:16 | 2024-09-12 17:43:43 | 3:39:27 | 3:30:38 | 0:08:49 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:25:30.363071+0000 mon.a (mon.0) 693 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896181 |
|
2024-09-09 06:34:32 | 2024-09-12 14:04:27 | 2024-09-12 17:17:15 | 3:12:48 | 3:00:54 | 0:11:54 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726151214.0577748 mon.a (mon.0) 588 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896182 |
|
2024-09-09 06:34:34 | 2024-09-12 14:05:47 | 2024-09-12 14:50:08 | 0:44:21 | 0:33:32 | 0:10:49 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-12T14:40:00.000136+0000 mon.smithi016 (mon.0) 393 : cluster [WRN] pg 2.1 is active+undersized+degraded, acting [1,0]" in cluster log |
||||||||||||||
| fail | 7896183 |
|
2024-09-09 06:34:35 | 2024-09-12 14:07:18 | 2024-09-12 17:45:30 | 3:38:12 | 3:26:58 | 0:11:14 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:27:32.486555+0000 mon.a (mon.0) 518 : cluster [WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log |
||||||||||||||
| fail | 7896184 |
|
2024-09-09 06:34:37 | 2024-09-12 14:08:19 | 2024-09-12 17:07:37 | 2:59:18 | 2:47:53 | 0:11:25 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726151519.652312 mon.a (mon.0) 674 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896185 |
|
2024-09-09 06:34:38 | 2024-09-12 14:10:29 | 2024-09-12 14:56:00 | 0:45:31 | 0:35:41 | 0:09:50 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896186 |
|
2024-09-09 06:34:39 | 2024-09-12 14:10:40 | 2024-09-12 17:41:46 | 3:31:06 | 3:20:19 | 0:10:47 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:34:14.111920+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896187 |
|
2024-09-09 06:34:41 | 2024-09-12 14:11:50 | 2024-09-12 17:17:38 | 3:05:48 | 2:55:36 | 0:10:12 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726151588.7251172 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896188 |
|
2024-09-09 06:34:42 | 2024-09-12 14:11:51 | 2024-09-12 14:54:42 | 0:42:51 | 0:31:37 | 0:11:14 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7896189 |
|
2024-09-09 06:34:43 | 2024-09-12 14:13:52 | 2024-09-12 17:21:36 | 3:07:44 | 2:55:38 | 0:12:06 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:35:53.782190+0000 mon.a (mon.0) 526 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896190 |
|
2024-09-09 06:34:45 | 2024-09-12 14:16:32 | 2024-09-12 16:56:17 | 2:39:45 | 2:28:51 | 0:10:54 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726151837.2276196 mon.a (mon.0) 507 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896191 |
|
2024-09-09 06:34:46 | 2024-09-12 14:16:33 | 2024-09-12 15:05:43 | 0:49:10 | 0:36:02 | 0:13:08 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896192 |
|
2024-09-09 06:34:47 | 2024-09-12 14:20:14 | 2024-09-12 18:15:08 | 3:54:54 | 3:44:14 | 0:10:40 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:41:46.326744+0000 mon.a (mon.0) 523 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896193 |
|
2024-09-09 06:34:49 | 2024-09-12 14:21:34 | 2024-09-12 17:36:21 | 3:14:47 | 3:03:35 | 0:11:12 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726152146.372678 mon.a (mon.0) 513 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7896194 |
|
2024-09-09 06:34:50 | 2024-09-12 14:23:15 | 2024-09-12 15:07:24 | 0:44:09 | 0:31:54 | 0:12:15 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-09-12T15:00:00.000162+0000 mon.smithi001 (mon.0) 614 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log |
||||||||||||||
| fail | 7896195 |
|
2024-09-09 06:34:51 | 2024-09-12 14:25:46 | 2024-09-12 17:38:25 | 3:12:39 | 3:01:46 | 0:10:53 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-09-12T14:48:20.046798+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896196 |
|
2024-09-09 06:34:53 | 2024-09-12 14:26:26 | 2024-09-12 17:10:14 | 2:43:48 | 2:34:38 | 0:09:10 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1726152503.3922617 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7896197 |
|
2024-09-09 06:34:54 | 2024-09-12 14:26:37 | 2024-09-12 15:16:13 | 0:49:36 | 0:36:44 | 0:12:52 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (50) after waiting for 300 seconds |
||||||||||||||
| fail | 7896198 |
|
2024-09-09 06:34:56 | 2024-09-12 14:28:58 | 2024-09-12 17:48:32 | 3:19:34 | 3:10:19 | 0:09:15 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-09-12T14:51:04.288110+0000 mon.a (mon.0) 681 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7896199 |
|
2024-09-09 06:34:57 | 2024-09-12 14:29:18 | 2024-09-12 17:35:29 | 3:06:11 | 2:54:08 | 0:12:03 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1726152497.9949691 mon.a (mon.0) 169 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| pass | 7896200 |
|
2024-09-09 06:34:58 | 2024-09-12 14:29:28 | 2024-09-12 15:58:48 | 1:29:20 | 1:18:37 | 0:10:43 | smithi | main | centos | 9.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/kernel_untar_build}} | 2 |
Pulpito