| User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
|---|---|---|---|---|---|---|---|---|---|---|---|
| lflores | 2024-04-29 20:31:53 | 2024-04-29 20:34:09 | 2024-04-30 09:25:04 | 12:50:55 | upgrade | wip-yuriw-testing-20240419.185239-main | smithi | 36c3715 | 10 | 51 | 1 |
| Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| pass | 7679145 |
|
2024-04-29 20:32:03 | 2024-04-29 20:34:09 | 2024-04-29 21:14:41 | 0:40:32 | 0:32:21 | 0:08:11 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
| fail | 7679146 |
|
2024-04-29 20:32:04 | 2024-04-29 20:34:49 | 2024-04-29 22:11:17 | 1:36:28 | 1:28:03 | 0:08:25 | smithi | main | centos | 9.stream | upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1714423934.8149796 mon.a (mon.0) 518 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679147 |
|
2024-04-29 20:32:06 | 2024-04-29 20:35:50 | 2024-04-29 23:48:27 | 3:12:37 | 3:03:34 | 0:09:03 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-04-29T21:03:19.184245+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7679148 |
|
2024-04-29 20:32:07 | 2024-04-29 20:37:11 | 2024-04-29 23:13:34 | 2:36:23 | 2:29:16 | 0:07:07 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714424025.76663 mon.a (mon.0) 369 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679149 |
|
2024-04-29 20:32:08 | 2024-04-29 20:37:41 | 2024-04-29 21:09:47 | 0:32:06 | 0:21:03 | 0:11:03 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 |
Failure Reason:
Command failed on smithi110 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell 4.1f deep-scrub' |
||||||||||||||
| fail | 7679150 |
|
2024-04-29 20:32:10 | 2024-04-29 20:38:22 | 2024-04-29 21:40:34 | 1:02:12 | 0:52:16 | 0:09:56 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679151 |
|
2024-04-29 20:32:11 | 2024-04-29 20:38:52 | 2024-04-29 21:45:22 | 1:06:30 | 0:51:43 | 0:14:47 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679152 |
|
2024-04-29 20:32:12 | 2024-04-29 20:41:33 | 2024-04-29 21:54:36 | 1:13:03 | 1:05:56 | 0:07:07 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679153 |
|
2024-04-29 20:32:13 | 2024-04-30 00:24:10 | 12640 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 | |||
Failure Reason:
"2024-04-29T21:21:27.555180+0000 mon.a (mon.0) 55 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679154 |
|
2024-04-29 20:32:15 | 2024-04-29 20:42:14 | 2024-04-30 00:13:02 | 3:30:48 | 3:20:18 | 0:10:30 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714425018.5235858 mon.a (mon.0) 511 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679155 |
|
2024-04-29 20:32:16 | 2024-04-29 20:43:14 | 2024-04-29 21:22:54 | 0:39:40 | 0:31:55 | 0:07:45 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-04-29T21:10:00.000161+0000 mon.smithi076 (mon.0) 274 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 44/225 objects degraded (19.556%), 17 pgs degraded" in cluster log |
||||||||||||||
| fail | 7679156 |
|
2024-04-29 20:32:17 | 2024-04-29 20:43:25 | 2024-04-29 23:48:51 | 3:05:26 | 2:58:20 | 0:07:06 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-04-29T21:09:51.854049+0000 mon.a (mon.0) 305 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7679157 |
|
2024-04-29 20:32:18 | 2024-04-29 20:43:35 | 2024-04-29 23:13:05 | 2:29:30 | 2:22:49 | 0:06:41 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714424408.890281 mon.a (mon.0) 520 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679158 |
|
2024-04-29 20:32:20 | 2024-04-29 20:43:46 | 2024-04-29 21:21:48 | 0:38:02 | 0:29:34 | 0:08:28 | smithi | main | centos | 9.stream | upgrade/telemetry-upgrade/quincy-x/{0-random-distro$/{centos_9.stream} 0-start 1-tasks} | 2 |
Failure Reason:
"1714424538.6770813 mon.a (mon.0) 661 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7679159 |
|
2024-04-29 20:32:21 | 2024-04-29 20:44:46 | 2024-04-29 21:20:33 | 0:35:47 | 0:26:13 | 0:09:34 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7679160 |
|
2024-04-29 20:32:22 | 2024-04-29 20:44:47 | 2024-04-29 21:04:14 | 0:19:27 | 0:10:06 | 0:09:21 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_latest} | 1 |
Failure Reason:
"2024-04-29T21:03:03.932656+0000 mon.a (mon.0) 38 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| pass | 7679161 |
|
2024-04-29 20:32:23 | 2024-04-29 20:44:47 | 2024-04-29 21:05:36 | 0:20:49 | 0:11:49 | 0:09:00 | smithi | main | centos | 8.stream | upgrade/cephfs/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}} | 1 |
| fail | 7679162 |
|
2024-04-29 20:32:25 | 2024-04-29 20:44:47 | 2024-04-29 22:58:26 | 2:13:39 | 2:04:37 | 0:09:02 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-04-29T21:25:29.036051+0000 mon.a (mon.0) 291 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| pass | 7679163 |
|
2024-04-29 20:32:26 | 2024-04-29 20:44:48 | 2024-04-29 21:36:10 | 0:51:22 | 0:40:32 | 0:10:50 | smithi | main | centos | 8.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap branch/nautilus centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/kernel_untar_build}} | 2 |
| fail | 7679164 |
|
2024-04-29 20:32:27 | 2024-04-29 20:44:58 | 2024-04-29 22:44:18 | 1:59:20 | 1:45:57 | 0:13:23 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1714425495.0270755 mon.a (mon.0) 670 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679165 |
|
2024-04-29 20:32:28 | 2024-04-29 20:44:58 | 2024-04-30 00:01:56 | 3:16:58 | 3:09:33 | 0:07:25 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-04-29T21:12:46.096108+0000 mon.a (mon.0) 270 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7679166 |
|
2024-04-29 20:32:30 | 2024-04-29 20:44:59 | 2024-04-29 21:44:26 | 0:59:27 | 0:52:19 | 0:07:08 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679167 |
|
2024-04-29 20:32:31 | 2024-04-29 20:44:59 | 2024-04-29 23:24:47 | 2:39:48 | 2:32:52 | 0:06:56 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714424635.1416936 mon.a (mon.0) 686 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679168 |
|
2024-04-29 20:32:32 | 2024-04-29 20:45:00 | 2024-04-29 22:09:07 | 1:24:07 | 1:13:25 | 0:10:42 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679169 |
|
2024-04-29 20:32:33 | 2024-04-29 20:45:00 | 2024-04-29 21:21:56 | 0:36:56 | 0:30:41 | 0:06:15 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-04-29T21:10:00.000162+0000 mon.smithi078 (mon.0) 283 : cluster [WRN] Health detail: HEALTH_WARN Degraded data redundancy: 71/363 objects degraded (19.559%), 19 pgs degraded" in cluster log |
||||||||||||||
| fail | 7679170 |
|
2024-04-29 20:32:35 | 2024-04-29 20:45:01 | 2024-04-29 23:51:38 | 3:06:37 | 2:56:03 | 0:10:34 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714424831.8205016 mon.a (mon.0) 182 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7679171 |
|
2024-04-29 20:32:36 | 2024-04-29 20:45:01 | 2024-04-29 23:57:01 | 3:12:00 | 3:01:26 | 0:10:34 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-04-29T21:14:45.710510+0000 mon.a (mon.0) 289 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| pass | 7679172 |
|
2024-04-29 20:32:37 | 2024-04-29 20:48:22 | 2024-04-29 21:19:01 | 0:30:39 | 0:20:06 | 0:10:33 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7679173 |
|
2024-04-29 20:32:38 | 2024-04-29 20:49:52 | 2024-04-29 23:41:19 | 2:51:27 | 2:43:24 | 0:08:03 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714424794.689462 mon.a (mon.0) 524 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679174 |
|
2024-04-29 20:32:40 | 2024-04-29 20:50:13 | 2024-04-29 23:10:50 | 2:20:37 | 2:04:47 | 0:15:50 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-04-29T21:34:12.036638+0000 mon.a (mon.0) 18 : cluster [ERR] Health check failed: 8 osds(s) are not reachable (OSD_UNREACHABLE)" in cluster log |
||||||||||||||
| fail | 7679175 |
|
2024-04-29 20:32:41 | 2024-04-29 20:54:44 | 2024-04-29 21:43:55 | 0:49:11 | 0:37:37 | 0:11:34 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679176 |
|
2024-04-29 20:32:42 | 2024-04-29 20:58:45 | 2024-04-29 22:33:18 | 1:34:33 | 1:27:02 | 0:07:31 | smithi | main | centos | 9.stream | upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1714425383.281757 mon.a (mon.0) 504 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679177 |
|
2024-04-29 20:32:43 | 2024-04-29 21:00:15 | 2024-04-29 23:57:37 | 2:57:22 | 2:51:04 | 0:06:18 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-04-29T21:27:39.002869+0000 mon.a (mon.0) 322 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log |
||||||||||||||
| fail | 7679178 |
|
2024-04-29 20:32:45 | 2024-04-29 21:00:16 | 2024-04-29 21:39:15 | 0:38:59 | 0:31:26 | 0:07:33 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
"2024-04-29T21:30:00.000115+0000 mon.smithi007 (mon.0) 654 : cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is degraded" in cluster log |
||||||||||||||
| fail | 7679179 |
|
2024-04-29 20:32:46 | 2024-04-29 21:00:26 | 2024-04-29 23:43:59 | 2:43:33 | 2:36:11 | 0:07:22 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714425540.6640089 mon.a (mon.0) 668 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| pass | 7679180 |
|
2024-04-29 20:32:47 | 2024-04-29 21:00:27 | 2024-04-29 21:25:36 | 0:25:09 | 0:16:22 | 0:08:47 | smithi | main | centos | 8.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap branch/pacific centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/iozone}} | 2 |
| fail | 7679181 |
|
2024-04-29 20:32:48 | 2024-04-29 21:00:27 | 2024-04-30 00:33:53 | 3:33:26 | 3:26:03 | 0:07:23 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-04-29T21:25:48.866810+0000 mon.a (mon.0) 59 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679182 |
|
2024-04-29 20:32:50 | 2024-04-29 21:00:27 | 2024-04-30 00:05:32 | 3:05:05 | 2:58:31 | 0:06:34 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714425512.65651 mon.a (mon.0) 680 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679183 |
|
2024-04-29 20:32:51 | 2024-04-29 21:00:28 | 2024-04-29 21:31:48 | 0:31:20 | 0:21:05 | 0:10:15 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} | 3 |
Failure Reason:
Command failed on smithi099 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell 4.39 deep-scrub' |
||||||||||||||
| fail | 7679184 |
|
2024-04-29 20:32:52 | 2024-04-29 21:00:28 | 2024-04-29 21:56:02 | 0:55:34 | 0:44:59 | 0:10:35 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679185 |
|
2024-04-29 20:32:53 | 2024-04-29 21:00:29 | 2024-04-29 23:43:19 | 2:42:50 | 2:31:01 | 0:11:49 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714425997.9133282 mon.a (mon.0) 369 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679186 |
|
2024-04-29 20:32:55 | 2024-04-29 21:01:29 | 2024-04-29 21:53:01 | 0:51:32 | 0:37:16 | 0:14:16 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679187 |
|
2024-04-29 20:32:56 | 2024-04-29 21:09:51 | 2024-04-30 00:34:47 | 3:24:56 | 3:11:22 | 0:13:34 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"2024-04-29T21:43:13.475285+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7679188 |
|
2024-04-29 20:32:57 | 2024-04-29 21:14:52 | 2024-04-30 00:02:43 | 2:47:51 | 2:40:00 | 0:07:51 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
"1714426446.1068873 mon.a (mon.0) 659 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| dead | 7679189 |
|
2024-04-29 20:32:58 | 2024-04-29 21:15:52 | 2024-04-30 09:25:04 | 12:09:12 | smithi | main | centos | 8.stream | upgrade/telemetry-upgrade/pacific-x/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks} | 2 | ||
Failure Reason:
hit max job timeout |
||||||||||||||
| pass | 7679190 |
|
2024-04-29 20:33:00 | 2024-04-29 21:15:53 | 2024-04-29 21:53:27 | 0:37:34 | 0:28:23 | 0:09:11 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} | 3 |
| fail | 7679191 |
|
2024-04-29 20:33:01 | 2024-04-29 21:15:53 | 2024-04-29 21:35:23 | 0:19:30 | 0:09:51 | 0:09:39 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/filestore-remove-check/{0-cluster/{openstack start} 1-ceph-install/quincy 2 - upgrade objectstore/filestore-xfs ubuntu_latest} | 1 |
Failure Reason:
"2024-04-29T21:34:05.318581+0000 mon.a (mon.0) 38 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| fail | 7679192 |
|
2024-04-29 20:33:02 | 2024-04-29 21:15:53 | 2024-04-29 22:01:01 | 0:45:08 | 0:37:47 | 0:07:21 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679193 |
|
2024-04-29 20:33:03 | 2024-04-29 21:15:54 | 2024-04-29 23:11:49 | 1:55:55 | 1:48:28 | 0:07:27 | smithi | main | centos | 9.stream | upgrade/reef-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-04-29T21:42:06.592930+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| pass | 7679194 |
|
2024-04-29 20:33:05 | 2024-04-29 21:16:44 | 2024-04-29 21:37:40 | 0:20:56 | 0:12:50 | 0:08:06 | smithi | main | centos | 8.stream | upgrade/cephfs/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-pacific 1-upgrade}} | 1 |
| fail | 7679195 |
|
2024-04-29 20:33:06 | 2024-04-29 21:16:44 | 2024-04-29 22:52:15 | 1:35:31 | 1:27:07 | 0:08:24 | smithi | main | centos | 9.stream | upgrade/quincy-x/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"1714426563.9879057 mon.a (mon.0) 661 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679196 |
|
2024-04-29 20:33:07 | 2024-04-29 21:19:05 | 2024-04-30 00:32:41 | 3:13:36 | 3:04:14 | 0:09:22 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-04-29T21:45:13.951042+0000 mon.a (mon.0) 15 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log |
||||||||||||||
| pass | 7679197 |
|
2024-04-29 20:33:08 | 2024-04-29 21:20:36 | 2024-04-29 22:28:06 | 1:07:30 | 0:57:29 | 0:10:01 | smithi | main | centos | 8.stream | upgrade/cephfs/upgraded_client/{bluestore-bitmap branch/pacific centos_8.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/dbench}} | 2 |
| fail | 7679198 |
|
2024-04-29 20:33:10 | 2024-04-29 21:20:36 | 2024-04-30 00:18:50 | 2:58:14 | 2:45:53 | 0:12:21 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714426961.0562851 mon.a (mon.0) 517 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log |
||||||||||||||
| fail | 7679199 |
|
2024-04-29 20:33:11 | 2024-04-29 21:25:37 | 2024-04-29 22:26:56 | 1:01:19 | 0:45:17 | 0:16:02 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679200 |
|
2024-04-29 20:33:12 | 2024-04-29 21:31:18 | 2024-04-30 01:16:45 | 3:45:27 | 3:35:27 | 0:10:00 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
Command failed (workunit test suites/fsstress.sh) on smithi189 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36c371567dddacf6207ea36f2535396ab31415fc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh' |
||||||||||||||
| fail | 7679201 |
|
2024-04-29 20:33:13 | 2024-04-29 21:31:19 | 2024-04-29 22:27:12 | 0:55:53 | 0:46:06 | 0:09:47 | smithi | main | ubuntu | 22.04 | upgrade/quincy-x/stress-split/{0-distro/ubuntu_22.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} | 2 |
Failure Reason:
timeout expired in wait_until_healthy |
||||||||||||||
| fail | 7679202 |
|
2024-04-29 20:33:15 | 2024-04-29 21:31:19 | 2024-04-30 00:32:33 | 3:01:14 | 2:55:01 | 0:06:13 | smithi | main | centos | 9.stream | upgrade/reef-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"2024-04-29T21:58:40.940007+0000 mon.a (mon.0) 592 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
| pass | 7679203 |
|
2024-04-29 20:33:16 | 2024-04-29 21:31:20 | 2024-04-29 22:28:41 | 0:57:21 | 0:45:46 | 0:11:35 | smithi | main | centos | 8.stream | upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} | 3 |
| fail | 7679204 |
|
2024-04-29 20:33:17 | 2024-04-29 21:31:20 | 2024-04-30 00:11:29 | 2:40:09 | 2:28:49 | 0:11:20 | smithi | main | centos | 9.stream | upgrade/quincy-x/stress-split/{0-distro/centos_9.stream 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} | 2 |
Failure Reason:
"1714427703.6107526 mon.a (mon.0) 670 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log |
||||||||||||||
| fail | 7679205 |
|
2024-04-29 20:33:18 | 2024-04-29 21:36:11 | 2024-04-29 22:22:51 | 0:46:40 | 0:38:09 | 0:08:31 | smithi | main | centos | 9.stream | upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} | 2 |
Failure Reason:
reached maximum tries (51) after waiting for 300 seconds |
||||||||||||||
| fail | 7679206 |
|
2024-04-29 20:33:20 | 2024-04-29 21:37:42 | 2024-04-29 23:53:19 | 2:15:37 | 2:05:59 | 0:09:38 | smithi | main | ubuntu | 22.04 | upgrade/reef-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} | 2 |
Failure Reason:
"2024-04-29T22:18:41.292562+0000 mon.a (mon.0) 401 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log |
||||||||||||||
Pulpito