Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7378464 2023-08-24 09:25:31 2023-08-24 09:30:35 2023-08-24 10:04:17 0:33:42 0:22:28 0:11:14 smithi main centos 8.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rados_api_tests} 2
Failure Reason:

Command failed on smithi111 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378465 2023-08-24 09:25:32 2023-08-24 09:30:35 2023-08-24 10:05:44 0:35:09 0:25:07 0:10:02 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_read} 1
Failure Reason:

Command failed on smithi134 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378466 2023-08-24 09:25:33 2023-08-24 09:30:36 2023-08-24 10:05:27 0:34:51 0:25:44 0:09:07 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests} 1
Failure Reason:

Command failed on smithi003 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378467 2023-08-24 09:25:34 2023-08-24 09:30:36 2023-08-24 10:03:39 0:33:03 0:23:49 0:09:14 smithi main centos 8.stream crimson-rados/singleton/{all/osd-backfill crimson-supported-all-distro/centos_8 crimson_qa_overrides objectstore/bluestore rados} 1
Failure Reason:

Command failed on smithi045 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378468 2023-08-24 09:25:35 2023-08-24 09:30:36 2023-08-24 10:05:46 0:35:10 0:25:32 0:09:38 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} 2
Failure Reason:

Command failed on smithi136 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378469 2023-08-24 09:25:36 2023-08-24 09:30:37 2023-08-24 10:03:30 0:32:53 0:22:12 0:10:41 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_rw} 1
Failure Reason:

"2023-08-24T09:51:31.330140+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378470 2023-08-24 09:25:37 2023-08-24 09:32:27 2023-08-24 10:06:05 0:33:38 0:22:50 0:10:48 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} 2
Failure Reason:

Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378471 2023-08-24 09:25:38 2023-08-24 09:33:08 2023-08-24 10:08:15 0:35:07 0:25:13 0:09:54 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_read} 1
Failure Reason:

Command failed on smithi073 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378472 2023-08-24 09:25:39 2023-08-24 09:33:08 2023-08-24 10:02:12 0:29:04 0:19:50 0:09:14 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} 1
Failure Reason:

"2023-08-24T09:52:53.970937+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive (PG_AVAILABILITY)" in cluster log

fail 7378473 2023-08-24 09:25:40 2023-08-24 09:33:39 2023-08-24 10:09:32 0:35:53 0:25:38 0:10:15 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench-high-concurrency} 2
Failure Reason:

Command failed on smithi071 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378474 2023-08-24 09:25:41 2023-08-24 09:33:59 2023-08-24 10:07:15 0:33:16 0:22:27 0:10:49 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_rw} 1
Failure Reason:

"2023-08-24T09:54:15.548502+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378475 2023-08-24 09:25:42 2023-08-24 09:33:59 2023-08-24 10:07:13 0:33:14 0:22:30 0:10:44 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} 2
Failure Reason:

Command failed on smithi078 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378476 2023-08-24 09:25:43 2023-08-24 09:34:00 2023-08-24 10:13:00 0:39:00 0:28:36 0:10:24 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_write} 1
Failure Reason:

Command failed on smithi019 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378477 2023-08-24 09:25:44 2023-08-24 09:34:10 2023-08-24 10:06:06 0:31:56 0:18:51 0:13:05 smithi main centos 9.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_python} 2
Failure Reason:

Command failed on smithi163 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378478 2023-08-24 09:25:45 2023-08-24 09:36:21 2023-08-24 10:11:05 0:34:44 0:25:30 0:09:14 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} 1
Failure Reason:

Command failed on smithi005 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378479 2023-08-24 09:25:46 2023-08-24 09:36:21 2023-08-24 10:13:26 0:37:05 0:26:41 0:10:24 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-balanced} 2
Failure Reason:

Command failed on smithi037 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378480 2023-08-24 09:25:47 2023-08-24 09:36:31 2023-08-24 10:07:11 0:30:40 0:22:32 0:08:08 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_rand_read} 1
Failure Reason:

"2023-08-24T09:55:29.968720+0000 mon.a (mon.0) 116 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log

fail 7378481 2023-08-24 09:25:48 2023-08-24 09:36:32 2023-08-24 10:09:22 0:32:50 0:22:50 0:10:00 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-localized} 2
Failure Reason:

Command failed on smithi129 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378482 2023-08-24 09:25:49 2023-08-24 09:36:32 2023-08-24 10:14:22 0:37:50 0:25:14 0:12:36 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_seq_read} 1
Failure Reason:

Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378483 2023-08-24 09:25:50 2023-08-24 09:39:23 2023-08-24 10:08:31 0:29:08 0:19:13 0:09:55 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_lock_and_fence} 1
Failure Reason:

"2023-08-24T09:58:32.570706+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378484 2023-08-24 09:25:52 2023-08-24 09:39:23 2023-08-24 10:11:02 0:31:39 0:22:15 0:09:24 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_rand_read} 1
Failure Reason:

"2023-08-24T09:59:03.469013+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378485 2023-08-24 09:25:53 2023-08-24 09:39:54 2023-08-24 10:16:23 0:36:29 0:25:25 0:11:04 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} 2
Failure Reason:

Command failed on smithi119 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378486 2023-08-24 09:25:54 2023-08-24 09:41:14 2023-08-24 10:16:16 0:35:02 0:25:07 0:09:55 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_seq_read} 1
Failure Reason:

Command failed on smithi121 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378487 2023-08-24 09:25:55 2023-08-24 09:41:15 2023-08-24 10:15:26 0:34:11 0:23:00 0:11:11 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} 2
Failure Reason:

Command failed on smithi052 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378488 2023-08-24 09:25:56 2023-08-24 09:42:15 2023-08-24 10:15:30 0:33:15 0:24:37 0:08:38 smithi main centos 8.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/readwrite} 2
Failure Reason:

Command failed on smithi006 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378489 2023-08-24 09:25:57 2023-08-24 09:42:36 2023-08-24 10:16:11 0:33:35 0:23:41 0:09:54 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests} 1
Failure Reason:

Command failed on smithi088 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378490 2023-08-24 09:25:58 2023-08-24 09:42:36 2023-08-24 10:14:35 0:31:59 0:22:41 0:09:18 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_write} 1
Failure Reason:

"2023-08-24T10:02:59.847754+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378491 2023-08-24 09:25:59 2023-08-24 09:43:27 2023-08-24 10:18:56 0:35:29 0:25:56 0:09:33 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} 2
Failure Reason:

Command failed on smithi081 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378492 2023-08-24 09:26:00 2023-08-24 09:43:47 2023-08-24 10:20:27 0:36:40 0:26:54 0:09:46 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_omap_write} 1
Failure Reason:

Command failed on smithi012 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378493 2023-08-24 09:26:01 2023-08-24 09:43:48 2023-08-24 10:17:51 0:34:03 0:22:47 0:11:16 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} 2
Failure Reason:

Command failed on smithi089 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378494 2023-08-24 09:26:02 2023-08-24 09:44:48 2023-08-24 10:11:37 0:26:49 0:18:53 0:07:56 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests_old_format} 1
Failure Reason:

"2023-08-24T10:03:11.880397+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378495 2023-08-24 09:26:03 2023-08-24 09:44:48 2023-08-24 10:16:43 0:31:55 0:22:38 0:09:17 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_fio} 1
Failure Reason:

"2023-08-24T10:05:09.644770+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378496 2023-08-24 09:26:04 2023-08-24 09:45:59 2023-08-24 10:23:19 0:37:20 0:26:56 0:10:24 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/write_fadvise_dontneed} 2
Failure Reason:

Command failed on smithi032 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378497 2023-08-24 09:26:05 2023-08-24 09:46:09 2023-08-24 10:21:13 0:35:04 0:25:19 0:09:45 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_radosbench} 1
Failure Reason:

Command failed on smithi086 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378498 2023-08-24 09:26:06 2023-08-24 09:46:10 2023-08-24 10:15:18 0:29:08 0:19:52 0:09:16 smithi main centos 9.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} 2
Failure Reason:

Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378499 2023-08-24 09:26:07 2023-08-24 09:46:20 2023-08-24 10:20:32 0:34:12 0:23:07 0:11:05 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_read} 1
Failure Reason:

"2023-08-24T10:08:08.469868+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378500 2023-08-24 09:26:08 2023-08-24 09:46:21 2023-08-24 10:15:37 0:29:16 0:19:40 0:09:36 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_api_tests} 1
Failure Reason:

Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378501 2023-08-24 09:26:09 2023-08-24 09:47:01 2023-08-24 10:16:06 0:29:05 0:19:51 0:09:14 smithi main centos 9.stream crimson-rados/singleton/{all/osd-backfill crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/bluestore rados} 1
Failure Reason:

"2023-08-24T10:07:26.705331+0000 mon.a (mon.0) 94 : cluster [WRN] Health check failed: 2 slow ops, oldest one blocked for 32 sec, mon.a has slow ops (SLOW_OPS)" in cluster log

fail 7378502 2023-08-24 09:26:10 2023-08-24 09:47:01 2023-08-24 10:18:40 0:31:39 0:22:36 0:09:03 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} 2
Failure Reason:

Command failed on smithi084 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378503 2023-08-24 09:26:11 2023-08-24 09:47:42 2023-08-24 10:22:51 0:35:09 0:25:39 0:09:30 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_rw} 1
Failure Reason:

Command failed on smithi179 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378504 2023-08-24 09:26:12 2023-08-24 09:47:42 2023-08-24 10:26:12 0:38:30 0:26:49 0:11:41 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} 2
Failure Reason:

Command failed on smithi026 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378505 2023-08-24 09:26:13 2023-08-24 09:48:53 2023-08-24 10:23:08 0:34:15 0:22:37 0:11:38 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_read} 1
Failure Reason:

"2023-08-24T10:10:26.385502+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378506 2023-08-24 09:26:14 2023-08-24 09:48:53 2023-08-24 10:26:50 0:37:57 0:24:50 0:13:07 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} 1
Failure Reason:

Command failed on smithi023 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378507 2023-08-24 09:26:15 2023-08-24 09:50:44 2023-08-24 10:24:07 0:33:23 0:22:33 0:10:50 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench-high-concurrency} 2
Failure Reason:

Command failed on smithi172 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378508 2023-08-24 09:26:16 2023-08-24 09:52:55 2023-08-24 10:27:48 0:34:53 0:26:01 0:08:52 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_rw} 1
Failure Reason:

Command failed on smithi131 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378509 2023-08-24 09:26:17 2023-08-24 09:52:55 2023-08-24 10:28:14 0:35:19 0:25:56 0:09:23 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} 2
Failure Reason:

Command failed on smithi155 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json'

fail 7378510 2023-08-24 09:26:18 2023-08-24 09:53:05 2023-08-24 10:24:14 0:31:09 0:22:17 0:08:52 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_write} 1
Failure Reason:

"2023-08-24T10:12:25.510448+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378511 2023-08-24 09:26:19 2023-08-24 09:53:06 2023-08-24 10:24:35 0:31:29 0:21:12 0:10:17 smithi main centos 8.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rados_python} 2
Failure Reason:

Command failed on smithi117 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378512 2023-08-24 09:26:20 2023-08-24 09:53:26 2023-08-24 10:22:44 0:29:18 0:19:22 0:09:56 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} 1
Failure Reason:

"2023-08-24T10:13:34.250010+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378513 2023-08-24 09:26:21 2023-08-24 09:53:27 2023-08-24 10:26:36 0:33:09 0:22:48 0:10:21 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-balanced} 2
Failure Reason:

Command failed on smithi053 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378514 2023-08-24 09:26:22 2023-08-24 09:53:27 2023-08-24 10:28:27 0:35:00 0:25:56 0:09:04 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_rand_read} 1
Failure Reason:

Command failed on smithi116 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378515 2023-08-24 09:26:23 2023-08-24 09:53:27 2023-08-24 10:29:15 0:35:48 0:25:53 0:09:55 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-localized} 2
Failure Reason:

Command failed on smithi082 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378516 2023-08-24 09:26:24 2023-08-24 09:54:18 2023-08-24 10:27:21 0:33:03 0:24:24 0:08:39 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_seq_read} 1
Failure Reason:

"2023-08-24T10:13:47.530442+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378517 2023-08-24 09:26:25 2023-08-24 09:54:18 2023-08-24 10:30:07 0:35:49 0:24:22 0:11:27 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_lock_and_fence} 1
Failure Reason:

Command failed on smithi090 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378518 2023-08-24 09:26:26 2023-08-24 09:54:19 2023-08-24 10:31:08 0:36:49 0:26:53 0:09:56 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_rand_read} 1
Failure Reason:

Command failed on smithi055 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378519 2023-08-24 09:26:27 2023-08-24 09:54:19 2023-08-24 10:27:22 0:33:03 0:22:44 0:10:19 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} 2
Failure Reason:

Command failed on smithi112 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378520 2023-08-24 09:26:28 2023-08-24 09:54:19 2023-08-24 10:27:54 0:33:35 0:22:23 0:11:12 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_seq_read} 1
Failure Reason:

"2023-08-24T10:14:30.346032+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378521 2023-08-24 09:26:29 2023-08-24 09:55:00 2023-08-24 10:30:33 0:35:33 0:25:44 0:09:49 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} 2
Failure Reason:

Command failed on smithi101 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378522 2023-08-24 09:26:30 2023-08-24 09:55:20 2023-08-24 10:25:24 0:30:04 0:19:41 0:10:23 smithi main centos 9.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/readwrite} 2
Failure Reason:

"2023-08-24T10:16:24.151989+0000 mon.a (mon.0) 133 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378523 2023-08-24 09:26:31 2023-08-24 09:56:21 2023-08-24 10:25:48 0:29:27 0:18:31 0:10:56 smithi main centos 9.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests} 1
Failure Reason:

"2023-08-24T10:15:28.166437+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 9 pgs inactive (PG_AVAILABILITY)" in cluster log

fail 7378524 2023-08-24 09:26:32 2023-08-24 09:56:21 2023-08-24 10:31:49 0:35:28 0:25:12 0:10:16 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_write} 1
Failure Reason:

Command failed on smithi159 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378525 2023-08-24 09:26:33 2023-08-24 09:56:32 2023-08-24 10:30:46 0:34:14 0:23:17 0:10:57 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} 2
Failure Reason:

Command failed on smithi033 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378526 2023-08-24 09:26:34 2023-08-24 09:57:22 2023-08-24 10:28:07 0:30:45 0:22:31 0:08:14 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_omap_write} 1
Failure Reason:

"2023-08-24T10:16:34.262534+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 7378527 2023-08-24 09:26:35 2023-08-24 09:57:23 2023-08-24 10:32:34 0:35:11 0:25:28 0:09:43 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} 2
Failure Reason:

Command failed on smithi149 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378528 2023-08-24 09:26:36 2023-08-24 09:57:33 2023-08-24 10:30:39 0:33:06 0:24:01 0:09:05 smithi main centos 8.stream crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests_old_format} 1
Failure Reason:

Command failed on smithi046 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378529 2023-08-24 09:26:37 2023-08-24 09:57:33 2023-08-24 10:34:41 0:37:08 0:25:16 0:11:52 smithi main centos 8.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_8 crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_fio} 1
Failure Reason:

Command failed on smithi158 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378530 2023-08-24 09:26:38 2023-08-24 09:58:44 2023-08-24 10:30:57 0:32:13 0:22:21 0:09:52 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/write_fadvise_dontneed} 2
Failure Reason:

Command failed on smithi097 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

fail 7378531 2023-08-24 09:26:39 2023-08-24 09:59:54 2023-08-24 10:30:57 0:31:03 0:22:18 0:08:45 smithi main centos 9.stream crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_radosbench} 1
Failure Reason:

"2023-08-24T10:19:07.288599+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log