Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 6535755 2021-11-30 19:40:58 2021-11-30 19:41:42 2021-11-30 20:11:00 0:29:18 0:19:21 0:09:57 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6535756 2021-11-30 19:40:58 2021-11-30 19:41:42 2021-11-30 21:30:54 1:49:12 1:38:53 0:10:19 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535757 2021-11-30 19:40:59 2021-11-30 19:41:43 2021-11-30 20:20:42 0:38:59 0:29:14 0:09:45 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535758 2021-11-30 19:41:00 2021-11-30 19:41:43 2021-11-30 21:30:03 1:48:20 1:36:56 0:11:24 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535759 2021-11-30 19:41:01 2021-11-30 19:41:43 2021-11-30 20:21:00 0:39:17 0:28:59 0:10:18 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 6535760 2021-11-30 19:41:02 2021-11-30 19:41:44 2021-11-30 21:22:54 1:41:10 1:30:54 0:10:16 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:00:13.755236+0000 mon.a (mon.0) 287 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6535761 2021-11-30 19:41:03 2021-11-30 19:41:44 2021-11-30 20:21:15 0:39:31 0:29:41 0:09:50 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535762 2021-11-30 19:41:03 2021-11-30 19:41:45 2021-11-30 21:33:03 1:51:18 1:40:03 0:11:15 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535763 2021-11-30 19:41:04 2021-11-30 19:41:45 2021-11-30 20:21:18 0:39:33 0:28:49 0:10:44 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535764 2021-11-30 19:41:05 2021-11-30 19:41:45 2021-11-30 21:22:56 1:41:11 1:31:12 0:09:59 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535765 2021-11-30 19:41:06 2021-11-30 19:41:46 2021-11-30 20:25:15 0:43:29 0:32:16 0:11:13 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535766 2021-11-30 19:41:07 2021-11-30 19:41:46 2021-11-30 21:25:05 1:43:19 1:31:49 0:11:30 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535767 2021-11-30 19:41:08 2021-11-30 19:41:46 2021-11-30 20:23:36 0:41:50 0:30:55 0:10:55 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535768 2021-11-30 19:41:08 2021-11-30 19:42:07 2021-11-30 21:35:41 1:53:34 1:40:37 0:12:57 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535769 2021-11-30 19:41:09 2021-11-30 19:42:07 2021-11-30 20:22:01 0:39:54 0:29:38 0:10:16 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535770 2021-11-30 19:41:10 2021-11-30 19:42:17 2021-11-30 21:29:38 1:47:21 1:36:35 0:10:46 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
dead 6535771 2021-11-30 19:41:11 2021-11-30 19:57:31 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

pass 6535772 2021-11-30 19:41:12 2021-11-30 19:42:28 2021-11-30 21:26:12 1:43:44 1:31:31 0:12:13 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535773 2021-11-30 19:41:13 2021-11-30 19:44:49 2021-11-30 20:24:25 0:39:36 0:29:17 0:10:19 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535774 2021-11-30 19:41:13 2021-11-30 19:45:09 2021-11-30 21:20:15 1:35:06 1:25:47 0:09:19 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535775 2021-11-30 19:41:14 2021-11-30 19:45:09 2021-11-30 20:28:44 0:43:35 0:33:01 0:10:34 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 6535776 2021-11-30 19:41:15 2021-11-30 19:46:46 2021-11-30 21:21:53 1:35:07 1:23:31 0:11:36 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:07:28.387446+0000 mon.a (mon.0) 126 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6535777 2021-11-30 19:41:16 2021-11-30 19:48:37 2021-11-30 20:28:32 0:39:55 0:30:11 0:09:44 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 6535778 2021-11-30 19:41:17 2021-11-30 19:48:57 2021-11-30 21:22:21 1:33:24 1:23:21 0:10:03 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:07:47.628722+0000 mon.a (mon.0) 126 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6535779 2021-11-30 19:41:18 2021-11-30 19:48:58 2021-11-30 20:18:22 0:29:24 0:19:08 0:10:16 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

Command failed on smithi074 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

fail 6535780 2021-11-30 19:41:18 2021-11-30 19:49:18 2021-11-30 21:18:45 1:29:27 1:19:37 0:09:50 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:07:47.854701+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6535781 2021-11-30 19:41:19 2021-11-30 19:49:29 2021-11-30 20:29:04 0:39:35 0:29:48 0:09:47 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 6535782 2021-11-30 19:41:20 2021-11-30 19:49:49 2021-11-30 20:21:01 0:31:12 0:19:44 0:11:28 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:08:54.956206+0000 mon.a (mon.0) 121 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6535783 2021-11-30 19:41:21 2021-11-30 19:49:49 2021-11-30 20:31:12 0:41:23 0:31:20 0:10:03 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535784 2021-11-30 19:41:22 2021-11-30 19:50:00 2021-11-30 21:28:37 1:38:37 1:27:32 0:11:05 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535785 2021-11-30 19:41:23 2021-11-30 19:51:20 2021-11-30 20:32:33 0:41:13 0:31:23 0:09:50 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535786 2021-11-30 19:41:23 2021-11-30 19:51:20 2021-11-30 21:28:58 1:37:38 1:27:18 0:10:20 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535787 2021-11-30 19:41:24 2021-11-30 19:51:41 2021-11-30 20:35:24 0:43:43 0:33:14 0:10:29 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535788 2021-11-30 19:41:25 2021-11-30 19:52:11 2021-11-30 21:28:57 1:36:46 1:25:01 0:11:45 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535789 2021-11-30 19:41:26 2021-11-30 19:53:22 2021-11-30 20:35:35 0:42:13 0:31:43 0:10:30 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535790 2021-11-30 19:41:27 2021-11-30 19:54:22 2021-11-30 21:22:39 1:28:17 1:18:13 0:10:04 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535791 2021-11-30 19:41:28 2021-11-30 19:55:23 2021-11-30 20:34:53 0:39:30 0:27:48 0:11:42 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
dead 6535792 2021-11-30 19:41:29 2021-11-30 19:57:13 2021-11-30 20:12:42 0:15:29 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

pass 6535793 2021-11-30 19:41:30 2021-11-30 19:57:44 2021-11-30 20:36:16 0:38:32 0:28:06 0:10:26 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 6535794 2021-11-30 19:41:30 2021-11-30 19:58:44 2021-11-30 20:30:37 0:31:53 0:20:11 0:11:42 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2021-11-30T20:18:10.534690+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6535795 2021-11-30 19:41:31 2021-11-30 19:59:25 2021-11-30 20:35:48 0:36:23 0:26:07 0:10:16 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2021-11-30T20:16:58.741638+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6535796 2021-11-30 19:41:32 2021-11-30 20:00:35 2021-11-30 21:46:07 1:45:32 1:35:20 0:10:12 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535797 2021-11-30 19:41:33 2021-11-30 20:00:46 2021-11-30 20:40:53 0:40:07 0:29:22 0:10:45 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535798 2021-11-30 19:41:34 2021-11-30 20:01:36 2021-11-30 21:54:04 1:52:28 1:40:07 0:12:21 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535799 2021-11-30 19:41:35 2021-11-30 20:02:47 2021-11-30 20:41:36 0:38:49 0:26:21 0:12:28 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535800 2021-11-30 19:41:36 2021-11-30 20:03:57 2021-11-30 21:55:27 1:51:30 1:42:18 0:09:12 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535801 2021-11-30 19:41:36 2021-11-30 20:04:18 2021-11-30 20:41:37 0:37:19 0:28:24 0:08:55 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535802 2021-11-30 19:41:37 2021-11-30 20:04:48 2021-11-30 21:58:34 1:53:46 1:41:50 0:11:56 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535803 2021-11-30 19:41:38 2021-11-30 20:04:58 2021-11-30 20:43:22 0:38:24 0:28:21 0:10:03 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535804 2021-11-30 19:41:39 2021-11-30 20:05:09 2021-11-30 21:56:57 1:51:48 1:39:05 0:12:43 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535805 2021-11-30 19:41:40 2021-11-30 20:07:09 2021-11-30 20:46:38 0:39:29 0:28:48 0:10:41 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535806 2021-11-30 19:41:41 2021-11-30 20:07:10 2021-11-30 22:00:45 1:53:35 1:44:07 0:09:28 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535807 2021-11-30 19:41:42 2021-11-30 20:07:50 2021-11-30 20:44:23 0:36:33 0:26:46 0:09:47 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535808 2021-11-30 19:41:42 2021-11-30 20:09:11 2021-11-30 22:00:59 1:51:48 1:41:11 0:10:37 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535809 2021-11-30 19:41:43 2021-11-30 20:09:11 2021-11-30 20:49:58 0:40:47 0:28:53 0:11:54 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535810 2021-11-30 19:41:44 2021-11-30 20:10:41 2021-11-30 21:42:01 1:31:20 1:20:51 0:10:29 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
dead 6535811 2021-11-30 19:41:45 2021-11-30 20:11:02 2021-11-30 20:27:59 0:16:57 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

pass 6535812 2021-11-30 19:41:46 2021-11-30 20:13:02 2021-11-30 21:57:00 1:43:58 1:28:50 0:15:08 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535813 2021-11-30 19:41:47 2021-11-30 20:54:25 1645 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535814 2021-11-30 19:41:48 2021-11-30 20:17:14 2021-11-30 21:55:47 1:38:33 1:26:36 0:11:57 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
fail 6535815 2021-11-30 19:41:49 2021-11-30 20:22:11 2021-11-30 20:51:39 0:29:28 0:18:22 0:11:06 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6535816 2021-11-30 19:41:50 2021-11-30 20:23:42 2021-11-30 22:03:25 1:39:43 1:28:52 0:10:51 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535817 2021-11-30 19:41:50 2021-11-30 20:24:32 2021-11-30 21:02:27 0:37:55 0:28:17 0:09:38 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535818 2021-11-30 19:41:51 2021-11-30 20:25:23 2021-11-30 22:00:24 1:35:01 1:24:03 0:10:58 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535819 2021-11-30 19:41:52 2021-11-30 20:26:53 2021-11-30 21:06:20 0:39:27 0:29:23 0:10:04 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535820 2021-11-30 19:41:53 2021-11-30 20:26:53 2021-11-30 21:57:20 1:30:27 1:18:54 0:11:33 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
dead 6535821 2021-11-30 19:41:54 2021-11-30 20:27:34 2021-11-30 20:43:11 0:15:37 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

pass 6535822 2021-11-30 19:41:54 2021-11-30 20:28:14 2021-11-30 22:10:52 1:42:38 1:30:59 0:11:39 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535823 2021-11-30 19:41:55 2021-11-30 20:28:35 2021-11-30 21:08:17 0:39:42 0:28:58 0:10:44 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535824 2021-11-30 19:41:56 2021-11-30 20:28:45 2021-11-30 22:09:07 1:40:22 1:29:27 0:10:55 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
pass 6535825 2021-11-30 19:41:57 2021-11-30 20:29:05 2021-11-30 21:08:05 0:39:00 0:28:06 0:10:54 smithi master centos 8.3 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-ioc-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
pass 6535826 2021-11-30 19:41:58 2021-11-30 20:30:46 2021-11-30 22:02:30 1:31:44 1:21:58 0:09:46 smithi master centos 8.0 rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2