Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7129750 2023-01-19 03:08:26 2023-01-19 03:09:12 2023-01-19 03:34:51 0:25:39 0:16:01 0:09:38 smithi main ubuntu 20.04 rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Cannot create secret

fail 7129751 2023-01-19 03:08:27 2023-01-19 03:09:12 2023-01-19 03:26:40 0:17:28 0:06:47 0:10:41 smithi main rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c55bec9a159ae6f4d6e6c5050fddf68f4b1c4a3

fail 7129752 2023-01-19 03:08:28 2023-01-19 03:09:12 2023-01-19 03:44:06 0:34:54 0:28:56 0:05:58 smithi main rhel 8.4 rgw/lifecycle/{cluster overrides supported-random-distro$/{rhel_8} tasks/rgw_s3tests} 1
Failure Reason:

"2023-01-19T03:31:11.915697+0000 mon.a (mon.0) 166 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129753 2023-01-19 03:08:29 2023-01-19 03:09:13 2023-01-19 03:28:58 0:19:45 0:10:24 0:09:21 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota ubuntu_latest} 2
fail 7129754 2023-01-19 03:08:30 2023-01-19 03:09:13 2023-01-19 04:14:11 1:04:58 0:46:40 0:18:18 smithi main ubuntu 20.04 rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{ubuntu_latest} tasks/test_multi} 2
Failure Reason:

rgw multisite test failures

fail 7129755 2023-01-19 03:08:31 2023-01-19 03:09:13 2023-01-19 03:57:43 0:48:30 0:33:34 0:14:56 smithi main centos 8.stream rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} 2
Failure Reason:

"2023-01-19T03:32:38.028516+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129756 2023-01-19 03:08:32 2023-01-19 03:09:14 2023-01-19 03:34:17 0:25:03 0:13:41 0:11:22 smithi main centos 8.stream rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} 2
pass 7129757 2023-01-19 03:08:33 2023-01-19 03:09:14 2023-01-19 03:47:14 0:38:00 0:27:40 0:10:20 smithi main ubuntu 20.04 rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}} 2
fail 7129758 2023-01-19 03:08:34 2023-01-19 03:09:14 2023-01-19 03:48:02 0:38:48 0:22:52 0:15:56 smithi main ubuntu 20.04 rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} 1
Failure Reason:

Command failed on smithi155 with status 1: 'cd /home/ubuntu/cephtest/tempest && /home/ubuntu/cephtest/tox-venv/bin/tox -e venv --notest'

pass 7129759 2023-01-19 03:08:35 2023-01-19 03:09:24 2023-01-19 03:34:09 0:24:45 0:13:47 0:10:58 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} 2
fail 7129760 2023-01-19 03:08:36 2023-01-19 03:09:25 2023-01-19 06:12:43 3:03:18 2:52:49 0:10:29 smithi main centos 8.stream rgw/tools/{centos_latest cluster tasks} 1
Failure Reason:

"2023-01-19T03:36:22.863073+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

fail 7129761 2023-01-19 03:08:37 2023-01-19 03:09:25 2023-01-19 03:52:06 0:42:41 0:30:51 0:11:50 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:31:19.750018+0000 mgr.y (mgr.4112) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129762 2023-01-19 03:08:38 2023-01-19 03:09:25 2023-01-19 03:44:56 0:35:31 0:18:54 0:16:37 smithi main ubuntu 20.04 rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} 2
fail 7129763 2023-01-19 03:08:39 2023-01-19 03:09:25 2023-01-19 04:20:17 1:10:52 0:57:27 0:13:25 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:31:01.213272+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129764 2023-01-19 03:08:41 2023-01-19 03:09:26 2023-01-19 03:37:56 0:28:30 0:11:34 0:16:56 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_multipart_upload ubuntu_latest} 2
fail 7129765 2023-01-19 03:08:42 2023-01-19 03:09:26 2023-01-19 03:53:10 0:43:44 0:32:02 0:11:42 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:29:59.438405+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129766 2023-01-19 03:08:43 2023-01-19 03:09:26 2023-01-19 04:20:20 1:10:54 0:56:11 0:14:43 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:33:06.923809+0000 mgr.x (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129767 2023-01-19 03:08:44 2023-01-19 03:09:27 2023-01-19 03:34:15 0:24:48 0:09:28 0:15:20 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi122 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

fail 7129768 2023-01-19 03:08:45 2023-01-19 03:09:27 2023-01-19 03:53:16 0:43:49 0:32:45 0:11:04 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:30:32.542523+0000 mgr.y (mgr.4109) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129769 2023-01-19 03:08:46 2023-01-19 03:09:27 2023-01-19 03:37:06 0:27:39 0:14:10 0:13:29 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_s3tests ubuntu_latest} 2
fail 7129770 2023-01-19 03:08:47 2023-01-19 03:09:27 2023-01-19 04:21:23 1:11:56 0:58:27 0:13:29 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:32:25.185911+0000 mgr.x (mgr.4117) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129771 2023-01-19 03:08:48 2023-01-19 03:09:28 2023-01-19 03:39:12 0:29:44 0:14:02 0:15:42 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} 2
fail 7129772 2023-01-19 03:08:49 2023-01-19 03:09:28 2023-01-19 03:54:21 0:44:53 0:34:30 0:10:23 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:29:36.257747+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129773 2023-01-19 03:08:50 2023-01-19 03:09:28 2023-01-19 03:37:05 0:27:37 0:12:00 0:15:37 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_user_quota ubuntu_latest} 2
fail 7129774 2023-01-19 03:08:51 2023-01-19 03:09:28 2023-01-19 04:09:46 1:00:18 0:48:17 0:12:01 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:30:30.637888+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129775 2023-01-19 03:08:52 2023-01-19 03:09:29 2023-01-19 03:37:16 0:27:47 0:17:40 0:10:07 smithi main rhel 8.4 rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} 1
Failure Reason:

"2023-01-19T03:34:24.330121+0000 mon.a (mon.0) 181 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129776 2023-01-19 03:08:53 2023-01-19 03:09:29 2023-01-19 03:34:05 0:24:36 0:10:40 0:13:56 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7129777 2023-01-19 03:08:54 2023-01-19 03:09:29 2023-01-19 03:36:55 0:27:26 0:13:42 0:13:44 smithi main centos 8.stream rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} 2
fail 7129778 2023-01-19 03:08:55 2023-01-19 03:09:30 2023-01-19 03:59:52 0:50:22 0:35:28 0:14:54 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:33:09.632642+0000 mgr.y (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129779 2023-01-19 03:08:56 2023-01-19 03:09:30 2023-01-19 04:11:57 1:02:27 0:49:08 0:13:19 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:33:00.013593+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129780 2023-01-19 03:08:57 2023-01-19 03:09:30 2023-01-19 03:30:53 0:21:23 0:10:04 0:11:19 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} 2
fail 7129781 2023-01-19 03:08:58 2023-01-19 03:09:30 2023-01-19 03:56:44 0:47:14 0:34:51 0:12:23 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:38:47.649989+0000 mon.a (mon.0) 444 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

fail 7129782 2023-01-19 03:08:59 2023-01-19 03:09:31 2023-01-19 04:15:11 1:05:40 0:51:43 0:13:57 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:32:27.216423+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129783 2023-01-19 03:09:00 2023-01-19 03:09:31 2023-01-19 03:39:47 0:30:16 0:08:56 0:21:20 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi159 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

pass 7129784 2023-01-19 03:09:01 2023-01-19 03:19:43 2023-01-19 04:02:28 0:42:45 0:25:58 0:16:47 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} 2
fail 7129785 2023-01-19 03:09:02 2023-01-19 03:26:44 2023-01-19 04:06:14 0:39:30 0:29:29 0:10:01 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:44:26.643133+0000 mgr.x (mgr.4114) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129786 2023-01-19 03:09:03 2023-01-19 03:29:04 2023-01-19 03:55:36 0:26:32 0:14:30 0:12:02 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} 2
fail 7129787 2023-01-19 03:09:04 2023-01-19 03:30:55 2023-01-19 04:41:19 1:10:24 0:59:47 0:10:37 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:49:48.505190+0000 mgr.x (mgr.4100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129788 2023-01-19 03:09:05 2023-01-19 03:34:16 2023-01-19 04:11:04 0:36:48 0:29:41 0:07:07 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:49:42.148397+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129789 2023-01-19 03:09:06 2023-01-19 03:34:16 2023-01-19 03:54:49 0:20:33 0:09:19 0:11:14 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} 2
fail 7129790 2023-01-19 03:09:07 2023-01-19 03:34:16 2023-01-19 04:41:49 1:07:33 0:59:42 0:07:51 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:50:30.027242+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129791 2023-01-19 03:09:08 2023-01-19 03:34:26 2023-01-19 03:55:39 0:21:13 0:14:44 0:06:29 smithi main centos 8.stream rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} 1
pass 7129792 2023-01-19 03:09:09 2023-01-19 03:34:57 2023-01-19 03:56:51 0:21:54 0:10:25 0:11:29 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7129793 2023-01-19 03:09:10 2023-01-19 03:36:57 2023-01-19 03:58:59 0:22:02 0:11:12 0:10:50 smithi main ubuntu 20.04 rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} 2
fail 7129794 2023-01-19 03:09:11 2023-01-19 03:37:08 2023-01-19 04:14:29 0:37:21 0:29:57 0:07:24 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:59:57.930420+0000 mon.a (mon.0) 498 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

fail 7129795 2023-01-19 03:09:13 2023-01-19 04:45:51 3576 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T03:54:51.092910+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129796 2023-01-19 03:09:14 2023-01-19 03:59:12 560 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} 2
pass 7129797 2023-01-19 03:09:15 2023-01-19 03:39:19 2023-01-19 04:03:46 0:24:27 0:12:50 0:11:37 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} 2
fail 7129798 2023-01-19 03:09:16 2023-01-19 03:39:49 2023-01-19 04:21:07 0:41:18 0:30:33 0:10:45 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T03:59:28.878818+0000 mgr.x (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129799 2023-01-19 03:09:17 2023-01-19 03:44:10 2023-01-19 04:44:07 0:59:57 0:51:34 0:08:23 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:00:56.846425+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129800 2023-01-19 03:09:18 2023-01-19 03:45:01 2023-01-19 04:07:35 0:22:34 0:08:43 0:13:51 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi154 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

fail 7129801 2023-01-19 03:09:19 2023-01-19 03:47:21 2023-01-19 04:29:35 0:42:14 0:30:12 0:12:02 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:08:02.352200+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129802 2023-01-19 03:09:20 2023-01-19 03:52:14 2023-01-19 04:17:37 0:25:23 0:13:18 0:12:05 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_s3tests ubuntu_latest} 2
fail 7129803 2023-01-19 03:09:21 2023-01-19 03:53:14 2023-01-19 04:50:31 0:57:17 0:50:22 0:06:55 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:09:19.610396+0000 mgr.y (mgr.4118) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129804 2023-01-19 03:09:22 2023-01-19 03:53:24 2023-01-19 04:33:57 0:40:33 0:31:38 0:08:55 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:11:00.179068+0000 mgr.x (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129805 2023-01-19 03:09:23 2023-01-19 03:54:25 2023-01-19 04:15:08 0:20:43 0:09:48 0:10:55 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_user_quota ubuntu_latest} 2
fail 7129806 2023-01-19 03:09:24 2023-01-19 03:54:55 2023-01-19 04:50:10 0:55:15 0:47:27 0:07:48 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:17:51.648569+0000 mon.a (mon.0) 434 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129807 2023-01-19 03:09:25 2023-01-19 03:55:38 2023-01-19 04:18:44 0:23:06 0:16:26 0:06:40 smithi main rhel 8.4 rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} 1
fail 7129808 2023-01-19 03:09:26 2023-01-19 03:55:38 2023-01-19 04:15:27 0:19:49 0:06:53 0:12:56 smithi main rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c55bec9a159ae6f4d6e6c5050fddf68f4b1c4a3

pass 7129809 2023-01-19 03:09:27 2023-01-19 03:56:49 2023-01-19 04:16:50 0:20:01 0:10:08 0:09:53 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7129810 2023-01-19 03:09:28 2023-01-19 03:56:59 2023-01-19 04:41:25 0:44:26 0:37:11 0:07:15 smithi main centos 8.stream rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} 2
pass 7129811 2023-01-19 03:09:29 2023-01-19 03:57:50 2023-01-19 04:20:05 0:22:15 0:14:35 0:07:40 smithi main rhel 8.4 rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} 2
pass 7129812 2023-01-19 03:09:30 2023-01-19 03:59:00 2023-01-19 04:23:24 0:24:24 0:14:09 0:10:15 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} 2
fail 7129813 2023-01-19 03:09:31 2023-01-19 03:59:21 2023-01-19 04:24:44 0:25:23 0:17:52 0:07:31 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
fail 7129814 2023-01-19 03:09:32 2023-01-19 04:00:02 2023-01-19 05:05:34 1:05:32 0:56:13 0:09:19 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:17:54.659329+0000 mgr.y (mgr.4115) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129815 2023-01-19 03:09:33 2023-01-19 04:02:32 2023-01-19 04:24:06 0:21:34 0:09:51 0:11:43 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_multipart_upload ubuntu_latest} 2
fail 7129816 2023-01-19 03:09:34 2023-01-19 04:03:53 2023-01-19 04:45:21 0:41:28 0:31:31 0:09:57 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:21:53.533874+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129817 2023-01-19 03:09:35 2023-01-19 04:06:23 2023-01-19 05:10:47 1:04:24 0:56:27 0:07:57 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:23:00.373225+0000 mgr.x (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129818 2023-01-19 03:09:36 2023-01-19 04:07:44 2023-01-19 04:28:09 0:20:25 0:08:19 0:12:06 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi053 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

fail 7129819 2023-01-19 03:09:37 2023-01-19 04:09:55 2023-01-19 04:51:50 0:41:55 0:33:31 0:08:24 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:26:45.513584+0000 mgr.y (mgr.4107) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129820 2023-01-19 03:09:38 2023-01-19 04:11:05 2023-01-19 04:37:50 0:26:45 0:14:46 0:11:59 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_s3tests ubuntu_latest} 2
fail 7129821 2023-01-19 03:09:39 2023-01-19 04:12:06 2023-01-19 05:13:14 1:01:08 0:54:53 0:06:15 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:27:30.527878+0000 mgr.y (mgr.4110) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129822 2023-01-19 03:09:40 2023-01-19 04:12:16 2023-01-19 04:38:03 0:25:47 0:13:17 0:12:30 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} 2
fail 7129823 2023-01-19 03:09:41 2023-01-19 04:14:16 2023-01-19 04:55:53 0:41:37 0:34:04 0:07:33 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:37:21.268307+0000 mon.a (mon.0) 477 : cluster [WRN] Health check failed: 2 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129824 2023-01-19 03:09:42 2023-01-19 04:14:37 2023-01-19 04:35:19 0:20:42 0:09:24 0:11:18 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_user_quota ubuntu_latest} 2
fail 7129825 2023-01-19 03:09:43 2023-01-19 04:15:17 2023-01-19 05:08:43 0:53:26 0:47:19 0:06:07 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:31:12.141262+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129826 2023-01-19 03:09:44 2023-01-19 04:15:17 2023-01-19 04:36:12 0:20:55 0:15:09 0:05:46 smithi main centos 8.stream rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} 1
pass 7129827 2023-01-19 03:09:45 2023-01-19 04:15:18 2023-01-19 04:38:47 0:23:29 0:10:30 0:12:59 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7129828 2023-01-19 03:09:46 2023-01-19 04:15:38 2023-01-19 04:38:03 0:22:25 0:14:47 0:07:38 smithi main rhel 8.4 rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} 2
fail 7129829 2023-01-19 03:09:47 2023-01-19 04:16:58 2023-01-19 04:57:33 0:40:35 0:32:02 0:08:33 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:34:08.237706+0000 mgr.x (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129830 2023-01-19 03:09:48 2023-01-19 04:17:39 2023-01-19 05:15:14 0:57:35 0:47:23 0:10:12 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:42:52.237299+0000 mon.a (mon.0) 474 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129831 2023-01-19 03:09:49 2023-01-19 04:20:09 2023-01-19 04:40:03 0:19:54 0:09:08 0:10:46 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} 2
fail 7129832 2023-01-19 03:09:50 2023-01-19 04:20:20 2023-01-19 04:59:49 0:39:29 0:32:39 0:06:50 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:36:20.465555+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129833 2023-01-19 03:09:51 2023-01-19 04:20:30 2023-01-19 05:16:09 0:55:39 0:47:42 0:07:57 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:36:27.948909+0000 mgr.x (mgr.4101) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129834 2023-01-19 03:09:52 2023-01-19 04:21:10 2023-01-19 04:41:15 0:20:05 0:08:57 0:11:08 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi077 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

pass 7129835 2023-01-19 03:09:53 2023-01-19 04:21:31 2023-01-19 04:51:19 0:29:48 0:17:22 0:12:26 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} 2
fail 7129836 2023-01-19 03:09:54 2023-01-19 04:23:31 2023-01-19 05:01:47 0:38:16 0:29:49 0:08:27 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:40:07.904311+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129837 2023-01-19 03:09:55 2023-01-19 04:24:12 2023-01-19 04:48:33 0:24:21 0:13:47 0:10:34 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} 2
fail 7129838 2023-01-19 03:09:56 2023-01-19 04:24:52 2023-01-19 05:33:36 1:08:44 0:58:33 0:10:11 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:44:23.428524+0000 mgr.y (mgr.4104) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129839 2023-01-19 03:09:57 2023-01-19 04:28:13 2023-01-19 05:09:04 0:40:51 0:31:26 0:09:25 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:45:30.461554+0000 mgr.y (mgr.4112) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129840 2023-01-19 03:09:58 2023-01-19 04:29:43 2023-01-19 04:54:43 0:25:00 0:09:57 0:15:03 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} 2
fail 7129841 2023-01-19 03:09:59 2023-01-19 04:34:04 2023-01-19 05:40:47 1:06:43 0:57:26 0:09:17 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:58:17.960153+0000 mon.a (mon.0) 546 : cluster [WRN] Health check failed: 1 mgr modules have recently crashed (RECENT_MGR_MODULE_CRASH)" in cluster log

pass 7129842 2023-01-19 03:10:00 2023-01-19 04:35:25 2023-01-19 04:58:44 0:23:19 0:15:34 0:07:45 smithi main centos 8.stream rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} 1
pass 7129843 2023-01-19 03:10:01 2023-01-19 04:35:25 2023-01-19 04:57:55 0:22:30 0:10:04 0:12:26 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} 2
pass 7129844 2023-01-19 03:10:02 2023-01-19 04:37:56 2023-01-19 04:57:11 0:19:15 0:13:03 0:06:12 smithi main centos 8.stream rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} 2
fail 7129845 2023-01-19 03:10:03 2023-01-19 04:38:07 2023-01-19 05:15:04 0:36:57 0:30:20 0:06:37 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:53:21.452256+0000 mgr.x (mgr.4102) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129846 2023-01-19 03:10:04 2023-01-19 04:38:07 2023-01-19 05:45:25 1:07:18 0:59:01 0:08:17 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:56:24.158054+0000 mgr.y (mgr.4120) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129847 2023-01-19 03:10:05 2023-01-19 04:38:57 2023-01-19 05:02:00 0:23:03 0:10:48 0:12:15 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} 2
pass 7129848 2023-01-19 03:10:06 2023-01-19 04:40:09 2023-01-19 05:05:07 0:24:58 0:13:25 0:11:33 smithi main ubuntu 20.04 rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} 2
fail 7129849 2023-01-19 03:10:07 2023-01-19 04:41:19 2023-01-19 05:22:17 0:40:58 0:33:36 0:07:22 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:56:54.683779+0000 mgr.x (mgr.4108) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129850 2023-01-19 03:10:08 2023-01-19 04:41:20 2023-01-19 05:39:19 0:57:59 0:50:07 0:07:52 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T04:58:11.317254+0000 mgr.y (mgr.4106) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129851 2023-01-19 03:10:09 2023-01-19 04:41:30 2023-01-19 05:01:57 0:20:27 0:08:24 0:12:03 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_ragweed ubuntu_latest} 2
Failure Reason:

Command failed on smithi167 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap'

fail 7129852 2023-01-19 03:10:10 2023-01-19 04:41:50 2023-01-19 05:23:14 0:41:24 0:31:50 0:09:34 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T04:59:56.171154+0000 mgr.y (mgr.4103) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129853 2023-01-19 03:10:11 2023-01-19 04:44:11 2023-01-19 05:11:05 0:26:54 0:15:14 0:11:40 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_s3tests ubuntu_latest} 2
fail 7129854 2023-01-19 03:10:12 2023-01-19 04:45:22 2023-01-19 05:43:18 0:57:56 0:50:32 0:07:24 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T05:02:31.855879+0000 mgr.y (mgr.4113) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7129855 2023-01-19 03:10:13 2023-01-19 04:45:52 2023-01-19 05:27:57 0:42:05 0:32:34 0:09:31 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} 2
Failure Reason:

"2023-01-19T05:04:38.272187+0000 mgr.x (mgr.4105) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

pass 7129856 2023-01-19 03:10:14 2023-01-19 04:48:53 2023-01-19 05:09:45 0:20:52 0:09:24 0:11:28 smithi main ubuntu 20.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_user_quota ubuntu_latest} 2
fail 7129857 2023-01-19 03:10:15 2023-01-19 04:50:13 2023-01-19 05:47:42 0:57:29 0:49:41 0:07:48 smithi main centos 8.stream rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} 2
Failure Reason:

"2023-01-19T05:06:27.793402+0000 mgr.y (mgr.4112) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log