Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6931222 2022-07-15 03:20:15 2022-07-17 01:03:54 2022-07-17 01:29:35 0:25:41 0:13:39 0:12:02 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_deep_flatten} 1
pass 6931223 2022-07-15 03:20:17 2022-07-17 01:06:15 2022-07-17 01:42:38 0:36:23 0:23:24 0:12:59 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6931224 2022-07-15 03:20:18 2022-07-17 01:06:35 2022-07-17 01:30:16 0:23:41 0:13:34 0:10:07 smithi main krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6931225 2022-07-15 03:20:19 2022-07-17 01:06:46 2022-07-17 01:48:45 0:41:59 0:31:01 0:10:58 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_fio} 3
pass 6931226 2022-07-15 03:20:21 2022-07-17 01:07:17 2022-07-17 01:42:34 0:35:17 0:22:36 0:12:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
fail 6931227 2022-07-15 03:20:22 2022-07-17 01:09:07 2022-07-17 06:06:22 4:57:15 4:43:12 0:14:03 smithi main krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi157 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude6ukqtmym -r -- '-g auto -g blockdev -x clone'"

pass 6931228 2022-07-15 03:20:23 2022-07-17 01:09:58 2022-07-17 02:07:07 0:57:09 0:44:35 0:12:34 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6931229 2022-07-15 03:20:25 2022-07-17 01:11:19 2022-07-17 01:18:42 0:07:23 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6931230 2022-07-15 03:20:26 2022-07-17 01:11:39 2022-07-17 01:36:00 0:24:21 0:11:57 0:12:24 smithi main krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
fail 6931231 2022-07-15 03:20:27 2022-07-17 01:12:30 2022-07-17 01:47:59 0:35:29 0:22:20 0:13:09 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-07-17T01:45:45.659415+0000 mon.a (mon.0) 257 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931232 2022-07-15 03:20:28 2022-07-17 01:14:10 2022-07-17 01:50:07 0:35:57 0:21:47 0:14:10 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6931233 2022-07-15 03:20:30 2022-07-17 01:15:21 2022-07-17 01:41:14 0:25:53 0:15:19 0:10:34 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6931234 2022-07-15 03:20:31 2022-07-17 01:15:32 2022-07-17 01:49:14 0:33:42 0:22:39 0:11:03 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-07-17T01:46:55.796435+0000 mon.a (mon.0) 291 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931235 2022-07-15 03:20:32 2022-07-17 01:15:32 2022-07-17 02:07:22 0:51:50 0:40:17 0:11:33 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6931236 2022-07-15 03:20:33 2022-07-17 01:17:43 2022-07-17 01:48:32 0:30:49 0:16:52 0:13:57 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6931237 2022-07-15 03:20:35 2022-07-17 01:18:23 2022-07-17 01:41:41 0:23:18 0:13:25 0:09:53 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_discard} 1
pass 6931238 2022-07-15 03:20:36 2022-07-17 01:18:24 2022-07-17 01:52:28 0:34:04 0:20:35 0:13:29 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6931239 2022-07-15 03:20:37 2022-07-17 01:18:44 2022-07-17 01:47:32 0:28:48 0:15:42 0:13:06 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
pass 6931240 2022-07-15 03:20:38 2022-07-17 01:19:15 2022-07-17 02:45:06 1:25:51 1:12:13 0:13:38 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6931241 2022-07-15 03:20:40 2022-07-17 01:19:15 2022-07-17 02:16:58 0:57:43 0:42:08 0:15:35 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6931242 2022-07-15 03:20:41 2022-07-17 01:23:26 2022-07-17 01:58:16 0:34:50 0:20:54 0:13:56 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6931243 2022-07-15 03:20:42 2022-07-17 01:26:37 2022-07-17 01:53:51 0:27:14 0:15:14 0:12:00 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6931244 2022-07-15 03:20:44 2022-07-17 01:28:28 2022-07-17 01:54:31 0:26:03 0:14:26 0:11:37 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6931245 2022-07-15 03:20:45 2022-07-17 01:29:09 2022-07-17 01:52:30 0:23:21 0:12:23 0:10:58 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
fail 6931246 2022-07-15 03:20:46 2022-07-17 01:29:09 2022-07-17 02:03:41 0:34:32 0:22:44 0:11:48 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2022-07-17T02:01:18.114046+0000 mon.a (mon.0) 233 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931247 2022-07-15 03:20:47 2022-07-17 01:30:20 2022-07-17 01:57:48 0:27:28 0:15:06 0:12:22 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6931248 2022-07-15 03:20:49 2022-07-17 01:32:21 2022-07-17 01:59:37 0:27:16 0:14:03 0:13:13 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6931249 2022-07-15 03:20:50 2022-07-17 01:36:02 2022-07-17 02:08:17 0:32:15 0:18:43 0:13:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_concurrent} 3
pass 6931250 2022-07-15 03:20:51 2022-07-17 01:38:53 2022-07-17 02:05:47 0:26:54 0:14:22 0:12:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6931251 2022-07-15 03:20:53 2022-07-17 01:39:33 2022-07-17 02:14:55 0:35:22 0:23:20 0:12:02 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
fail 6931252 2022-07-15 03:20:54 2022-07-17 01:41:24 2022-07-17 02:14:47 0:33:23 0:22:38 0:10:45 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
Failure Reason:

"2022-07-17T02:13:29.031213+0000 mon.a (mon.0) 218 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

pass 6931253 2022-07-15 03:20:55 2022-07-17 01:41:25 2022-07-17 02:04:38 0:23:13 0:12:03 0:11:10 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_modprobe} 1
pass 6931254 2022-07-15 03:20:56 2022-07-17 01:41:25 2022-07-17 02:12:06 0:30:41 0:19:22 0:11:19 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6931255 2022-07-15 03:20:58 2022-07-17 01:42:36 2022-07-17 02:56:52 1:14:16 1:02:43 0:11:33 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6931256 2022-07-15 03:20:59 2022-07-17 01:42:46 2022-07-17 02:17:47 0:35:01 0:22:40 0:12:21 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-07-17T02:15:34.700837+0000 mon.a (mon.0) 225 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931257 2022-07-15 03:21:00 2022-07-17 01:44:17 2022-07-17 02:19:04 0:34:47 0:20:34 0:14:13 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6931258 2022-07-15 03:21:02 2022-07-17 01:47:38 2022-07-17 02:11:14 0:23:36 0:13:43 0:09:53 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6931259 2022-07-15 03:21:03 2022-07-17 01:47:38 2022-07-17 02:21:56 0:34:18 0:23:17 0:11:01 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-07-17T02:09:46.490694+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931260 2022-07-15 03:21:04 2022-07-17 01:47:39 2022-07-17 02:13:45 0:26:06 0:15:44 0:10:22 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6931261 2022-07-15 03:21:05 2022-07-17 01:48:09 2022-07-17 02:28:40 0:40:31 0:29:16 0:11:15 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
pass 6931262 2022-07-15 03:21:07 2022-07-17 01:48:40 2022-07-17 02:10:13 0:21:33 0:12:20 0:09:13 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_msgr_segments} 1
pass 6931263 2022-07-15 03:21:08 2022-07-17 01:48:50 2022-07-17 02:28:48 0:39:58 0:29:57 0:10:01 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
fail 6931264 2022-07-15 03:21:09 2022-07-17 01:49:20 2022-07-17 02:25:04 0:35:44 0:22:47 0:12:57 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-07-17T02:23:58.006611+0000 mon.a (mon.0) 275 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6931265 2022-07-15 03:21:11 2022-07-17 01:50:11 2022-07-17 02:29:56 0:39:45 0:27:36 0:12:09 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6931266 2022-07-15 03:21:12 2022-07-17 01:52:32 2022-07-17 02:18:00 0:25:28 0:15:32 0:09:56 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
fail 6931267 2022-07-15 03:21:14 2022-07-17 01:52:33 2022-07-17 02:27:24 0:34:51 0:22:31 0:12:20 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-07-17T02:24:41.593944+0000 mon.a (mon.0) 216 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931268 2022-07-15 03:21:15 2022-07-17 01:53:54 2022-07-17 02:19:56 0:26:02 0:15:22 0:10:40 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
pass 6931269 2022-07-15 03:21:16 2022-07-17 01:54:34 2022-07-17 02:23:01 0:28:27 0:14:49 0:13:38 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6931270 2022-07-15 03:21:18 2022-07-17 01:57:05 2022-07-17 02:49:23 0:52:18 0:41:04 0:11:14 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6931271 2022-07-15 03:21:19 2022-07-17 01:57:56 2022-07-17 02:26:09 0:28:13 0:17:50 0:10:23 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6931272 2022-07-15 03:21:20 2022-07-17 01:58:27 2022-07-17 02:21:09 0:22:42 0:11:57 0:10:45 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6931273 2022-07-15 03:21:21 2022-07-17 01:59:48 2022-07-17 02:33:54 0:34:06 0:21:57 0:12:09 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6931274 2022-07-15 03:21:22 2022-07-17 02:00:08 2022-07-17 02:27:44 0:27:36 0:14:54 0:12:42 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_namespaces} 3
pass 6931275 2022-07-15 03:21:24 2022-07-17 02:00:09 2022-07-17 03:14:22 1:14:13 1:02:28 0:11:45 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6931276 2022-07-15 03:21:25 2022-07-17 02:00:39 2022-07-17 02:36:07 0:35:28 0:22:10 0:13:18 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-07-17T02:33:24.635226+0000 mon.a (mon.0) 223 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931277 2022-07-15 03:21:26 2022-07-17 02:02:30 2022-07-17 02:35:04 0:32:34 0:20:41 0:11:53 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6931278 2022-07-15 03:21:28 2022-07-17 02:03:51 2022-07-17 02:40:06 0:36:15 0:22:29 0:13:46 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-07-17T02:28:28.625642+0000 mon.a (mon.0) 133 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6931279 2022-07-15 03:21:29 2022-07-17 02:05:51 2022-07-17 02:32:57 0:27:06 0:14:25 0:12:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6931280 2022-07-15 03:21:30 2022-07-17 02:07:12 2022-07-17 02:30:43 0:23:31 0:12:32 0:10:59 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_read_only} 1
pass 6931281 2022-07-15 03:21:31 2022-07-17 02:07:12 2022-07-17 02:38:44 0:31:32 0:21:34 0:09:58 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6931282 2022-07-15 03:21:33 2022-07-17 02:07:23 2022-07-17 02:59:43 0:52:20 0:41:35 0:10:45 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6931283 2022-07-15 03:21:34 2022-07-17 02:08:05 2022-07-17 02:35:02 0:26:57 0:14:41 0:12:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6931284 2022-07-15 03:21:35 2022-07-17 02:08:05 2022-07-17 02:33:52 0:25:47 0:13:55 0:11:52 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6931285 2022-07-15 03:21:36 2022-07-17 02:08:26 2022-07-17 06:04:01 3:55:35 3:42:19 0:13:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

Command failed (workunit test rbd/concurrent.sh) on smithi119 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=759f326524ac0cb9ad0709ead97b1d154ede5e27 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/concurrent.sh'

pass 6931286 2022-07-15 03:21:38 2022-07-17 02:10:16 2022-07-17 02:34:46 0:24:30 0:13:41 0:10:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6931287 2022-07-15 03:21:39 2022-07-17 02:11:17 2022-07-17 02:45:39 0:34:22 0:22:21 0:12:01 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-07-17T02:42:46.561537+0000 mon.a (mon.0) 355 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931288 2022-07-15 03:21:41 2022-07-17 02:12:08 2022-07-17 02:39:33 0:27:25 0:14:38 0:12:47 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6931289 2022-07-15 03:21:42 2022-07-17 02:13:48 2022-07-17 02:37:58 0:24:10 0:12:45 0:11:25 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_whole_object_zeroout} 1
pass 6931290 2022-07-15 03:21:44 2022-07-17 02:14:39 2022-07-17 02:48:02 0:33:23 0:22:18 0:11:05 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6931291 2022-07-15 03:21:45 2022-07-17 02:14:49 2022-07-17 02:38:38 0:23:49 0:13:36 0:10:13 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6931292 2022-07-15 03:21:46 2022-07-17 02:15:00 2022-07-17 02:46:30 0:31:30 0:19:40 0:11:50 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6931293 2022-07-15 03:21:47 2022-07-17 02:17:01 2022-07-17 02:41:03 0:24:02 0:13:38 0:10:24 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6931294 2022-07-15 03:21:49 2022-07-17 02:17:51 2022-07-17 02:47:33 0:29:42 0:18:52 0:10:50 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
fail 6931295 2022-07-15 03:21:50 2022-07-17 02:18:02 2022-07-17 02:52:04 0:34:02 0:22:27 0:11:35 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2022-07-17T02:49:43.432781+0000 mon.a (mon.0) 323 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931296 2022-07-15 03:21:51 2022-07-17 02:18:32 2022-07-17 03:02:27 0:43:55 0:32:25 0:11:30 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_simple_big} 3
pass 6931297 2022-07-15 03:21:53 2022-07-17 02:18:33 2022-07-17 02:42:29 0:23:56 0:12:26 0:11:30 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
fail 6931298 2022-07-15 03:21:54 2022-07-17 02:18:33 2022-07-17 02:46:41 0:28:08 0:17:03 0:11:05 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Command failed on smithi183 with status 120: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0'

fail 6931299 2022-07-15 03:21:55 2022-07-17 02:19:14 2022-07-17 02:53:36 0:34:22 0:22:04 0:12:18 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_fio} 3
Failure Reason:

"2022-07-17T02:50:56.284769+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931300 2022-07-15 03:21:56 2022-07-17 02:20:04 2022-07-17 02:56:15 0:36:11 0:22:16 0:13:55 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6931301 2022-07-15 03:21:58 2022-07-17 02:22:05 2022-07-17 02:47:07 0:25:02 0:12:11 0:12:51 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6931302 2022-07-15 03:21:59 2022-07-17 02:23:05 2022-07-17 03:01:59 0:38:54 0:23:59 0:14:55 smithi main krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
pass 6931303 2022-07-15 03:22:00 2022-07-17 02:25:06 2022-07-17 02:51:53 0:26:47 0:14:54 0:11:53 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6931304 2022-07-15 03:22:02 2022-07-17 02:26:17 2022-07-17 03:00:45 0:34:28 0:22:34 0:11:54 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-07-17T02:58:15.551889+0000 mon.a (mon.0) 281 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931305 2022-07-15 03:22:03 2022-07-17 02:27:27 2022-07-17 02:53:22 0:25:55 0:14:12 0:11:43 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_fallocate} 3
pass 6931306 2022-07-15 03:22:04 2022-07-17 02:27:48 2022-07-17 02:54:51 0:27:03 0:14:49 0:12:14 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6931307 2022-07-15 03:22:05 2022-07-17 02:28:48 2022-07-17 03:20:30 0:51:42 0:40:06 0:11:36 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6931308 2022-07-15 03:22:07 2022-07-17 02:28:49 2022-07-17 02:57:21 0:28:32 0:16:41 0:11:51 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6931309 2022-07-15 03:22:08 2022-07-17 02:29:59 2022-07-17 02:52:51 0:22:52 0:12:45 0:10:07 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_discard} 1
pass 6931310 2022-07-15 03:22:10 2022-07-17 02:30:00 2022-07-17 03:04:47 0:34:47 0:20:28 0:14:19 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6931311 2022-07-15 03:22:11 2022-07-17 02:33:01 2022-07-17 03:57:03 1:24:02 1:11:38 0:12:24 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6931312 2022-07-15 03:22:12 2022-07-17 02:33:21 2022-07-17 03:03:52 0:30:31 0:16:22 0:14:09 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
fail 6931313 2022-07-15 03:22:14 2022-07-17 02:34:02 2022-07-17 03:09:32 0:35:30 0:22:24 0:13:06 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-07-17T02:56:33.067864+0000 mon.a (mon.0) 168 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931314 2022-07-15 03:22:15 2022-07-17 02:34:03 2022-07-17 03:30:23 0:56:20 0:43:01 0:13:19 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6931315 2022-07-15 03:22:16 2022-07-17 02:34:54 2022-07-17 03:10:01 0:35:07 0:21:23 0:13:44 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6931316 2022-07-15 03:22:17 2022-07-17 02:35:04 2022-07-17 03:02:32 0:27:28 0:16:09 0:11:19 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6931317 2022-07-15 03:22:19 2022-07-17 02:35:05 2022-07-17 03:01:45 0:26:40 0:15:14 0:11:26 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6931318 2022-07-15 03:22:20 2022-07-17 02:36:15 2022-07-17 02:59:40 0:23:25 0:12:29 0:10:56 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_huge_image} 1
pass 6931319 2022-07-15 03:22:21 2022-07-17 02:36:16 2022-07-17 03:14:06 0:37:50 0:24:27 0:13:23 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6931320 2022-07-15 03:22:23 2022-07-17 02:38:46 2022-07-17 03:04:24 0:25:38 0:15:32 0:10:06 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6931321 2022-07-15 03:22:24 2022-07-17 02:38:47 2022-07-17 03:07:25 0:28:38 0:15:13 0:13:25 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6931322 2022-07-15 03:22:25 2022-07-17 02:39:37 2022-07-17 03:09:27 0:29:50 0:19:06 0:10:44 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
pass 6931323 2022-07-15 03:22:27 2022-07-17 02:39:48 2022-07-17 03:06:10 0:26:22 0:14:30 0:11:52 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6931324 2022-07-15 03:22:28 2022-07-17 02:40:08 2022-07-17 03:14:36 0:34:28 0:22:51 0:11:37 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6931325 2022-07-15 03:22:29 2022-07-17 02:41:09 2022-07-17 03:06:28 0:25:19 0:13:52 0:11:27 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_image_read} 3
pass 6931326 2022-07-15 03:22:30 2022-07-17 02:42:30 2022-07-17 03:06:52 0:24:22 0:11:50 0:12:32 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6931327 2022-07-15 03:22:32 2022-07-17 02:45:10 2022-07-17 03:19:03 0:33:53 0:22:30 0:11:23 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6931328 2022-07-15 03:22:33 2022-07-17 02:45:41 2022-07-17 03:14:06 0:28:25 0:16:10 0:12:15 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
pass 6931329 2022-07-15 03:22:35 2022-07-17 02:46:31 2022-07-17 03:17:57 0:31:26 0:20:58 0:10:28 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6931330 2022-07-15 03:22:36 2022-07-17 02:46:42 2022-07-17 03:11:07 0:24:25 0:13:26 0:10:59 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6931331 2022-07-15 03:22:38 2022-07-17 02:47:13 2022-07-17 03:21:12 0:33:59 0:22:30 0:11:29 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-07-17T03:19:13.960697+0000 mon.a (mon.0) 318 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6931332 2022-07-15 03:22:39 2022-07-17 02:47:43 2022-07-17 03:13:25 0:25:42 0:14:40 0:11:02 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
fail 6931333 2022-07-15 03:22:41 2022-07-17 02:48:04 2022-07-17 03:22:51 0:34:47 0:22:26 0:12:21 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

"2022-07-17T03:09:37.196037+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931334 2022-07-15 03:22:42 2022-07-17 02:49:24 2022-07-17 03:12:43 0:23:19 0:12:40 0:10:39 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_msgr_segments} 1
pass 6931335 2022-07-15 03:22:43 2022-07-17 02:49:25 2022-07-17 03:31:35 0:42:10 0:27:12 0:14:58 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6931336 2022-07-15 03:22:45 2022-07-17 02:51:56 2022-07-17 03:17:23 0:25:27 0:14:29 0:10:58 smithi main krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
pass 6931337 2022-07-15 03:22:46 2022-07-17 02:52:06 2022-07-17 03:35:37 0:43:31 0:31:43 0:11:48 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_fio} 3
pass 6931338 2022-07-15 03:22:47 2022-07-17 02:52:07 2022-07-17 03:31:07 0:39:00 0:26:24 0:12:36 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_data_pool} 3
fail 6931339 2022-07-15 03:22:49 2022-07-17 02:53:28 2022-07-17 08:01:19 5:07:51 4:57:02 0:10:49 smithi main krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi125 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude4_k1qgd6 -r -- '-g auto -g blockdev -x clone'"

pass 6931340 2022-07-15 03:22:50 2022-07-17 02:53:38 2022-07-17 04:17:07 1:23:29 1:11:54 0:11:35 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6931341 2022-07-15 03:22:51 2022-07-17 02:54:59 2022-07-17 03:22:42 0:27:43 0:15:04 0:12:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6931342 2022-07-15 03:22:53 2022-07-17 02:56:20 2022-07-17 03:29:02 0:32:42 0:21:40 0:11:02 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6931343 2022-07-15 03:22:54 2022-07-17 02:57:00 2022-07-17 03:31:27 0:34:27 0:22:48 0:11:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-07-17T03:29:13.829203+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931344 2022-07-15 03:22:56 2022-07-17 02:57:31 2022-07-17 03:26:22 0:28:51 0:14:45 0:14:06 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6931345 2022-07-15 03:22:57 2022-07-17 02:59:21 2022-07-17 03:51:22 0:52:01 0:41:00 0:11:01 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6931346 2022-07-15 03:22:58 2022-07-17 02:59:52 2022-07-17 03:34:13 0:34:21 0:22:40 0:11:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-07-17T03:32:00.250994+0000 mon.a (mon.0) 245 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931347 2022-07-15 03:23:00 2022-07-17 03:00:52 2022-07-17 03:24:20 0:23:28 0:12:10 0:11:18 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6931348 2022-07-15 03:23:01 2022-07-17 03:00:53 2022-07-17 03:33:39 0:32:46 0:21:58 0:10:48 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6931349 2022-07-15 03:23:03 2022-07-17 03:01:53 2022-07-17 03:29:07 0:27:14 0:14:54 0:12:20 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6931350 2022-07-15 03:23:04 2022-07-17 03:02:04 2022-07-17 04:16:09 1:14:05 1:02:54 0:11:11 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6931351 2022-07-15 03:23:06 2022-07-17 03:02:34 2022-07-17 03:58:09 0:55:35 0:45:38 0:09:57 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6931352 2022-07-15 03:23:07 2022-07-17 03:02:35 2022-07-17 03:35:41 0:33:06 0:20:43 0:12:23 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6931353 2022-07-15 03:23:09 2022-07-17 03:04:31 2022-07-17 03:32:30 0:27:59 0:16:13 0:11:46 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6931354 2022-07-15 03:23:10 2022-07-17 03:04:51 2022-07-17 03:32:29 0:27:38 0:14:54 0:12:44 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6931355 2022-07-15 03:23:12 2022-07-17 03:06:12 2022-07-17 03:27:49 0:21:37 0:12:22 0:09:15 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6931356 2022-07-15 03:23:13 2022-07-17 03:06:12 2022-07-17 03:48:35 0:42:23 0:31:43 0:10:40 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6931357 2022-07-15 03:23:14 2022-07-17 03:06:33 2022-07-17 03:33:18 0:26:45 0:15:00 0:11:45 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6931358 2022-07-15 03:23:16 2022-07-17 03:07:34 2022-07-17 03:34:53 0:27:19 0:14:18 0:13:01 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6931359 2022-07-15 03:23:18 2022-07-17 03:09:34 2022-07-17 03:39:14 0:29:40 0:18:51 0:10:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
pass 6931360 2022-07-15 03:23:19 2022-07-17 03:09:35 2022-07-17 03:34:08 0:24:33 0:13:55 0:10:38 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6931361 2022-07-15 03:23:21 2022-07-17 03:09:45 2022-07-17 03:49:04 0:39:19 0:25:50 0:13:29 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-07-17T03:41:06.686599+0000 osd.7 (osd.7) 3 : cluster [WRN] 1 slow requests (by type [ 'waiting for sub ops' : 1 ] most affected pool [ 'rbd' : 1 ])" in cluster log

fail 6931362 2022-07-15 03:23:22 2022-07-17 03:10:06 2022-07-17 03:45:18 0:35:12 0:22:09 0:13:03 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-07-17T03:32:13.331512+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931363 2022-07-15 03:23:24 2022-07-17 03:11:16 2022-07-17 03:34:56 0:23:40 0:12:40 0:11:00 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_whole_object_zeroout} 1
pass 6931364 2022-07-15 03:23:25 2022-07-17 03:11:17 2022-07-17 03:42:53 0:31:36 0:17:30 0:14:06 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6931365 2022-07-15 03:23:27 2022-07-17 03:13:28 2022-07-17 04:09:33 0:56:05 0:42:34 0:13:31 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6931366 2022-07-15 03:23:28 2022-07-17 03:14:08 2022-07-17 03:39:35 0:25:27 0:13:49 0:11:38 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6931367 2022-07-15 03:23:30 2022-07-17 03:14:09 2022-07-17 03:46:09 0:32:00 0:20:12 0:11:48 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6931368 2022-07-15 03:23:31 2022-07-17 03:14:29 2022-07-17 03:40:05 0:25:36 0:14:15 0:11:21 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6931369 2022-07-15 03:23:33 2022-07-17 03:14:40 2022-07-17 03:46:54 0:32:14 0:18:35 0:13:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6931370 2022-07-15 03:23:34 2022-07-17 03:17:31 2022-07-17 03:43:31 0:26:00 0:16:01 0:09:59 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6931371 2022-07-15 03:23:35 2022-07-17 03:18:01 2022-07-17 03:52:43 0:34:42 0:22:22 0:12:20 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-07-17T03:39:56.149118+0000 mon.a (mon.0) 170 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931372 2022-07-15 03:23:37 2022-07-17 03:19:12 2022-07-17 03:42:17 0:23:05 0:12:45 0:10:20 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_deep_flatten} 1
pass 6931373 2022-07-15 03:23:38 2022-07-17 03:19:12 2022-07-17 04:00:17 0:41:05 0:29:03 0:12:02 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6931374 2022-07-15 03:23:40 2022-07-17 03:20:33 2022-07-17 04:02:39 0:42:06 0:31:47 0:10:19 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6931375 2022-07-15 03:23:41 2022-07-17 03:21:13 2022-07-17 03:56:53 0:35:40 0:22:36 0:13:04 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6931376 2022-07-15 03:23:42 2022-07-17 03:22:44 2022-07-17 03:44:30 0:21:46 0:12:02 0:09:44 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6931377 2022-07-15 03:23:43 2022-07-17 03:22:55 2022-07-17 03:59:56 0:37:01 0:24:25 0:12:36 smithi main krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
fail 6931378 2022-07-15 03:23:45 2022-07-17 03:24:25 2022-07-17 04:03:27 0:39:02 0:23:10 0:15:52 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-07-17T04:00:50.989720+0000 mon.a (mon.0) 293 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931379 2022-07-15 03:23:46 2022-07-17 03:26:26 2022-07-17 04:05:10 0:38:44 0:24:18 0:14:26 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6931380 2022-07-15 03:23:48 2022-07-17 03:29:07 2022-07-17 03:58:16 0:29:09 0:15:47 0:13:22 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
fail 6931381 2022-07-15 03:23:49 2022-07-17 03:29:17 2022-07-17 04:03:54 0:34:37 0:23:35 0:11:02 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-07-17T03:51:56.072960+0000 mon.a (mon.0) 183 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931382 2022-07-15 03:23:50 2022-07-17 03:30:28 2022-07-17 04:24:47 0:54:19 0:42:06 0:12:13 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6931383 2022-07-15 03:23:51 2022-07-17 03:31:09 2022-07-17 04:00:43 0:29:34 0:18:00 0:11:34 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6931384 2022-07-15 03:23:53 2022-07-17 03:31:29 2022-07-17 03:55:02 0:23:33 0:13:34 0:09:59 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6931385 2022-07-15 03:23:54 2022-07-17 03:31:30 2022-07-17 04:03:15 0:31:45 0:21:23 0:10:22 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6931386 2022-07-15 03:23:56 2022-07-17 03:31:40 2022-07-17 04:08:47 0:37:07 0:23:39 0:13:28 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-07-17T04:06:17.977748+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931387 2022-07-15 03:23:57 2022-07-17 03:32:31 2022-07-17 04:54:25 1:21:54 1:11:48 0:10:06 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6931388 2022-07-15 03:23:59 2022-07-17 03:32:31 2022-07-17 04:27:05 0:54:34 0:43:06 0:11:28 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6931389 2022-07-15 03:24:00 2022-07-17 08:22:46 2022-07-17 08:57:43 0:34:57 0:22:54 0:12:03 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-07-17T08:45:44.300116+0000 mon.a (mon.0) 125 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931390 2022-07-15 03:24:01 2022-07-17 08:23:57 2022-07-17 08:51:40 0:27:43 0:15:54 0:11:49 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6931391 2022-07-15 03:24:03 2022-07-17 08:24:08 2022-07-17 08:50:54 0:26:46 0:14:41 0:12:05 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6931392 2022-07-15 03:24:04 2022-07-17 08:25:29 2022-07-17 08:47:20 0:21:51 0:11:59 0:09:52 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_huge_image} 1
pass 6931393 2022-07-15 03:24:05 2022-07-17 08:25:59 2022-07-17 09:05:36 0:39:37 0:26:26 0:13:11 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6931394 2022-07-15 03:24:06 2022-07-17 08:26:40 2022-07-17 09:54:39 1:27:59 1:17:43 0:10:16 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6931395 2022-07-15 03:24:08 2022-07-17 08:27:01 2022-07-17 09:01:19 0:34:18 0:23:30 0:10:48 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-07-17T08:48:55.911985+0000 mon.a (mon.0) 161 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6931396 2022-07-15 03:24:09 2022-07-17 08:27:51 2022-07-17 08:53:50 0:25:59 0:15:48 0:10:11 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6931397 2022-07-15 03:24:10 2022-07-17 08:28:22 2022-07-17 08:58:23 0:30:01 0:20:20 0:09:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
pass 6931398 2022-07-15 03:24:12 2022-07-17 08:28:52 2022-07-17 08:54:55 0:26:03 0:16:30 0:09:33 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6931399 2022-07-15 03:24:13 2022-07-17 08:29:33 2022-07-17 09:10:05 0:40:32 0:26:35 0:13:57 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6931400 2022-07-15 03:24:14 2022-07-17 08:30:13 2022-07-17 08:59:40 0:29:27 0:17:12 0:12:15 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6931401 2022-07-15 03:24:16 2022-07-17 08:30:14 2022-07-17 08:53:35 0:23:21 0:13:20 0:10:01 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_modprobe} 1
pass 6931402 2022-07-15 03:24:17 2022-07-17 08:30:14 2022-07-17 09:15:49 0:45:35 0:33:14 0:12:21 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6931403 2022-07-15 03:24:18 2022-07-17 08:30:15 2022-07-17 09:07:45 0:37:30 0:26:08 0:11:22 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-07-17T09:06:10.124313+0000 mon.a (mon.0) 225 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931404 2022-07-15 03:24:20 2022-07-17 08:30:15 2022-07-17 09:07:58 0:37:43 0:24:59 0:12:44 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6931405 2022-07-15 03:24:21 2022-07-17 08:30:16 2022-07-17 09:00:17 0:30:01 0:17:25 0:12:36 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6931406 2022-07-15 03:24:22 2022-07-17 08:30:26 2022-07-17 09:05:40 0:35:14 0:22:19 0:12:55 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6931407 2022-07-15 03:24:23 2022-07-17 08:31:06 2022-07-17 09:03:10 0:32:04 0:19:26 0:12:38 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6931408 2022-07-15 03:24:25 2022-07-17 08:31:27 2022-07-17 09:15:17 0:43:50 0:32:50 0:11:00 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6931409 2022-07-15 03:24:26 2022-07-17 08:31:57 2022-07-17 08:57:55 0:25:58 0:14:01 0:11:57 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6931410 2022-07-15 03:24:28 2022-07-17 08:31:58 2022-07-17 09:17:47 0:45:49 0:34:33 0:11:16 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6931411 2022-07-15 03:24:29 2022-07-17 08:32:18 2022-07-17 09:20:37 0:48:19 0:36:20 0:11:59 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_fio} 3
fail 6931412 2022-07-15 03:24:30 2022-07-17 08:32:59 2022-07-17 09:11:16 0:38:17 0:26:41 0:11:36 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-07-17T09:09:21.676331+0000 mon.a (mon.0) 230 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931413 2022-07-15 03:24:32 2022-07-17 08:33:29 2022-07-17 09:02:58 0:29:29 0:18:44 0:10:45 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6931414 2022-07-15 03:24:33 2022-07-17 08:33:30 2022-07-17 09:09:44 0:36:14 0:25:50 0:10:24 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6931415 2022-07-15 03:24:35 2022-07-17 08:34:30 2022-07-17 09:05:05 0:30:35 0:17:34 0:13:01 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_fallocate} 3
pass 6931416 2022-07-15 03:24:36 2022-07-17 08:34:51 2022-07-17 09:04:10 0:29:19 0:17:09 0:12:10 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6931417 2022-07-15 03:24:37 2022-07-17 08:35:11 2022-07-17 09:12:03 0:36:52 0:24:52 0:12:00 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-07-17T08:59:42.808525+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6931418 2022-07-15 03:24:39 2022-07-17 08:36:32 2022-07-17 09:14:21 0:37:49 0:24:05 0:13:44 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-07-17T09:11:44.117033+0000 mon.a (mon.0) 227 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931419 2022-07-15 03:24:40 2022-07-17 08:38:53 2022-07-17 09:02:56 0:24:03 0:13:34 0:10:29 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_parent_overlap} 1
pass 6931420 2022-07-15 03:24:41 2022-07-17 08:38:53 2022-07-17 09:11:16 0:32:23 0:21:50 0:10:33 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6931421 2022-07-15 03:24:43 2022-07-17 08:39:33 2022-07-17 09:40:39 1:01:06 0:49:15 0:11:51 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6931422 2022-07-15 03:24:44 2022-07-17 08:40:54 2022-07-17 09:12:01 0:31:07 0:15:05 0:16:02 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6931423 2022-07-15 03:24:45 2022-07-17 09:20:47 2022-07-17 10:36:52 1:16:05 1:05:32 0:10:33 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6931424 2022-07-15 03:24:47 2022-07-17 09:20:57 2022-07-17 09:56:34 0:35:37 0:24:01 0:11:36 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-07-17T09:53:29.093556+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931425 2022-07-15 03:24:48 2022-07-17 09:20:58 2022-07-17 09:54:05 0:33:07 0:22:07 0:11:00 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6931426 2022-07-15 03:24:49 2022-07-17 09:22:29 2022-07-17 09:58:11 0:35:42 0:24:21 0:11:21 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-07-17T09:56:20.244400+0000 mon.a (mon.0) 293 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6931427 2022-07-15 03:24:51 2022-07-17 09:22:40 2022-07-17 09:57:42 0:35:02 0:24:00 0:11:02 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-07-17T09:56:31.514374+0000 mon.a (mon.0) 225 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931428 2022-07-15 03:24:52 2022-07-17 09:24:11 2022-07-17 09:47:40 0:23:29 0:13:14 0:10:15 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_read_only} 1
pass 6931429 2022-07-15 03:24:54 2022-07-17 09:24:12 2022-07-17 10:03:22 0:39:10 0:27:31 0:11:39 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6931430 2022-07-15 03:24:55 2022-07-17 09:25:23 2022-07-17 09:54:40 0:29:17 0:16:48 0:12:29 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6931431 2022-07-15 03:24:57 2022-07-17 09:55:01 984 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6931432 2022-07-15 03:24:58 2022-07-17 09:27:35 2022-07-17 09:59:48 0:32:13 0:20:59 0:11:14 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
pass 6931433 2022-07-15 03:25:00 2022-07-17 09:27:46 2022-07-17 09:57:30 0:29:44 0:16:43 0:13:01 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6931434 2022-07-15 03:25:01 2022-07-17 09:28:47 2022-07-17 10:05:42 0:36:55 0:25:12 0:11:43 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-07-17T10:03:59.162172+0000 mon.a (mon.0) 257 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931435 2022-07-15 03:25:02 2022-07-17 09:29:48 2022-07-17 09:57:55 0:28:07 0:17:03 0:11:04 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_image_read} 3
pass 6931436 2022-07-15 03:25:04 2022-07-17 09:30:19 2022-07-17 09:53:45 0:23:26 0:13:45 0:09:41 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
fail 6931437 2022-07-15 03:25:05 2022-07-17 10:06:28 1499 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2022-07-17T10:04:17.847293+0000 mon.a (mon.0) 290 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931438 2022-07-15 03:25:06 2022-07-17 09:31:00 2022-07-17 09:57:33 0:26:33 0:16:17 0:10:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6931439 2022-07-15 03:25:08 2022-07-17 09:32:11 2022-07-17 10:05:55 0:33:44 0:22:49 0:10:55 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6931440 2022-07-15 03:25:09 2022-07-17 09:32:22 2022-07-17 10:12:03 0:39:41 0:26:09 0:13:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-07-17T10:09:10.022724+0000 mon.a (mon.0) 201 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931441 2022-07-15 03:25:10 2022-07-17 09:34:13 2022-07-17 10:05:52 0:31:39 0:22:05 0:09:34 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
fail 6931442 2022-07-15 03:25:11 2022-07-17 09:34:14 2022-07-17 10:12:36 0:38:22 0:26:03 0:12:19 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2022-07-17T10:09:25.115008+0000 mon.a (mon.0) 227 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6931443 2022-07-15 03:25:13 2022-07-17 09:34:45 2022-07-17 10:21:55 0:47:10 0:36:18 0:10:52 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3