Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6896117 2022-06-24 03:20:18 2022-06-24 22:14:06 2022-06-24 22:41:56 0:27:50 0:10:01 0:17:49 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_deep_flatten} 1
pass 6896119 2022-06-24 03:20:19 2022-06-24 22:14:36 2022-06-24 22:54:53 0:40:17 0:20:29 0:19:48 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6896121 2022-06-24 03:20:21 2022-06-24 22:15:17 2022-06-24 22:47:22 0:32:05 0:12:39 0:19:26 smithi main krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6896123 2022-06-24 03:20:22 2022-06-24 22:15:47 2022-06-24 23:04:35 0:48:48 0:28:55 0:19:53 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_fio} 3
pass 6896125 2022-06-24 03:20:23 2022-06-24 22:17:08 2022-06-24 22:56:52 0:39:44 0:20:34 0:19:10 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
fail 6896127 2022-06-24 03:20:25 2022-06-24 22:17:18 2022-06-25 03:16:12 4:58:54 4:38:56 0:19:58 smithi main krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi111 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludejtja2yob -r -- '-g auto -g blockdev -x clone'"

pass 6896129 2022-06-24 03:20:26 2022-06-24 22:17:58 2022-06-24 23:14:51 0:56:53 0:37:52 0:19:01 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6896131 2022-06-24 03:20:27 2022-06-24 22:18:59 2022-06-24 22:27:17 0:08:18 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6896133 2022-06-24 03:20:28 2022-06-24 22:19:20 2022-06-24 22:46:21 0:27:01 0:11:00 0:16:01 smithi main krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
fail 6896135 2022-06-24 03:20:30 2022-06-24 22:19:20 2022-06-24 22:57:16 0:37:56 0:20:28 0:17:28 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-06-24T22:44:54.638197+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896137 2022-06-24 03:20:31 2022-06-24 22:19:40 2022-06-24 22:55:56 0:36:16 0:20:05 0:16:11 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6896139 2022-06-24 03:20:32 2022-06-24 22:20:31 2022-06-24 22:52:04 0:31:33 0:14:03 0:17:30 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6896141 2022-06-24 03:20:33 2022-06-24 22:20:42 2022-06-24 22:59:18 0:38:36 0:21:22 0:17:14 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-06-24T22:56:42.365400+0000 mon.a (mon.0) 278 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896143 2022-06-24 03:20:35 2022-06-24 22:21:52 2022-06-24 23:17:19 0:55:27 0:39:11 0:16:16 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6896145 2022-06-24 03:20:36 2022-06-24 22:22:03 2022-06-24 22:53:34 0:31:31 0:15:45 0:15:46 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6896147 2022-06-24 03:20:37 2022-06-24 22:24:03 2022-06-24 22:49:23 0:25:20 0:11:58 0:13:22 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_discard} 1
pass 6896149 2022-06-24 03:20:38 2022-06-24 22:24:04 2022-06-24 22:56:22 0:32:18 0:17:30 0:14:48 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6896151 2022-06-24 03:20:40 2022-06-24 22:24:14 2022-06-24 22:59:05 0:34:51 0:20:08 0:14:43 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-06-24T22:57:05.724283+0000 mon.a (mon.0) 289 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896153 2022-06-24 03:20:41 2022-06-24 22:25:35 2022-06-24 23:49:22 1:23:47 1:09:18 0:14:29 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6896155 2022-06-24 03:20:42 2022-06-24 22:27:25 2022-06-24 23:20:49 0:53:24 0:40:36 0:12:48 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6896157 2022-06-24 03:20:44 2022-06-24 22:27:46 2022-06-24 23:01:12 0:33:26 0:19:44 0:13:42 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-06-24T22:47:52.386284+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896158 2022-06-24 03:20:45 2022-06-24 22:27:46 2022-06-24 22:57:16 0:29:30 0:12:05 0:17:25 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6896160 2022-06-24 03:20:46 2022-06-24 22:29:27 2022-06-24 22:58:38 0:29:11 0:11:22 0:17:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6896163 2022-06-24 03:20:48 2022-06-24 22:53:50 550 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6896164 2022-06-24 03:20:49 2022-06-24 22:31:08 2022-06-24 23:13:17 0:42:09 0:23:02 0:19:07 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6896166 2022-06-24 03:20:50 2022-06-24 22:32:29 2022-06-24 23:01:46 0:29:17 0:12:46 0:16:31 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6896168 2022-06-24 03:20:52 2022-06-24 22:33:29 2022-06-24 23:09:27 0:35:58 0:19:25 0:16:33 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-06-24T22:56:06.265490+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896170 2022-06-24 03:20:53 2022-06-24 22:34:00 2022-06-24 23:06:01 0:32:01 0:17:00 0:15:01 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_concurrent} 3
pass 6896172 2022-06-24 03:20:54 2022-06-24 22:34:10 2022-06-24 23:03:03 0:28:53 0:11:34 0:17:19 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6896174 2022-06-24 03:20:55 2022-06-24 22:34:11 2022-06-24 23:10:43 0:36:32 0:20:14 0:16:18 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6896176 2022-06-24 03:20:57 2022-06-24 22:34:22 2022-06-24 23:03:07 0:28:45 0:12:04 0:16:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_image_read} 3
pass 6896178 2022-06-24 03:20:58 2022-06-24 22:35:42 2022-06-24 22:59:04 0:23:22 0:09:30 0:13:52 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_modprobe} 1
pass 6896180 2022-06-24 03:21:00 2022-06-24 22:35:43 2022-06-24 23:09:07 0:33:24 0:17:06 0:16:18 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6896182 2022-06-24 03:21:01 2022-06-24 22:35:43 2022-06-24 23:52:40 1:16:57 1:00:39 0:16:18 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6896184 2022-06-24 03:21:02 2022-06-24 22:37:04 2022-06-24 23:04:04 0:27:00 0:13:12 0:13:48 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
pass 6896186 2022-06-24 03:21:03 2022-06-24 22:38:34 2022-06-24 23:11:30 0:32:56 0:18:18 0:14:38 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6896188 2022-06-24 03:21:05 2022-06-24 22:42:06 2022-06-24 23:02:22 0:20:16 0:10:12 0:10:04 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6896190 2022-06-24 03:21:06 2022-06-24 22:42:56 2022-06-24 23:10:49 0:27:53 0:16:14 0:11:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6896192 2022-06-24 03:21:07 2022-06-24 22:43:07 2022-06-24 23:06:56 0:23:49 0:12:04 0:11:45 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6896194 2022-06-24 03:21:09 2022-06-24 22:43:17 2022-06-24 23:21:29 0:38:12 0:26:17 0:11:55 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
pass 6896196 2022-06-24 03:21:10 2022-06-24 22:43:17 2022-06-24 23:03:10 0:19:53 0:09:15 0:10:38 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_msgr_segments} 1
pass 6896198 2022-06-24 03:21:11 2022-06-24 22:43:38 2022-06-24 23:20:18 0:36:40 0:24:17 0:12:23 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6896200 2022-06-24 03:21:13 2022-06-24 22:44:48 2022-06-24 23:25:31 0:40:43 0:28:20 0:12:23 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6896202 2022-06-24 03:21:14 2022-06-24 22:46:29 2022-06-24 23:23:00 0:36:31 0:23:58 0:12:33 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6896204 2022-06-24 03:21:15 2022-06-24 22:47:30 2022-06-24 23:15:39 0:28:09 0:11:37 0:16:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6896206 2022-06-24 03:21:17 2022-06-24 22:52:11 2022-06-24 23:22:12 0:30:01 0:18:59 0:11:02 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6896208 2022-06-24 03:21:18 2022-06-24 22:52:52 2022-06-24 23:17:13 0:24:21 0:11:49 0:12:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
pass 6896210 2022-06-24 03:21:19 2022-06-24 22:53:43 2022-06-24 23:16:39 0:22:56 0:11:47 0:11:09 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6896212 2022-06-24 03:21:21 2022-06-24 22:55:03 2022-06-24 23:27:26 0:32:23 0:19:08 0:13:15 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-06-24T23:24:28.400992+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896214 2022-06-24 03:21:22 2022-06-24 22:56:04 2022-06-24 23:24:35 0:28:31 0:15:01 0:13:30 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6896216 2022-06-24 03:21:23 2022-06-24 22:56:24 2022-06-24 23:15:16 0:18:52 0:09:28 0:09:24 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6896218 2022-06-24 03:21:25 2022-06-24 22:56:25 2022-06-24 23:26:33 0:30:08 0:18:50 0:11:18 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6896220 2022-06-24 03:21:26 2022-06-24 22:56:55 2022-06-24 23:20:45 0:23:50 0:11:27 0:12:23 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_namespaces} 3
pass 6896222 2022-06-24 03:21:27 2022-06-24 22:57:26 2022-06-25 00:09:16 1:11:50 0:58:29 0:13:21 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6896224 2022-06-24 03:21:29 2022-06-24 22:57:27 2022-06-24 23:30:29 0:33:02 0:20:30 0:12:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-06-24T23:18:44.488203+0000 mon.a (mon.0) 113 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896226 2022-06-24 03:21:30 2022-06-24 22:58:47 2022-06-24 23:26:30 0:27:43 0:16:30 0:11:13 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6896228 2022-06-24 03:21:31 2022-06-24 22:59:08 2022-06-24 23:24:33 0:25:25 0:13:50 0:11:35 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6896229 2022-06-24 03:21:33 2022-06-24 22:59:08 2022-06-24 23:22:42 0:23:34 0:12:38 0:10:56 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6896230 2022-06-24 03:21:34 2022-06-24 22:59:29 2022-06-24 23:22:00 0:22:31 0:10:53 0:11:38 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_read_only} 1
pass 6896231 2022-06-24 03:21:35 2022-06-24 22:59:29 2022-06-24 23:31:33 0:32:04 0:20:01 0:12:03 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6896232 2022-06-24 03:21:37 2022-06-24 22:59:59 2022-06-25 00:00:31 1:00:32 0:48:15 0:12:17 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6896233 2022-06-24 03:21:38 2022-06-24 23:01:20 2022-06-24 23:26:03 0:24:43 0:11:44 0:12:59 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6896234 2022-06-24 03:21:39 2022-06-24 23:01:50 2022-06-24 23:24:04 0:22:14 0:10:47 0:11:27 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6896235 2022-06-24 03:21:40 2022-06-24 23:02:31 2022-06-25 00:08:49 1:06:18 0:53:35 0:12:43 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-06-24T23:58:04.778446+0000 osd.3 (osd.3) 3 : cluster [WRN] 2 slow requests (by type [ 'started' : 2 ] most affected pool [ 'rbd' : 2 ])" in cluster log

pass 6896236 2022-06-24 03:21:42 2022-06-24 23:03:11 2022-06-24 23:26:39 0:23:28 0:12:07 0:11:21 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6896237 2022-06-24 03:21:43 2022-06-24 23:03:12 2022-06-24 23:38:33 0:35:21 0:21:53 0:13:28 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6896238 2022-06-24 03:21:44 2022-06-24 23:03:32 2022-06-24 23:27:43 0:24:11 0:11:48 0:12:23 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6896239 2022-06-24 03:21:46 2022-06-24 23:04:13 2022-06-24 23:26:21 0:22:08 0:10:26 0:11:42 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6896241 2022-06-24 03:21:47 2022-06-24 23:04:43 2022-06-24 23:35:48 0:31:05 0:19:04 0:12:01 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6896243 2022-06-24 03:21:48 2022-06-24 23:06:04 2022-06-24 23:28:47 0:22:43 0:10:16 0:12:27 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6896245 2022-06-24 03:21:49 2022-06-24 23:07:04 2022-06-24 23:36:48 0:29:44 0:16:36 0:13:08 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6896246 2022-06-24 03:21:51 2022-06-24 23:09:15 2022-06-24 23:30:43 0:21:28 0:10:34 0:10:54 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6896247 2022-06-24 03:21:52 2022-06-24 23:09:36 2022-06-24 23:39:08 0:29:32 0:15:39 0:13:53 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6896248 2022-06-24 03:21:54 2022-06-24 23:10:46 2022-06-24 23:34:37 0:23:51 0:12:27 0:11:24 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6896249 2022-06-24 03:21:55 2022-06-24 23:10:57 2022-06-24 23:43:05 0:32:08 0:19:09 0:12:59 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-06-24T23:29:55.191301+0000 mon.a (mon.0) 155 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896250 2022-06-24 03:21:56 2022-06-24 23:11:38 2022-06-24 23:31:00 0:19:22 0:09:23 0:09:59 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6896252 2022-06-24 03:21:58 2022-06-24 23:11:38 2022-06-24 23:46:07 0:34:29 0:21:04 0:13:25 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6896254 2022-06-24 03:21:59 2022-06-24 23:13:19 2022-06-24 23:54:50 0:41:31 0:28:29 0:13:02 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6896256 2022-06-24 03:22:00 2022-06-24 23:15:00 2022-06-24 23:45:04 0:30:04 0:19:07 0:10:57 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6896257 2022-06-24 03:22:01 2022-06-24 23:15:40 2022-06-24 23:37:09 0:21:29 0:09:46 0:11:43 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
fail 6896258 2022-06-24 03:22:03 2022-06-24 23:15:41 2022-06-24 23:48:00 0:32:19 0:19:42 0:12:37 smithi main krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
Failure Reason:

"2022-06-24T23:46:15.743197+0000 mon.a (mon.0) 267 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6896259 2022-06-24 03:22:04 2022-06-24 23:16:11 2022-06-24 23:48:22 0:32:11 0:20:32 0:11:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-06-24T23:46:59.578935+0000 mon.a (mon.0) 231 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896260 2022-06-24 03:22:05 2022-06-24 23:16:42 2022-06-24 23:48:50 0:32:08 0:20:05 0:12:03 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6896261 2022-06-24 03:22:06 2022-06-24 23:17:23 2022-06-24 23:40:57 0:23:34 0:12:10 0:11:24 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_fallocate} 3
pass 6896262 2022-06-24 03:22:08 2022-06-24 23:17:23 2022-06-24 23:41:06 0:23:43 0:12:42 0:11:01 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6896263 2022-06-24 03:22:09 2022-06-24 23:17:43 2022-06-25 00:07:57 0:50:14 0:38:04 0:12:10 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6896264 2022-06-24 03:22:11 2022-06-24 23:26:08 2022-06-24 23:52:07 0:25:59 0:13:43 0:12:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6896265 2022-06-24 03:22:12 2022-06-24 23:26:38 2022-06-24 23:45:53 0:19:15 0:09:47 0:09:28 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_discard} 1
pass 6896266 2022-06-24 03:22:13 2022-06-24 23:26:38 2022-06-24 23:56:18 0:29:40 0:17:43 0:11:57 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
dead 6896267 2022-06-24 03:22:15 2022-06-24 23:26:39 2022-06-24 23:42:11 0:15:32 0:03:53 0:11:39 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
Failure Reason:

{'smithi053.front.sepia.ceph.com': {'_ansible_no_log': False, 'cache_update_time': 1656113884, 'cache_updated': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'only_upgrade': False, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'libfcgi0ldbl', 'python-dev', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 5072 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 5072 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 5072 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}}

fail 6896268 2022-06-24 03:22:16 2022-06-24 23:26:49 2022-06-24 23:59:25 0:32:36 0:19:44 0:12:52 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-06-24T23:56:39.303978+0000 mon.a (mon.0) 226 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6896269 2022-06-24 03:22:17 2022-06-24 23:27:30 2022-06-24 23:59:14 0:31:44 0:19:21 0:12:23 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-06-24T23:56:25.830914+0000 mon.a (mon.0) 325 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896270 2022-06-24 03:22:18 2022-06-24 23:27:50 2022-06-25 00:18:34 0:50:44 0:38:46 0:11:58 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6896271 2022-06-24 03:22:20 2022-06-24 23:28:51 2022-06-25 00:01:49 0:32:58 0:19:33 0:13:25 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-06-24T23:48:11.416721+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896272 2022-06-24 03:22:21 2022-06-24 23:30:32 2022-06-24 23:54:30 0:23:58 0:12:08 0:11:50 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6896273 2022-06-24 03:22:22 2022-06-24 23:30:53 2022-06-24 23:52:16 0:21:23 0:11:18 0:10:05 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6896274 2022-06-24 03:22:24 2022-06-24 23:30:53 2022-06-24 23:50:22 0:19:29 0:08:44 0:10:45 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_huge_image} 1
pass 6896275 2022-06-24 03:22:25 2022-06-24 23:31:04 2022-06-25 00:03:14 0:32:10 0:20:59 0:11:11 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6896276 2022-06-24 03:22:27 2022-06-24 23:31:34 2022-06-24 23:57:33 0:25:59 0:11:47 0:14:12 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6896277 2022-06-24 03:22:28 2022-06-24 23:33:25 2022-06-25 00:02:40 0:29:15 0:19:14 0:10:01 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-06-24T23:50:37.656681+0000 mon.a (mon.0) 111 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896278 2022-06-24 03:22:30 2022-06-24 23:33:25 2022-06-24 23:59:09 0:25:44 0:15:18 0:10:26 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6896279 2022-06-24 03:22:31 2022-06-24 23:33:56 2022-06-25 00:06:32 0:32:36 0:19:15 0:13:21 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-06-25T00:03:47.033364+0000 mon.a (mon.0) 239 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896280 2022-06-24 03:22:33 2022-06-24 23:34:37 2022-06-25 00:06:30 0:31:53 0:19:24 0:12:29 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6896281 2022-06-24 03:22:34 2022-06-24 23:34:47 2022-06-24 23:57:44 0:22:57 0:10:55 0:12:02 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_image_read} 3
pass 6896282 2022-06-24 03:22:35 2022-06-24 23:35:58 2022-06-24 23:55:39 0:19:41 0:08:42 0:10:59 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
fail 6896283 2022-06-24 03:22:37 2022-06-24 23:35:58 2022-06-25 00:08:14 0:32:16 0:19:25 0:12:51 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2022-06-25T00:05:57.658935+0000 mon.a (mon.0) 266 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896284 2022-06-24 03:22:38 2022-06-24 23:36:49 2022-06-24 23:58:39 0:21:50 0:10:34 0:11:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
pass 6896285 2022-06-24 03:22:39 2022-06-24 23:37:09 2022-06-25 00:08:02 0:30:53 0:19:36 0:11:17 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6896286 2022-06-24 03:22:41 2022-06-24 23:38:40 2022-06-25 00:01:39 0:22:59 0:10:20 0:12:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6896287 2022-06-24 03:22:42 2022-06-24 23:38:40 2022-06-25 00:09:31 0:30:51 0:19:18 0:11:33 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-06-25T00:07:54.633626+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896288 2022-06-24 03:22:43 2022-06-24 23:39:11 2022-06-25 00:04:36 0:25:25 0:11:48 0:13:37 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6896289 2022-06-24 03:22:44 2022-06-24 23:48:33 2022-06-25 00:28:00 0:39:27 0:28:16 0:11:11 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6896290 2022-06-24 03:22:46 2022-06-24 23:48:33 2022-06-25 00:07:56 0:19:23 0:09:07 0:10:16 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_msgr_segments} 1
pass 6896291 2022-06-24 03:22:47 2022-06-24 23:48:34 2022-06-25 00:22:18 0:33:44 0:22:23 0:11:21 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6896292 2022-06-24 03:22:48 2022-06-24 23:48:54 2022-06-25 00:13:27 0:24:33 0:11:52 0:12:41 smithi main krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
fail 6896293 2022-06-24 03:22:50 2022-06-24 23:49:25 2022-06-25 00:21:48 0:32:23 0:19:44 0:12:39 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-06-25T00:19:04.205939+0000 mon.a (mon.0) 224 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896294 2022-06-24 03:22:51 2022-06-24 23:50:05 2022-06-25 00:24:27 0:34:22 0:23:55 0:10:27 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_data_pool} 3
fail 6896295 2022-06-24 03:22:52 2022-06-24 23:50:46 2022-06-25 04:50:08 4:59:22 4:47:31 0:11:51 smithi main krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy-rxbounce} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi171 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludexhgu20wo -r -- '-g auto -g blockdev -x clone'"

pass 6896296 2022-06-24 03:22:54 2022-06-24 23:52:17 2022-06-25 01:15:46 1:23:29 1:13:27 0:10:02 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6896297 2022-06-24 03:22:55 2022-06-24 23:52:17 2022-06-25 00:14:20 0:22:03 0:11:39 0:10:24 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6896298 2022-06-24 03:22:56 2022-06-24 23:52:48 2022-06-25 00:27:22 0:34:34 0:19:04 0:15:30 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6896299 2022-06-24 03:22:57 2022-06-24 23:54:28 2022-06-25 00:19:51 0:25:23 0:13:20 0:12:03 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
pass 6896300 2022-06-24 03:22:59 2022-06-24 23:54:29 2022-06-25 00:18:01 0:23:32 0:12:10 0:11:22 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6896301 2022-06-24 03:23:00 2022-06-24 23:54:39 2022-06-25 00:24:50 0:30:11 0:19:32 0:10:39 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-06-25T00:13:08.660824+0000 mon.a (mon.0) 162 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896302 2022-06-24 03:23:01 2022-06-24 23:55:00 2022-06-25 00:21:22 0:26:22 0:14:33 0:11:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6896303 2022-06-24 03:23:02 2022-06-24 23:55:40 2022-06-25 00:15:54 0:20:14 0:09:22 0:10:52 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_parent_overlap} 1
pass 6896304 2022-06-24 03:23:03 2022-06-24 23:56:21 2022-06-25 00:26:05 0:29:44 0:19:11 0:10:33 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6896305 2022-06-24 03:23:05 2022-06-24 23:56:21 2022-06-25 00:20:12 0:23:51 0:12:54 0:10:57 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6896306 2022-06-24 03:23:06 2022-06-24 23:56:42 2022-06-25 01:07:09 1:10:27 0:59:17 0:11:10 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6896307 2022-06-24 03:23:07 2022-06-24 23:57:42 2022-06-25 00:27:32 0:29:50 0:19:34 0:10:16 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-06-25T00:15:31.505259+0000 mon.a (mon.0) 202 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896308 2022-06-24 03:23:08 2022-06-24 23:57:52 2022-06-25 00:28:18 0:30:26 0:17:09 0:13:17 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6896309 2022-06-24 03:23:10 2022-06-24 23:58:03 2022-06-25 00:30:13 0:32:10 0:19:26 0:12:44 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-06-25T00:28:18.288376+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6896310 2022-06-24 03:23:11 2022-06-24 23:58:43 2022-06-25 00:30:23 0:31:40 0:19:34 0:12:06 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-06-25T00:18:08.249014+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896311 2022-06-24 03:23:12 2022-06-24 23:59:04 2022-06-25 00:20:10 0:21:06 0:09:58 0:11:08 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6896312 2022-06-24 03:23:13 2022-06-24 23:59:04 2022-06-25 00:40:43 0:41:39 0:29:18 0:12:21 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6896313 2022-06-24 03:23:15 2022-06-24 23:59:15 2022-06-25 00:24:43 0:25:28 0:13:01 0:12:27 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6896314 2022-06-24 03:23:16 2022-06-24 23:59:25 2022-06-25 00:24:53 0:25:28 0:12:41 0:12:47 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6896315 2022-06-24 03:23:17 2022-06-24 23:59:36 2022-06-25 00:30:03 0:30:27 0:17:38 0:12:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
pass 6896316 2022-06-24 03:23:18 2022-06-25 00:00:36 2022-06-25 00:24:31 0:23:55 0:12:09 0:11:46 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6896317 2022-06-24 03:23:20 2022-06-25 00:01:17 2022-06-25 00:38:40 0:37:23 0:22:46 0:14:37 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6896318 2022-06-24 03:23:21 2022-06-25 00:01:47 2022-06-25 00:33:16 0:31:29 0:19:52 0:11:37 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-06-25T00:31:14.562538+0000 mon.a (mon.0) 216 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896319 2022-06-24 03:23:22 2022-06-25 00:01:58 2022-06-25 00:23:45 0:21:47 0:10:00 0:11:47 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6896320 2022-06-24 03:23:24 2022-06-25 00:01:58 2022-06-25 00:28:17 0:26:19 0:14:47 0:11:32 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6896321 2022-06-24 03:23:25 2022-06-25 00:02:49 2022-06-25 00:52:09 0:49:20 0:37:31 0:11:49 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6896322 2022-06-24 03:23:26 2022-06-25 00:02:49 2022-06-25 00:24:49 0:22:00 0:10:43 0:11:17 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6896323 2022-06-24 03:23:27 2022-06-25 00:03:20 2022-06-25 00:32:13 0:28:53 0:16:25 0:12:28 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6896324 2022-06-24 03:23:28 2022-06-25 00:04:40 2022-06-25 00:25:42 0:21:02 0:10:30 0:10:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6896325 2022-06-24 03:23:30 2022-06-25 00:06:11 2022-06-25 00:34:19 0:28:08 0:15:27 0:12:41 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6896326 2022-06-24 03:23:31 2022-06-25 00:06:21 2022-06-25 00:28:11 0:21:50 0:12:10 0:09:40 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6896327 2022-06-24 03:23:32 2022-06-25 00:06:32 2022-06-25 00:46:42 0:40:10 0:27:35 0:12:35 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_simple_big} 3
pass 6896328 2022-06-24 03:23:33 2022-06-25 00:06:42 2022-06-25 00:26:05 0:19:23 0:09:20 0:10:03 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_deep_flatten} 1
pass 6896329 2022-06-24 03:23:35 2022-06-25 00:06:43 2022-06-25 00:47:27 0:40:44 0:26:10 0:14:34 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6896330 2022-06-24 03:23:36 2022-06-25 00:08:04 2022-06-25 00:49:35 0:41:31 0:29:17 0:12:14 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6896331 2022-06-24 03:23:37 2022-06-25 00:08:05 2022-06-25 00:39:47 0:31:42 0:19:53 0:11:49 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6896332 2022-06-24 03:23:38 2022-06-25 00:08:25 2022-06-25 00:30:01 0:21:36 0:09:25 0:12:11 smithi main krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6896333 2022-06-24 03:23:40 2022-06-25 00:08:26 2022-06-25 00:42:17 0:33:51 0:21:08 0:12:43 smithi main krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6896334 2022-06-24 03:23:41 2022-06-25 00:08:56 2022-06-25 00:33:14 0:24:18 0:11:46 0:12:32 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6896335 2022-06-24 03:23:42 2022-06-25 00:09:27 2022-06-25 00:42:57 0:33:30 0:20:32 0:12:58 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6896336 2022-06-24 03:23:44 2022-06-25 00:09:37 2022-06-25 00:34:19 0:24:42 0:11:29 0:13:13 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_fallocate} 3
pass 6896337 2022-06-24 03:23:45 2022-06-25 00:12:08 2022-06-25 00:38:10 0:26:02 0:11:58 0:14:04 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6896338 2022-06-24 03:23:46 2022-06-25 00:13:29 2022-06-25 01:02:00 0:48:31 0:37:14 0:11:17 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6896339 2022-06-24 03:23:47 2022-06-25 00:14:30 2022-06-25 00:42:34 0:28:04 0:14:16 0:13:48 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6896340 2022-06-24 03:23:49 2022-06-25 00:16:00 2022-06-25 00:34:56 0:18:56 0:10:13 0:08:43 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6896341 2022-06-24 03:23:50 2022-06-25 00:16:21 2022-06-25 00:43:32 0:27:11 0:16:03 0:11:08 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6896342 2022-06-24 03:23:51 2022-06-25 00:18:12 2022-06-25 00:42:24 0:24:12 0:13:01 0:11:11 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
pass 6896343 2022-06-24 03:23:52 2022-06-25 00:18:42 2022-06-25 01:39:23 1:20:41 1:08:42 0:11:59 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6896344 2022-06-24 03:23:53 2022-06-25 00:19:53 2022-06-25 01:09:47 0:49:54 0:38:26 0:11:28 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6896345 2022-06-24 03:23:55 2022-06-25 00:20:14 2022-06-25 00:53:01 0:32:47 0:19:27 0:13:20 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-06-25T00:49:56.422484+0000 mon.a (mon.0) 223 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896346 2022-06-24 03:23:56 2022-06-25 00:21:24 2022-06-25 00:45:36 0:24:12 0:12:02 0:12:10 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6896347 2022-06-24 03:23:57 2022-06-25 00:21:55 2022-06-25 00:43:42 0:21:47 0:11:24 0:10:23 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6896348 2022-06-24 03:23:58 2022-06-25 00:22:25 2022-06-25 00:41:37 0:19:12 0:09:11 0:10:01 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_huge_image} 1
pass 6896349 2022-06-24 03:23:59 2022-06-25 00:22:26 2022-06-25 00:58:01 0:35:35 0:20:55 0:14:40 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6896350 2022-06-24 03:24:01 2022-06-25 00:24:36 2022-06-25 01:40:18 1:15:42 1:03:14 0:12:28 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6896351 2022-06-24 03:24:02 2022-06-25 00:24:37 2022-06-25 00:50:20 0:25:43 0:11:58 0:13:45 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6896352 2022-06-24 03:24:03 2022-06-25 00:24:47 2022-06-25 00:50:23 0:25:36 0:12:14 0:13:22 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6896353 2022-06-24 03:24:04 2022-06-25 00:24:58 2022-06-25 00:54:36 0:29:38 0:17:32 0:12:06 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
fail 6896354 2022-06-24 03:24:05 2022-06-25 00:24:58 2022-06-25 00:59:04 0:34:06 0:20:29 0:13:37 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-06-25T00:56:37.060274+0000 mon.a (mon.0) 315 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896355 2022-06-24 03:24:07 2022-06-25 00:24:59 2022-06-25 00:58:40 0:33:41 0:21:10 0:12:31 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6896356 2022-06-24 03:24:08 2022-06-25 00:25:19 2022-06-25 00:51:21 0:26:02 0:12:11 0:13:51 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6896357 2022-06-24 03:24:09 2022-06-25 00:25:50 2022-06-25 00:47:43 0:21:53 0:09:08 0:12:45 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_modprobe} 1
pass 6896358 2022-06-24 03:24:10 2022-06-25 00:26:10 2022-06-25 00:59:39 0:33:29 0:21:31 0:11:58 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6896359 2022-06-24 03:24:11 2022-06-25 00:26:11 2022-06-25 00:52:23 0:26:12 0:10:57 0:15:15 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
pass 6896360 2022-06-24 03:24:13 2022-06-25 00:27:31 2022-06-25 00:57:19 0:29:48 0:18:31 0:11:17 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6896361 2022-06-24 03:24:14 2022-06-25 00:27:42 2022-06-25 00:49:29 0:21:47 0:10:57 0:10:50 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6896362 2022-06-24 03:24:15 2022-06-25 00:28:02 2022-06-25 00:58:35 0:30:33 0:16:51 0:13:42 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6896363 2022-06-24 03:24:16 2022-06-25 00:28:13 2022-06-25 00:52:42 0:24:29 0:12:30 0:11:59 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6896364 2022-06-24 03:24:18 2022-06-25 00:28:23 2022-06-25 01:07:53 0:39:30 0:26:51 0:12:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6896365 2022-06-24 03:24:19 2022-06-25 00:28:23 2022-06-25 00:51:19 0:22:56 0:09:37 0:13:19 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6896366 2022-06-24 03:24:20 2022-06-25 00:30:04 2022-06-25 01:07:41 0:37:37 0:26:52 0:10:45 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6896367 2022-06-24 03:24:21 2022-06-25 00:30:04 2022-06-25 01:01:30 0:31:26 0:19:55 0:11:31 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-06-25T00:49:25.887086+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896368 2022-06-24 03:24:22 2022-06-25 00:30:15 2022-06-25 01:05:42 0:35:27 0:24:08 0:11:19 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
pass 6896369 2022-06-24 03:24:24 2022-06-25 00:30:25 2022-06-25 00:53:53 0:23:28 0:11:51 0:11:37 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6896370 2022-06-24 03:24:25 2022-06-25 00:32:16 2022-06-25 01:03:03 0:30:47 0:18:32 0:12:15 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6896371 2022-06-24 03:24:26 2022-06-25 00:33:17 2022-06-25 00:54:48 0:21:31 0:11:43 0:09:48 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_fallocate} 3
pass 6896372 2022-06-24 03:24:27 2022-06-25 00:33:27 2022-06-25 00:56:39 0:23:12 0:11:22 0:11:50 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6896373 2022-06-24 03:24:29 2022-06-25 00:34:28 2022-06-25 01:24:56 0:50:28 0:37:35 0:12:53 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6896374 2022-06-24 03:24:30 2022-06-25 00:34:28 2022-06-25 01:08:22 0:33:54 0:19:08 0:14:46 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-06-25T01:07:10.913222+0000 mon.a (mon.0) 252 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

pass 6896375 2022-06-24 03:24:31 2022-06-25 00:38:19 2022-06-25 00:57:10 0:18:51 0:09:13 0:09:38 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_parent_overlap} 1
pass 6896376 2022-06-24 03:24:32 2022-06-25 00:38:19 2022-06-25 01:05:50 0:27:31 0:15:27 0:12:04 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6896377 2022-06-24 03:24:33 2022-06-25 00:38:50 2022-06-25 01:29:27 0:50:37 0:39:59 0:10:38 smithi main krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6896378 2022-06-24 03:24:35 2022-06-25 00:39:51 2022-06-25 01:02:16 0:22:25 0:11:12 0:11:13 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6896379 2022-06-24 03:24:36 2022-06-25 00:40:51 2022-06-25 01:51:01 1:10:10 0:59:35 0:10:35 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6896380 2022-06-24 03:24:37 2022-06-25 00:41:42 2022-06-25 01:35:58 0:54:16 0:43:26 0:10:50 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-06-25T01:04:32.903243+0000 osd.5 (osd.5) 3 : cluster [WRN] 2 slow requests (by type [ 'started' : 2 ] most affected pool [ 'rbd' : 2 ])" in cluster log

pass 6896381 2022-06-24 03:24:38 2022-06-25 00:42:22 2022-06-25 01:08:13 0:25:51 0:15:43 0:10:08 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6896382 2022-06-24 03:24:39 2022-06-25 00:42:33 2022-06-25 01:15:31 0:32:58 0:19:14 0:13:44 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-06-25T01:12:59.861722+0000 mon.a (mon.0) 310 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6896383 2022-06-24 03:24:41 2022-06-25 00:42:43 2022-06-25 01:14:28 0:31:45 0:19:43 0:12:02 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-06-25T01:12:08.435788+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896384 2022-06-24 03:24:42 2022-06-25 00:43:04 2022-06-25 01:04:57 0:21:53 0:09:32 0:12:21 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_read_only} 1
pass 6896385 2022-06-24 03:24:43 2022-06-25 00:43:34 2022-06-25 01:17:12 0:33:38 0:23:03 0:10:35 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6896386 2022-06-24 03:24:44 2022-06-25 00:43:45 2022-06-25 01:09:39 0:25:54 0:11:51 0:14:03 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6896387 2022-06-24 03:24:46 2022-06-25 00:45:46 2022-06-25 01:08:45 0:22:59 0:11:05 0:11:54 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6896388 2022-06-24 03:24:47 2022-06-25 00:46:46 2022-06-25 01:19:01 0:32:15 0:19:15 0:13:00 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-06-25T01:16:10.252631+0000 mon.a (mon.0) 278 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6896389 2022-06-24 03:24:48 2022-06-25 00:47:37 2022-06-25 01:09:26 0:21:49 0:11:07 0:10:42 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6896390 2022-06-24 03:24:49 2022-06-25 00:47:47 2022-06-25 01:21:07 0:33:20 0:19:55 0:13:25 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-06-25T01:08:12.733941+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6896391 2022-06-24 03:24:51 2022-06-25 00:49:38 2022-06-25 01:13:13 0:23:35 0:11:49 0:11:46 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_image_read} 3
pass 6896392 2022-06-24 03:24:52 2022-06-25 00:49:38 2022-06-25 01:09:46 0:20:08 0:09:50 0:10:18 smithi main krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6896393 2022-06-24 03:24:53 2022-06-25 00:50:29 2022-06-25 01:18:01 0:27:32 0:17:36 0:09:56 smithi main krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6896394 2022-06-24 03:24:54 2022-06-25 00:50:29 2022-06-25 01:12:56 0:22:27 0:10:48 0:11:39 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6896395 2022-06-24 03:24:55 2022-06-25 00:51:30 2022-06-25 01:18:57 0:27:27 0:16:29 0:10:58 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6896396 2022-06-24 03:24:56 2022-06-25 00:51:30 2022-06-25 01:13:41 0:22:11 0:10:38 0:11:33 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6896397 2022-06-24 03:24:58 2022-06-25 00:52:11 2022-06-25 01:19:20 0:27:09 0:15:50 0:11:19 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6896398 2022-06-24 03:24:59 2022-06-25 00:52:31 2022-06-25 01:17:07 0:24:36 0:12:27 0:12:09 smithi main krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6896399 2022-06-24 03:25:00 2022-06-25 00:52:52 2022-06-25 01:32:31 0:39:39 0:29:04 0:10:35 smithi main krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3