Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6835118 2022-05-15 21:50:53 2022-05-15 21:53:02 2022-05-15 22:17:28 0:24:26 0:11:53 0:12:33 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_deep_flatten} 1
pass 6835119 2022-05-15 21:50:54 2022-05-15 21:53:32 2022-05-15 22:28:59 0:35:27 0:22:55 0:12:32 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6835120 2022-05-15 21:50:55 2022-05-15 21:55:33 2022-05-15 22:18:55 0:23:22 0:13:12 0:10:10 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6835121 2022-05-15 21:50:56 2022-05-15 21:55:53 2022-05-15 22:38:11 0:42:18 0:30:50 0:11:28 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_fio} 3
pass 6835122 2022-05-15 21:50:57 2022-05-15 21:56:04 2022-05-15 22:28:05 0:32:01 0:21:36 0:10:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_data_pool} 3
fail 6835123 2022-05-15 21:50:58 2022-05-15 21:56:44 2022-05-16 02:46:03 4:49:19 4:36:29 0:12:50 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi053 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeab0ihoxk -r -- '-g auto -g blockdev -x clone'"

pass 6835124 2022-05-15 21:50:59 2022-05-15 21:58:15 2022-05-15 22:53:23 0:55:08 0:42:05 0:13:03 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
pass 6835125 2022-05-15 21:51:01 2022-05-15 21:59:56 2022-05-15 22:23:38 0:23:42 0:11:39 0:12:03 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
fail 6835126 2022-05-15 21:51:02 2022-05-15 22:02:16 2022-05-15 22:34:46 0:32:30 0:21:54 0:10:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-05-15T22:22:56.761758+0000 mon.a (mon.0) 195 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835127 2022-05-15 21:51:03 2022-05-15 22:03:17 2022-05-15 22:37:20 0:34:03 0:21:50 0:12:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6835128 2022-05-15 21:51:04 2022-05-15 22:05:47 2022-05-15 22:31:34 0:25:47 0:14:06 0:11:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
pass 6835129 2022-05-15 21:51:05 2022-05-15 22:05:58 2022-05-15 22:31:49 0:25:51 0:14:35 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6835130 2022-05-15 21:51:06 2022-05-15 22:06:18 2022-05-15 23:00:28 0:54:10 0:39:53 0:14:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6835131 2022-05-15 21:51:07 2022-05-15 22:08:59 2022-05-15 22:36:47 0:27:48 0:17:08 0:10:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6835132 2022-05-15 21:51:08 2022-05-15 22:09:29 2022-05-15 22:33:36 0:24:07 0:12:44 0:11:23 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_discard} 1
pass 6835133 2022-05-15 21:51:09 2022-05-15 22:09:30 2022-05-15 22:44:30 0:35:00 0:20:39 0:14:21 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6835134 2022-05-15 21:51:10 2022-05-15 22:11:00 2022-05-15 22:39:06 0:28:06 0:14:56 0:13:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
pass 6835135 2022-05-15 21:51:11 2022-05-15 22:11:21 2022-05-15 23:33:02 1:21:41 1:09:46 0:11:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6835136 2022-05-15 21:51:12 2022-05-15 22:11:31 2022-05-15 23:09:37 0:58:06 0:44:03 0:14:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6835137 2022-05-15 21:51:13 2022-05-15 22:13:02 2022-05-15 22:46:19 0:33:17 0:21:38 0:11:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-05-15T22:44:26.008389+0000 mon.a (mon.0) 237 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835138 2022-05-15 21:51:15 2022-05-15 22:13:22 2022-05-15 22:41:49 0:28:27 0:15:07 0:13:20 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6835139 2022-05-15 21:51:16 2022-05-15 22:13:43 2022-05-15 22:39:03 0:25:20 0:14:00 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6835140 2022-05-15 21:51:17 2022-05-15 22:13:43 2022-05-15 22:37:10 0:23:27 0:11:54 0:11:33 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6835141 2022-05-15 21:51:18 2022-05-15 22:13:43 2022-05-15 22:50:27 0:36:44 0:24:52 0:11:52 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6835142 2022-05-15 21:51:19 2022-05-15 22:14:14 2022-05-15 22:50:04 0:35:50 0:22:32 0:13:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-15T22:36:41.952741+0000 mon.a (mon.0) 198 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835143 2022-05-15 21:51:20 2022-05-15 22:14:24 2022-05-15 22:41:09 0:26:45 0:14:10 0:12:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6835144 2022-05-15 21:51:21 2022-05-15 22:15:25 2022-05-15 22:48:32 0:33:07 0:18:20 0:14:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_concurrent} 3
pass 6835145 2022-05-15 21:51:22 2022-05-15 22:16:55 2022-05-15 22:43:41 0:26:46 0:13:27 0:13:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6835146 2022-05-15 21:51:23 2022-05-15 22:17:26 2022-05-15 22:52:11 0:34:45 0:22:12 0:12:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6835147 2022-05-15 21:51:24 2022-05-15 22:17:36 2022-05-15 22:43:55 0:26:19 0:13:37 0:12:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6835148 2022-05-15 21:51:25 2022-05-15 22:18:57 2022-05-15 22:43:22 0:24:25 0:11:34 0:12:51 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_modprobe} 1
pass 6835149 2022-05-15 21:51:26 2022-05-15 22:19:27 2022-05-15 22:55:15 0:35:48 0:19:07 0:16:41 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6835150 2022-05-15 21:51:28 2022-05-15 22:23:48 2022-05-15 23:33:56 1:10:08 0:58:10 0:11:58 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6835151 2022-05-15 21:51:29 2022-05-15 22:24:28 2022-05-15 22:52:03 0:27:35 0:13:38 0:13:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
fail 6835152 2022-05-15 21:51:30 2022-05-15 22:25:09 2022-05-15 23:01:37 0:36:28 0:21:55 0:14:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-05-15T22:58:46.432283+0000 mon.a (mon.0) 270 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835153 2022-05-15 21:51:31 2022-05-15 22:28:10 2022-05-15 22:52:26 0:24:16 0:12:57 0:11:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6835154 2022-05-15 21:51:32 2022-05-15 22:29:00 2022-05-15 23:03:53 0:34:53 0:21:42 0:13:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-05-15T23:01:32.947896+0000 mon.a (mon.0) 247 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835155 2022-05-15 21:51:33 2022-05-15 22:30:21 2022-05-15 22:57:17 0:26:56 0:14:30 0:12:26 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6835156 2022-05-15 21:51:34 2022-05-15 22:31:42 2022-05-15 23:07:19 0:35:37 0:25:31 0:10:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6835157 2022-05-15 21:51:35 2022-05-15 22:31:52 2022-05-15 22:55:49 0:23:57 0:11:31 0:12:26 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_msgr_segments} 1
pass 6835158 2022-05-15 21:51:36 2022-05-15 22:33:42 2022-05-15 23:10:14 0:36:32 0:25:02 0:11:30 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6835159 2022-05-15 21:51:37 2022-05-15 22:34:53 2022-05-15 23:18:12 0:43:19 0:30:44 0:12:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6835160 2022-05-15 21:51:38 2022-05-15 22:36:54 2022-05-15 23:14:55 0:38:01 0:26:55 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6835161 2022-05-15 21:51:40 2022-05-15 22:37:24 2022-05-15 23:04:14 0:26:50 0:14:13 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6835162 2022-05-15 21:51:41 2022-05-15 22:38:15 2022-05-15 23:10:33 0:32:18 0:21:33 0:10:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6835163 2022-05-15 21:51:42 2022-05-15 22:39:05 2022-05-15 23:13:01 0:33:56 0:21:47 0:12:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-15T22:59:52.320841+0000 mon.a (mon.0) 134 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835164 2022-05-15 21:51:43 2022-05-15 22:39:15 2022-05-15 23:09:01 0:29:46 0:14:00 0:15:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6835165 2022-05-15 21:51:44 2022-05-15 22:41:16 2022-05-15 23:32:42 0:51:26 0:40:25 0:11:01 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-15T23:11:02.562480+0000 osd.3 (osd.3) 3 : cluster [WRN] 2 slow requests (by type [ 'waiting for sub ops' : 2 ] most affected pool [ 'rbd' : 2 ])" in cluster log

pass 6835166 2022-05-15 21:51:45 2022-05-15 22:41:16 2022-05-15 23:12:10 0:30:54 0:17:47 0:13:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6835167 2022-05-15 21:51:46 2022-05-15 22:41:57 2022-05-15 23:05:18 0:23:21 0:11:38 0:11:43 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6835168 2022-05-15 21:51:47 2022-05-15 22:41:57 2022-05-15 23:15:59 0:34:02 0:20:20 0:13:42 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6835169 2022-05-15 21:51:48 2022-05-15 22:43:48 2022-05-15 23:09:00 0:25:12 0:14:36 0:10:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6835170 2022-05-15 21:51:49 2022-05-15 22:43:58 2022-05-15 23:52:14 1:08:16 0:57:11 0:11:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6835171 2022-05-15 21:51:50 2022-05-15 22:44:39 2022-05-15 23:41:27 0:56:48 0:45:19 0:11:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6835172 2022-05-15 21:51:52 2022-05-15 22:46:29 2022-05-15 23:16:21 0:29:52 0:17:34 0:12:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6835173 2022-05-15 21:51:53 2022-05-15 22:48:40 2022-05-15 23:15:39 0:26:59 0:15:03 0:11:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6835174 2022-05-15 21:51:54 2022-05-15 22:50:11 2022-05-15 23:24:42 0:34:31 0:21:46 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-15T23:21:54.381028+0000 mon.a (mon.0) 281 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835175 2022-05-15 21:51:55 2022-05-15 22:50:31 2022-05-15 23:14:33 0:24:02 0:12:04 0:11:58 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_read_only} 1
pass 6835176 2022-05-15 21:51:56 2022-05-15 22:50:31 2022-05-15 23:24:50 0:34:19 0:19:57 0:14:22 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6835177 2022-05-15 21:51:57 2022-05-15 22:52:12 2022-05-15 23:47:52 0:55:40 0:43:58 0:11:42 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6835178 2022-05-15 21:51:58 2022-05-15 22:52:12 2022-05-15 23:18:16 0:26:04 0:14:23 0:11:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6835179 2022-05-15 21:51:59 2022-05-15 22:52:33 2022-05-15 23:17:00 0:24:27 0:13:20 0:11:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6835180 2022-05-15 21:52:00 2022-05-15 22:53:23 2022-05-15 23:24:58 0:31:35 0:18:34 0:13:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_concurrent} 3
pass 6835181 2022-05-15 21:52:01 2022-05-15 22:55:24 2022-05-15 23:20:51 0:25:27 0:13:08 0:12:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6835182 2022-05-15 21:52:02 2022-05-15 22:57:25 2022-05-15 23:31:32 0:34:07 0:21:53 0:12:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-15T23:28:49.694872+0000 mon.a (mon.0) 282 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835183 2022-05-15 21:52:03 2022-05-15 22:57:45 2022-05-15 23:24:04 0:26:19 0:13:43 0:12:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6835184 2022-05-15 21:52:05 2022-05-15 23:00:36 2022-05-15 23:22:49 0:22:13 0:11:50 0:10:23 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_whole_object_zeroout} 1
pass 6835185 2022-05-15 21:52:06 2022-05-15 23:00:36 2022-05-15 23:33:01 0:32:25 0:21:49 0:10:36 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6835186 2022-05-15 21:52:07 2022-05-15 23:01:47 2022-05-15 23:27:16 0:25:29 0:12:57 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6835187 2022-05-15 21:52:08 2022-05-15 23:03:58 2022-05-15 23:36:22 0:32:24 0:19:41 0:12:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6835188 2022-05-15 21:52:09 2022-05-15 23:04:18 2022-05-15 23:37:56 0:33:38 0:21:48 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-15T23:35:25.824797+0000 mon.a (mon.0) 218 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835189 2022-05-15 21:52:10 2022-05-15 23:04:29 2022-05-15 23:35:11 0:30:42 0:18:03 0:12:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6835190 2022-05-15 21:52:11 2022-05-15 23:05:29 2022-05-15 23:32:57 0:27:28 0:15:51 0:11:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6835191 2022-05-15 21:52:12 2022-05-15 23:07:20 2022-05-15 23:40:57 0:33:37 0:21:58 0:11:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-15T23:29:24.972246+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835192 2022-05-15 21:52:13 2022-05-15 23:09:10 2022-05-15 23:30:08 0:20:58 0:12:22 0:08:36 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6835193 2022-05-15 21:52:14 2022-05-15 23:09:11 2022-05-15 23:44:40 0:35:29 0:24:16 0:11:13 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6835194 2022-05-15 21:52:16 2022-05-15 23:09:11 2022-05-15 23:52:21 0:43:10 0:31:04 0:12:06 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6835195 2022-05-15 21:52:17 2022-05-15 23:09:42 2022-05-15 23:41:52 0:32:10 0:21:20 0:10:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6835196 2022-05-15 21:52:18 2022-05-15 23:10:22 2022-05-15 23:32:09 0:21:47 0:11:33 0:10:14 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6835197 2022-05-15 21:52:19 2022-05-15 23:10:42 2022-05-15 23:48:27 0:37:45 0:24:21 0:13:24 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
pass 6835198 2022-05-15 21:52:20 2022-05-15 23:12:13 2022-05-15 23:38:36 0:26:23 0:14:33 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6835199 2022-05-15 21:52:21 2022-05-15 23:13:03 2022-05-15 23:48:32 0:35:29 0:22:14 0:13:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6835200 2022-05-15 21:52:22 2022-05-15 23:15:04 2022-05-15 23:39:26 0:24:22 0:13:47 0:10:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
pass 6835201 2022-05-15 21:52:23 2022-05-15 23:15:45 2022-05-15 23:42:11 0:26:26 0:14:36 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6835202 2022-05-15 21:52:24 2022-05-15 23:16:05 2022-05-16 00:08:04 0:51:59 0:39:55 0:12:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6835203 2022-05-15 21:52:25 2022-05-15 23:16:25 2022-05-15 23:44:45 0:28:20 0:16:32 0:11:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6835204 2022-05-15 21:52:26 2022-05-15 23:17:06 2022-05-15 23:41:00 0:23:54 0:12:35 0:11:19 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_discard} 1
pass 6835205 2022-05-15 21:52:27 2022-05-15 23:17:06 2022-05-15 23:49:42 0:32:36 0:19:42 0:12:54 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6835206 2022-05-15 21:52:28 2022-05-15 23:18:17 2022-05-16 00:29:38 1:11:21 1:01:44 0:09:37 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6835207 2022-05-15 21:52:29 2022-05-15 23:18:17 2022-05-15 23:46:30 0:28:13 0:14:53 0:13:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
pass 6835208 2022-05-15 21:52:30 2022-05-15 23:20:58 2022-05-16 00:43:39 1:22:41 1:09:46 0:12:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6835209 2022-05-15 21:52:31 2022-05-15 23:24:09 2022-05-16 00:16:52 0:52:43 0:41:12 0:11:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6835210 2022-05-15 21:52:32 2022-05-15 23:24:49 2022-05-15 23:52:21 0:27:32 0:17:45 0:09:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6835211 2022-05-15 21:52:34 2022-05-15 23:25:00 2022-05-15 23:51:32 0:26:32 0:14:35 0:11:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6835212 2022-05-15 21:52:35 2022-05-15 23:25:00 2022-05-15 23:50:50 0:25:50 0:13:47 0:12:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6835213 2022-05-15 21:52:36 2022-05-15 23:27:21 2022-05-15 23:49:30 0:22:09 0:11:25 0:10:44 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_huge_image} 1
pass 6835214 2022-05-15 21:52:37 2022-05-15 23:27:21 2022-05-16 00:09:29 0:42:08 0:25:23 0:16:45 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6835215 2022-05-15 21:52:38 2022-05-15 23:31:42 2022-05-15 23:59:40 0:27:58 0:15:46 0:12:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6835216 2022-05-15 21:52:39 2022-05-15 23:32:13 2022-05-16 00:07:40 0:35:27 0:23:18 0:12:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-16T00:05:34.932500+0000 mon.a (mon.0) 234 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835217 2022-05-15 21:52:40 2022-05-15 23:32:23 2022-05-16 00:04:07 0:31:44 0:19:31 0:12:13 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6835218 2022-05-15 21:52:41 2022-05-15 23:32:54 2022-05-16 00:08:28 0:35:34 0:23:03 0:12:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-16T00:06:00.561866+0000 mon.a (mon.0) 297 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835219 2022-05-15 21:52:42 2022-05-15 23:33:04 2022-05-16 00:08:32 0:35:28 0:23:47 0:11:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6835220 2022-05-15 21:52:43 2022-05-15 23:33:05 2022-05-15 23:58:38 0:25:33 0:14:37 0:10:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_image_read} 3
pass 6835221 2022-05-15 21:52:44 2022-05-15 23:33:05 2022-05-15 23:56:27 0:23:22 0:12:33 0:10:49 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6835222 2022-05-15 21:52:45 2022-05-15 23:33:05 2022-05-16 00:07:41 0:34:36 0:23:19 0:11:17 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6835223 2022-05-15 21:52:47 2022-05-15 23:34:06 2022-05-16 00:07:57 0:33:51 0:22:01 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-05-16T00:06:43.569056+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6835224 2022-05-15 21:52:48 2022-05-15 23:35:16 2022-05-16 00:10:27 0:35:11 0:22:14 0:12:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-05-15T23:57:52.104427+0000 mon.a (mon.0) 159 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835225 2022-05-15 21:52:49 2022-05-15 23:36:27 2022-05-16 00:01:14 0:24:47 0:13:22 0:11:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6835226 2022-05-15 21:52:50 2022-05-15 23:37:57 2022-05-16 00:06:21 0:28:24 0:18:13 0:10:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6835227 2022-05-15 21:52:51 2022-05-15 23:38:38 2022-05-16 00:03:50 0:25:12 0:14:08 0:11:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6835228 2022-05-15 21:52:52 2022-05-15 23:39:28 2022-05-16 00:20:30 0:41:02 0:28:00 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6835229 2022-05-15 21:52:53 2022-05-15 23:40:59 2022-05-16 00:02:14 0:21:15 0:11:33 0:09:42 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_msgr_segments} 1
dead 6835230 2022-05-15 21:52:54 2022-05-15 23:40:59 2022-05-16 00:10:32 0:29:33 0:18:55 0:10:38 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

{'smithi018.front.sepia.ceph.com': {'_ansible_no_log': False, 'attempts': 24, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'only_upgrade': False, 'package': None, 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': True, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': 'Failed to update apt cache: unknown reason'}}

pass 6835231 2022-05-15 21:52:55 2022-05-15 23:41:30 2022-05-16 00:05:27 0:23:57 0:13:55 0:10:02 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
pass 6835232 2022-05-15 21:52:56 2022-05-15 23:42:00 2022-05-16 00:24:28 0:42:28 0:30:56 0:11:32 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_fio} 3
pass 6835233 2022-05-15 21:52:58 2022-05-15 23:42:21 2022-05-16 00:20:12 0:37:51 0:25:48 0:12:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_data_pool} 3
fail 6835234 2022-05-15 21:52:59 2022-05-15 23:44:41 2022-05-16 00:16:50 0:32:09 0:21:16 0:10:53 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

"2022-05-16T00:06:16.810985+0000 mon.a (mon.0) 97 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6835235 2022-05-15 21:53:00 2022-05-15 23:44:52 2022-05-16 01:02:13 1:17:21 1:05:10 0:12:11 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6835236 2022-05-15 21:53:01 2022-05-15 23:46:32 2022-05-16 00:13:30 0:26:58 0:14:12 0:12:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6835237 2022-05-15 21:53:02 2022-05-15 23:47:53 2022-05-16 00:20:07 0:32:14 0:20:56 0:11:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6835238 2022-05-15 21:53:03 2022-05-15 23:48:34 2022-05-16 00:16:48 0:28:14 0:14:38 0:13:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
fail 6835239 2022-05-15 21:53:04 2022-05-15 23:48:34 2022-05-16 00:23:09 0:34:35 0:21:58 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-05-16T00:10:33.785414+0000 mon.a (mon.0) 192 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6835240 2022-05-15 21:53:05 2022-05-15 23:49:45 2022-05-16 00:24:23 0:34:38 0:21:47 0:12:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-16T00:12:32.393609+0000 mon.a (mon.0) 163 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

fail 6835241 2022-05-15 21:53:06 2022-05-15 23:50:55 2022-05-16 00:26:06 0:35:11 0:21:57 0:13:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-16T00:24:09.107534+0000 mon.a (mon.0) 305 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835242 2022-05-15 21:53:07 2022-05-15 23:51:36 2022-05-16 00:15:42 0:24:06 0:12:05 0:12:01 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6835243 2022-05-15 21:53:08 2022-05-15 23:51:36 2022-05-16 00:23:51 0:32:15 0:20:43 0:11:32 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6835244 2022-05-15 21:53:10 2022-05-15 23:52:16 2022-05-16 00:17:57 0:25:41 0:14:46 0:10:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
fail 6835245 2022-05-15 21:53:11 2022-05-15 23:52:27 2022-05-16 00:27:06 0:34:39 0:21:50 0:12:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-16T00:24:39.699087+0000 mon.a (mon.0) 290 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6835246 2022-05-15 21:53:12 2022-05-15 23:52:27 2022-05-16 00:32:19 0:39:52 0:21:43 0:18:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-05-16T00:30:06.306103+0000 mon.a (mon.0) 261 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6835247 2022-05-15 21:53:13 2022-05-15 23:58:49 2022-05-16 00:29:11 0:30:22 0:17:32 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6835248 2022-05-15 21:53:14 2022-05-15 23:59:49 2022-05-16 00:34:44 0:34:55 0:22:01 0:12:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-16T00:32:01.770507+0000 mon.a (mon.0) 226 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835249 2022-05-15 21:53:15 2022-05-16 00:01:20 2022-05-16 00:29:43 0:28:23 0:14:22 0:14:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6835250 2022-05-15 21:53:16 2022-05-16 00:04:01 2022-05-16 00:27:22 0:23:21 0:12:30 0:10:51 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6835251 2022-05-15 21:53:17 2022-05-16 00:04:01 2022-05-16 00:45:38 0:41:37 0:29:46 0:11:51 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6835252 2022-05-15 21:53:18 2022-05-16 00:04:11 2022-05-16 00:31:03 0:26:52 0:14:12 0:12:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_symlinks} 3
fail 6835253 2022-05-15 21:53:19 2022-05-16 00:05:32 2022-05-16 00:40:06 0:34:34 0:22:16 0:12:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-16T00:37:39.450041+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6835254 2022-05-15 21:53:20 2022-05-16 00:06:23 2022-05-16 00:43:16 0:36:53 0:21:52 0:15:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-16T00:40:17.718320+0000 mon.a (mon.0) 279 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835255 2022-05-15 21:53:21 2022-05-16 00:07:43 2022-05-16 00:35:03 0:27:20 0:13:30 0:13:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6835256 2022-05-15 21:53:22 2022-05-16 00:07:44 2022-05-16 00:43:52 0:36:08 0:22:19 0:13:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-16T00:31:31.976121+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835257 2022-05-15 21:53:24 2022-05-16 00:08:04 2022-05-16 00:35:42 0:27:38 0:14:49 0:12:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
pass 6835258 2022-05-15 21:53:25 2022-05-16 00:08:04 2022-05-16 00:33:34 0:25:30 0:12:05 0:13:25 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_whole_object_zeroout} 1
pass 6835259 2022-05-15 21:53:26 2022-05-16 00:08:05 2022-05-16 00:39:57 0:31:52 0:18:43 0:13:09 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6835260 2022-05-15 21:53:27 2022-05-16 00:08:35 2022-05-16 01:01:51 0:53:16 0:41:16 0:12:00 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6835261 2022-05-15 21:53:28 2022-05-16 00:08:35 2022-05-16 00:35:35 0:27:00 0:13:30 0:13:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_kernel} 3
pass 6835262 2022-05-15 21:53:29 2022-05-16 00:09:36 2022-05-16 00:41:32 0:31:56 0:19:39 0:12:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6835263 2022-05-15 21:53:30 2022-05-16 00:10:36 2022-05-16 00:36:26 0:25:50 0:13:04 0:12:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6835264 2022-05-15 21:53:31 2022-05-16 00:10:37 2022-05-16 00:41:11 0:30:34 0:17:54 0:12:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6835265 2022-05-15 21:53:32 2022-05-16 00:13:38 2022-05-16 00:45:08 0:31:30 0:15:32 0:15:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6835266 2022-05-15 21:53:33 2022-05-16 00:16:48 2022-05-16 00:57:12 0:40:24 0:27:47 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6835267 2022-05-15 21:53:34 2022-05-16 00:16:59 2022-05-16 00:40:05 0:23:06 0:12:29 0:10:37 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_deep_flatten} 1
pass 6835268 2022-05-15 21:53:36 2022-05-16 00:16:59 2022-05-16 00:54:42 0:37:43 0:27:16 0:10:27 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6835269 2022-05-15 21:53:37 2022-05-16 00:17:00 2022-05-16 00:59:25 0:42:25 0:30:52 0:11:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6835270 2022-05-15 21:53:38 2022-05-16 00:18:00 2022-05-16 00:53:41 0:35:41 0:22:06 0:13:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
fail 6835271 2022-05-15 21:53:39 2022-05-16 00:20:11 2022-05-16 00:43:45 0:23:34 0:11:58 0:11:36 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
Failure Reason:

Command failed on smithi097 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t'

pass 6835272 2022-05-15 21:53:40 2022-05-16 00:20:21 2022-05-16 00:56:13 0:35:52 0:23:46 0:12:06 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
fail 6835273 2022-05-15 21:53:41 2022-05-16 00:20:32 2022-05-16 00:54:24 0:33:52 0:21:59 0:11:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-05-16T00:53:09.611697+0000 mon.a (mon.0) 299 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6835274 2022-05-15 21:53:42 2022-05-16 00:23:13 2022-05-16 00:57:23 0:34:10 0:22:06 0:12:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-16T00:54:17.348221+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835275 2022-05-15 21:53:43 2022-05-16 00:23:53 2022-05-16 00:50:35 0:26:42 0:14:10 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
pass 6835276 2022-05-15 21:53:44 2022-05-16 00:24:34 2022-05-16 00:49:51 0:25:17 0:14:29 0:10:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6835277 2022-05-15 21:53:46 2022-05-16 00:24:34 2022-05-16 01:18:43 0:54:09 0:40:01 0:14:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6835278 2022-05-15 21:53:47 2022-05-16 00:26:15 2022-05-16 00:55:44 0:29:29 0:16:42 0:12:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6835279 2022-05-15 21:53:48 2022-05-16 00:27:15 2022-05-16 00:50:15 0:23:00 0:12:49 0:10:11 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6835280 2022-05-15 21:53:49 2022-05-16 00:27:15 2022-05-16 00:58:40 0:31:25 0:18:06 0:13:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6835281 2022-05-15 21:53:50 2022-05-16 00:29:16 2022-05-16 01:03:26 0:34:10 0:21:47 0:12:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-05-16T00:50:23.543760+0000 mon.a (mon.0) 144 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835282 2022-05-15 21:53:51 2022-05-16 00:29:47 2022-05-16 01:48:17 1:18:30 1:05:27 0:13:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6835283 2022-05-15 21:53:52 2022-05-16 00:29:47 2022-05-16 01:24:41 0:54:54 0:41:42 0:13:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6835284 2022-05-15 21:53:53 2022-05-16 00:31:08 2022-05-16 01:05:48 0:34:40 0:21:55 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-05-16T00:53:51.694971+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835285 2022-05-15 21:53:54 2022-05-16 00:32:28 2022-05-16 00:59:50 0:27:22 0:14:59 0:12:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6835286 2022-05-15 21:53:56 2022-05-16 00:33:39 2022-05-16 01:00:07 0:26:28 0:14:04 0:12:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6835287 2022-05-15 21:53:57 2022-05-16 00:34:49 2022-05-16 00:58:32 0:23:43 0:11:27 0:12:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_huge_image} 1
pass 6835288 2022-05-15 21:53:58 2022-05-16 00:35:10 2022-05-16 01:11:35 0:36:25 0:23:47 0:12:38 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6835289 2022-05-15 21:53:59 2022-05-16 00:35:40 2022-05-16 01:53:33 1:17:53 1:06:15 0:11:38 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6835290 2022-05-15 21:54:00 2022-05-16 00:35:50 2022-05-16 01:04:33 0:28:43 0:14:30 0:14:13 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6835291 2022-05-15 21:54:01 2022-05-16 00:36:31 2022-05-16 01:15:29 0:38:58 0:22:40 0:16:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-16T01:12:32.825508+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835292 2022-05-15 21:54:02 2022-05-16 00:40:02 2022-05-16 01:11:43 0:31:41 0:19:06 0:12:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
fail 6835293 2022-05-15 21:54:03 2022-05-16 00:40:12 2022-05-16 01:15:41 0:35:29 0:22:46 0:12:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-16T01:12:47.429742+0000 mon.a (mon.0) 331 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6835294 2022-05-15 21:54:04 2022-05-16 00:40:12 2022-05-16 01:14:51 0:34:39 0:23:23 0:11:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
fail 6835295 2022-05-15 21:54:05 2022-05-16 00:41:13 2022-05-16 01:14:41 0:33:28 0:22:29 0:10:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
Failure Reason:

"2022-05-16T01:12:42.853487+0000 mon.a (mon.0) 217 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835296 2022-05-15 21:54:06 2022-05-16 00:41:33 2022-05-16 01:04:29 0:22:56 0:12:01 0:10:55 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_modprobe} 1
pass 6835297 2022-05-15 21:54:08 2022-05-16 00:43:24 2022-05-16 01:15:17 0:31:53 0:21:16 0:10:37 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6835298 2022-05-15 21:54:09 2022-05-16 00:43:44 2022-05-16 01:09:28 0:25:44 0:13:52 0:11:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_kernel} 3
fail 6835299 2022-05-15 21:54:10 2022-05-16 00:43:55 2022-05-16 01:17:25 0:33:30 0:22:26 0:11:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-05-16T01:05:03.960842+0000 mon.a (mon.0) 139 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835300 2022-05-15 21:54:11 2022-05-16 00:43:55 2022-05-16 01:11:29 0:27:34 0:13:01 0:14:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6835301 2022-05-15 21:54:12 2022-05-16 00:45:16 2022-05-16 01:15:10 0:29:54 0:18:06 0:11:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6835302 2022-05-15 21:54:13 2022-05-16 00:45:46 2022-05-16 01:15:23 0:29:37 0:14:32 0:15:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
fail 6835303 2022-05-15 21:54:14 2022-05-16 00:49:57 2022-05-16 01:24:44 0:34:47 0:21:51 0:12:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-16T01:22:14.371597+0000 mon.a (mon.0) 281 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835304 2022-05-15 21:54:15 2022-05-16 00:50:38 2022-05-16 01:11:34 0:20:56 0:11:41 0:09:15 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6835305 2022-05-15 21:54:16 2022-05-16 00:50:38 2022-05-16 01:31:22 0:40:44 0:26:49 0:13:55 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6835306 2022-05-15 21:54:17 2022-05-16 00:53:49 2022-05-16 01:27:38 0:33:49 0:21:52 0:11:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-05-16T01:25:49.254865+0000 mon.a (mon.0) 283 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835307 2022-05-15 21:54:18 2022-05-16 00:54:29 2022-05-16 01:32:45 0:38:16 0:25:35 0:12:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
pass 6835308 2022-05-15 21:54:19 2022-05-16 00:54:50 2022-05-16 01:22:31 0:27:41 0:14:09 0:13:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6835309 2022-05-15 21:54:20 2022-05-16 00:55:50 2022-05-16 01:28:00 0:32:10 0:21:02 0:11:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6835310 2022-05-15 21:54:21 2022-05-16 00:56:21 2022-05-16 01:31:28 0:35:07 0:21:51 0:13:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-16T01:29:15.478805+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835311 2022-05-15 21:54:22 2022-05-16 00:57:21 2022-05-16 01:22:48 0:25:27 0:14:28 0:10:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6835312 2022-05-15 21:54:23 2022-05-16 00:57:32 2022-05-16 01:32:06 0:34:34 0:21:56 0:12:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-16T01:29:21.609354+0000 mon.a (mon.0) 205 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6835313 2022-05-15 21:54:25 2022-05-16 00:58:42 2022-05-16 01:33:04 0:34:22 0:22:12 0:12:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-16T01:31:09.850494+0000 mon.a (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835314 2022-05-15 21:54:26 2022-05-16 00:59:33 2022-05-16 01:23:03 0:23:30 0:11:50 0:11:40 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_parent_overlap} 1
pass 6835315 2022-05-15 21:54:27 2022-05-16 00:59:33 2022-05-16 01:32:04 0:32:31 0:18:58 0:13:33 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6835316 2022-05-15 21:54:28 2022-05-16 00:59:54 2022-05-16 01:53:46 0:53:52 0:41:59 0:11:53 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6835317 2022-05-15 21:54:29 2022-05-16 01:00:14 2022-05-16 01:27:18 0:27:04 0:14:10 0:12:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6835318 2022-05-15 21:54:30 2022-05-16 01:01:55 2022-05-16 02:13:53 1:11:58 0:59:24 0:12:34 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6835319 2022-05-15 21:54:31 2022-05-16 01:02:15 2022-05-16 02:01:09 0:58:54 0:45:57 0:12:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6835320 2022-05-15 21:54:32 2022-05-16 01:03:36 2022-05-16 01:34:37 0:31:01 0:17:54 0:13:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6835321 2022-05-15 21:54:33 2022-05-16 01:04:36 2022-05-16 01:39:16 0:34:40 0:21:46 0:12:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-16T01:26:24.544481+0000 mon.a (mon.0) 137 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835322 2022-05-15 21:54:34 2022-05-16 01:05:57 2022-05-16 01:35:16 0:29:19 0:13:53 0:15:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6835323 2022-05-15 21:54:35 2022-05-16 01:09:38 2022-05-16 01:30:49 0:21:11 0:11:43 0:09:28 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_read_only} 1
pass 6835324 2022-05-15 21:54:36 2022-05-16 01:09:38 2022-05-16 01:47:39 0:38:01 0:24:00 0:14:01 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6835325 2022-05-15 21:54:37 2022-05-16 01:11:39 2022-05-16 01:37:21 0:25:42 0:15:04 0:10:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6835326 2022-05-15 21:54:38 2022-05-16 01:11:39 2022-05-16 01:37:29 0:25:50 0:14:04 0:11:46 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6835327 2022-05-15 21:54:39 2022-05-16 01:11:50 2022-05-16 03:47:54 2:36:04 2:22:55 0:13:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-16T03:19:43.019639+0000 osd.3 (osd.3) 3 : cluster [WRN] 2 slow requests (by type [ 'started' : 2 ] most affected pool [ 'rbd' : 2 ])" in cluster log

pass 6835328 2022-05-15 21:54:41 2022-05-16 01:14:50 2022-05-16 01:40:32 0:25:42 0:13:30 0:12:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6835329 2022-05-15 21:54:42 2022-05-16 01:15:01 2022-05-16 01:48:38 0:33:37 0:21:59 0:11:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-16T01:36:20.365290+0000 mon.a (mon.0) 172 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6835330 2022-05-15 21:54:43 2022-05-16 01:15:11 2022-05-16 01:50:53 0:35:42 0:22:29 0:13:13 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-05-16T01:37:39.113156+0000 mon.a (mon.0) 139 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6835331 2022-05-15 21:54:44 2022-05-16 01:15:22 2022-05-16 01:39:38 0:24:16 0:12:20 0:11:56 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
fail 6835332 2022-05-15 21:54:45 2022-05-16 01:15:22 2022-05-16 01:48:51 0:33:29 0:21:59 0:11:30 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2022-05-16T01:47:12.450274+0000 mon.a (mon.0) 252 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835333 2022-05-15 21:54:46 2022-05-16 01:15:32 2022-05-16 01:41:04 0:25:32 0:13:31 0:12:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6835334 2022-05-15 21:54:47 2022-05-16 01:15:33 2022-05-16 01:47:04 0:31:31 0:19:22 0:12:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6835335 2022-05-15 21:54:48 2022-05-16 01:15:43 2022-05-16 01:50:57 0:35:14 0:22:16 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-16T01:48:35.591882+0000 mon.a (mon.0) 305 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6835336 2022-05-15 21:54:49 2022-05-16 01:17:34 2022-05-16 01:49:18 0:31:44 0:18:05 0:13:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6835337 2022-05-15 21:54:51 2022-05-16 01:18:44 2022-05-16 01:51:19 0:32:35 0:16:20 0:16:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6835338 2022-05-15 21:54:52 2022-05-16 01:22:35 2022-05-16 02:02:22 0:39:47 0:29:30 0:10:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3