Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6812164 2022-04-29 03:20:13 2022-05-04 06:53:31 2022-05-04 07:12:52 0:19:21 0:08:49 0:10:32 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_deep_flatten} 1
pass 6812165 2022-04-29 03:20:14 2022-05-04 06:53:31 2022-05-04 07:24:58 0:31:27 0:18:28 0:12:59 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6812166 2022-04-29 03:20:15 2022-05-04 06:55:02 2022-05-04 07:17:21 0:22:19 0:10:09 0:12:10 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6812167 2022-04-29 03:20:16 2022-05-04 06:55:32 2022-05-04 07:34:38 0:39:06 0:27:38 0:11:28 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6812168 2022-04-29 03:20:17 2022-05-04 06:57:03 2022-05-04 07:27:28 0:30:25 0:18:27 0:11:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_data_pool} 3
fail 6812169 2022-04-29 03:20:18 2022-05-04 06:57:53 2022-05-04 11:47:33 4:49:40 4:35:53 0:13:47 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi146 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludewaxu7139 -r -- '-g auto -g blockdev -x clone'"

pass 6812170 2022-04-29 03:20:19 2022-05-04 07:01:34 2022-05-04 07:53:41 0:52:07 0:41:03 0:11:04 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6812171 2022-04-29 03:20:20 2022-05-04 07:02:05 2022-05-04 07:09:17 0:07:12 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6812172 2022-04-29 03:20:21 2022-05-04 07:02:25 2022-05-04 07:21:11 0:18:46 0:08:45 0:10:01 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6812173 2022-04-29 03:20:22 2022-05-04 07:02:25 2022-05-04 07:24:26 0:22:01 0:11:09 0:10:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6812174 2022-04-29 03:20:23 2022-05-04 07:02:56 2022-05-04 07:33:21 0:30:25 0:19:03 0:11:22 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-04T07:20:59.851113+0000 mon.a (mon.0) 124 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812175 2022-04-29 03:20:24 2022-05-04 07:03:46 2022-05-04 07:26:47 0:23:01 0:10:18 0:12:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6812176 2022-04-29 03:20:25 2022-05-04 07:05:17 2022-05-04 07:37:48 0:32:31 0:18:33 0:13:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-05-04T07:36:27.343181+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812177 2022-04-29 03:20:26 2022-05-04 07:06:17 2022-05-04 07:58:10 0:51:53 0:36:51 0:15:02 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6812178 2022-04-29 03:20:27 2022-05-04 07:06:38 2022-05-04 07:34:13 0:27:35 0:13:03 0:14:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6812179 2022-04-29 03:20:28 2022-05-04 07:06:48 2022-05-04 07:30:19 0:23:31 0:08:52 0:14:39 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_discard} 1
pass 6812180 2022-04-29 03:20:29 2022-05-04 07:06:58 2022-05-04 07:38:30 0:31:32 0:16:51 0:14:41 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6812181 2022-04-29 03:20:30 2022-05-04 07:06:59 2022-05-04 07:40:57 0:33:58 0:18:28 0:15:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-05-04T07:38:29.067580+0000 mon.a (mon.0) 326 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6812182 2022-04-29 03:20:31 2022-05-04 07:09:19 2022-05-04 07:41:06 0:31:47 0:18:34 0:13:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-04T07:28:28.229128+0000 mon.a (mon.0) 116 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812183 2022-04-29 03:20:32 2022-05-04 07:09:40 2022-05-04 08:01:58 0:52:18 0:39:16 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6812184 2022-04-29 03:20:33 2022-05-04 07:10:11 2022-05-04 07:38:14 0:28:03 0:14:42 0:13:21 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6812185 2022-04-29 03:20:34 2022-05-04 07:10:51 2022-05-04 07:34:41 0:23:50 0:11:43 0:12:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6812186 2022-04-29 03:20:35 2022-05-04 07:11:01 2022-05-04 07:34:43 0:23:42 0:10:32 0:13:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6812187 2022-04-29 03:20:36 2022-05-04 07:13:12 2022-05-04 07:35:30 0:22:18 0:08:07 0:14:11 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6812188 2022-04-29 03:20:37 2022-05-04 07:13:12 2022-05-04 07:47:15 0:34:03 0:21:58 0:12:05 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6812189 2022-04-29 03:20:38 2022-05-04 07:13:53 2022-05-04 07:44:07 0:30:14 0:18:29 0:11:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-04T07:42:33.353618+0000 mon.a (mon.0) 325 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6812190 2022-04-29 03:20:39 2022-05-04 07:14:03 2022-05-04 07:43:47 0:29:44 0:18:23 0:11:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-04T07:31:54.264336+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812191 2022-04-29 03:20:40 2022-05-04 07:14:14 2022-05-04 07:43:11 0:28:57 0:15:13 0:13:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
pass 6812192 2022-04-29 03:20:41 2022-05-04 07:15:04 2022-05-04 07:39:29 0:24:25 0:09:54 0:14:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6812193 2022-04-29 03:20:42 2022-05-04 07:17:25 2022-05-04 07:51:24 0:33:59 0:19:18 0:14:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6812194 2022-04-29 03:20:43 2022-05-04 07:19:25 2022-05-04 07:42:10 0:22:45 0:10:15 0:12:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6812195 2022-04-29 03:20:44 2022-05-04 07:19:56 2022-05-04 07:39:13 0:19:17 0:08:18 0:10:59 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_modprobe} 1
pass 6812196 2022-04-29 03:20:45 2022-05-04 07:19:56 2022-05-04 07:50:00 0:30:04 0:14:50 0:15:14 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6812197 2022-04-29 03:20:46 2022-05-04 07:24:27 2022-05-04 08:33:00 1:08:33 0:56:24 0:12:09 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6812198 2022-04-29 03:20:47 2022-05-04 07:25:07 2022-05-04 07:48:21 0:23:14 0:09:58 0:13:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_kernel} 3
pass 6812199 2022-04-29 03:20:48 2022-05-04 07:26:48 2022-05-04 07:57:02 0:30:14 0:16:50 0:13:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6812200 2022-04-29 03:20:49 2022-05-04 07:27:28 2022-05-04 07:49:02 0:21:34 0:09:35 0:11:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6812201 2022-04-29 03:20:50 2022-05-04 07:27:29 2022-05-04 07:55:09 0:27:40 0:14:56 0:12:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6812202 2022-04-29 03:20:51 2022-05-04 07:29:19 2022-05-04 08:01:21 0:32:02 0:14:23 0:17:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
fail 6812203 2022-04-29 03:20:52 2022-05-04 07:33:00 2022-05-04 08:02:53 0:29:53 0:18:37 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-04T08:01:25.052001+0000 mon.a (mon.0) 300 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812204 2022-04-29 03:20:53 2022-05-04 07:33:31 2022-05-04 07:52:51 0:19:20 0:08:33 0:10:47 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_msgr_segments} 1
pass 6812205 2022-04-29 03:20:54 2022-05-04 07:33:31 2022-05-04 08:07:44 0:34:13 0:21:59 0:12:14 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
fail 6812206 2022-04-29 03:20:55 2022-05-04 07:34:21 2022-05-04 08:06:12 0:31:51 0:18:59 0:12:52 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-05-04T07:53:41.632747+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6812207 2022-04-29 03:20:56 2022-05-04 07:34:42 2022-05-04 08:06:28 0:31:46 0:18:47 0:12:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-05-04T08:04:14.059663+0000 mon.a (mon.0) 220 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812208 2022-04-29 03:20:57 2022-05-04 07:34:42 2022-05-04 08:00:18 0:25:36 0:13:53 0:11:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6812209 2022-04-29 03:20:58 2022-05-04 07:34:52 2022-05-04 08:11:09 0:36:17 0:23:15 0:13:02 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6812210 2022-04-29 03:20:59 2022-05-04 07:37:53 2022-05-04 08:03:56 0:26:03 0:14:40 0:11:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
pass 6812211 2022-04-29 03:21:00 2022-05-04 07:38:23 2022-05-04 08:04:07 0:25:44 0:14:43 0:11:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6812212 2022-04-29 03:21:01 2022-05-04 07:38:34 2022-05-04 08:33:28 0:54:54 0:40:35 0:14:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6812213 2022-04-29 03:21:02 2022-05-04 07:39:34 2022-05-04 08:13:05 0:33:31 0:19:11 0:14:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-04T07:59:58.448605+0000 mon.a (mon.0) 150 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812214 2022-04-29 03:21:03 2022-05-04 07:41:05 2022-05-04 08:04:38 0:23:33 0:12:33 0:11:00 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6812215 2022-04-29 03:21:04 2022-05-04 07:41:05 2022-05-04 08:10:34 0:29:29 0:18:38 0:10:51 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6812216 2022-04-29 03:21:05 2022-05-04 07:41:16 2022-05-04 08:08:35 0:27:19 0:13:49 0:13:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6812217 2022-04-29 03:21:06 2022-05-04 07:42:16 2022-05-04 08:55:28 1:13:12 0:56:56 0:16:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6812218 2022-04-29 03:21:07 2022-05-04 07:43:17 2022-05-04 08:41:37 0:58:20 0:42:02 0:16:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6812219 2022-04-29 03:21:08 2022-05-04 07:43:57 2022-05-04 08:16:13 0:32:16 0:14:32 0:17:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6812220 2022-04-29 03:21:09 2022-05-04 07:44:17 2022-05-04 08:24:40 0:40:23 0:14:16 0:26:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6812221 2022-04-29 03:21:10 2022-05-04 07:47:18 2022-05-04 08:31:25 0:44:07 0:20:40 0:23:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-04T08:29:27.453773+0000 mon.a (mon.0) 297 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812222 2022-04-29 03:21:11 2022-05-04 07:47:58 2022-05-04 08:24:25 0:36:27 0:12:47 0:23:40 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_read_only} 1
pass 6812223 2022-04-29 03:21:12 2022-05-04 07:47:59 2022-05-04 08:28:01 0:40:02 0:18:11 0:21:51 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6812224 2022-04-29 03:21:13 2022-05-04 07:48:29 2022-05-04 08:53:00 1:04:31 0:41:33 0:22:58 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6812225 2022-04-29 03:21:14 2022-05-04 07:49:10 2022-05-04 08:23:41 0:34:31 0:13:03 0:21:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6812226 2022-04-29 03:21:15 2022-05-04 07:50:10 2022-05-04 08:23:25 0:33:15 0:13:02 0:20:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6812227 2022-04-29 03:21:16 2022-05-04 07:51:31 2022-05-04 12:13:39 4:22:08 4:01:22 0:20:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

Command failed (workunit test rbd/concurrent.sh) on smithi109 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2f322761b2322e2c35dab6de99a1468a9e535948 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/concurrent.sh'

pass 6812228 2022-04-29 03:21:17 2022-05-04 07:53:51 2022-05-04 08:25:10 0:31:19 0:12:52 0:18:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6812229 2022-04-29 03:21:18 2022-05-04 07:55:12 2022-05-04 08:34:43 0:39:31 0:23:41 0:15:50 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-04T08:20:53.510094+0000 osd.1 (osd.1) 3 : cluster [WRN] 1 slow requests (by type [ 'started' : 1 ] most affected pool [ 'rbd' : 1 ])" in cluster log

pass 6812230 2022-04-29 03:21:19 2022-05-04 07:57:02 2022-05-04 08:23:52 0:26:50 0:13:00 0:13:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6812231 2022-04-29 03:21:20 2022-05-04 07:58:13 2022-05-04 08:23:39 0:25:26 0:12:23 0:13:03 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6812232 2022-04-29 03:21:21 2022-05-04 07:58:13 2022-05-04 08:31:55 0:33:42 0:19:10 0:14:32 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6812233 2022-04-29 03:21:22 2022-05-04 08:00:24 2022-05-04 08:25:48 0:25:24 0:11:27 0:13:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6812234 2022-04-29 03:21:23 2022-05-04 08:01:24 2022-05-04 08:29:45 0:28:21 0:16:10 0:12:11 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6812235 2022-04-29 03:21:24 2022-05-04 08:26:17 681 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6812236 2022-04-29 03:21:25 2022-05-04 08:02:55 2022-05-04 08:31:44 0:28:49 0:16:26 0:12:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6812237 2022-04-29 03:21:26 2022-05-04 08:04:07 2022-05-04 08:27:48 0:23:41 0:12:43 0:10:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6812238 2022-04-29 03:21:27 2022-05-04 08:04:17 2022-05-04 08:42:19 0:38:02 0:26:10 0:11:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6812239 2022-04-29 03:21:28 2022-05-04 08:04:47 2022-05-04 08:27:30 0:22:43 0:10:01 0:12:42 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6812240 2022-04-29 03:21:29 2022-05-04 08:06:18 2022-05-04 08:38:03 0:31:45 0:21:01 0:10:44 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6812241 2022-04-29 03:21:30 2022-05-04 08:06:38 2022-05-04 08:39:19 0:32:41 0:18:34 0:14:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_fio} 3
Failure Reason:

"2022-05-04T08:36:37.214103+0000 mon.a (mon.0) 280 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812242 2022-04-29 03:21:31 2022-05-04 08:07:49 2022-05-04 08:42:55 0:35:06 0:18:06 0:17:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6812243 2022-04-29 03:21:32 2022-05-04 08:08:39 2022-05-04 08:30:07 0:21:28 0:08:27 0:13:01 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
fail 6812244 2022-04-29 03:21:33 2022-05-04 08:08:40 2022-05-04 08:44:11 0:35:31 0:18:47 0:16:44 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
Failure Reason:

"2022-05-04T08:42:01.614308+0000 mon.a (mon.0) 248 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812245 2022-04-29 03:21:34 2022-05-04 08:10:40 2022-05-04 08:37:34 0:26:54 0:11:06 0:15:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6812246 2022-04-29 03:21:35 2022-05-04 08:11:01 2022-05-04 08:44:46 0:33:45 0:18:54 0:14:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-04T08:42:40.821742+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6812247 2022-04-29 03:21:36 2022-05-04 08:11:11 2022-05-04 08:44:49 0:33:38 0:18:30 0:15:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-04T08:43:02.539105+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812248 2022-04-29 03:21:37 2022-05-04 08:13:12 2022-05-04 08:40:26 0:27:14 0:10:55 0:16:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6812249 2022-04-29 03:21:38 2022-05-04 08:16:23 2022-05-04 09:09:50 0:53:27 0:36:23 0:17:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6812250 2022-04-29 03:21:39 2022-05-04 08:21:44 2022-05-04 08:49:38 0:27:54 0:12:55 0:14:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6812251 2022-04-29 03:21:40 2022-05-04 08:23:35 2022-05-04 08:43:10 0:19:35 0:08:45 0:10:50 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_discard} 1
pass 6812252 2022-04-29 03:21:41 2022-05-04 08:51:32 981 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6812253 2022-04-29 03:21:42 2022-05-04 08:23:46 2022-05-04 09:39:40 1:15:54 1:04:29 0:11:25 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6812254 2022-04-29 03:21:43 2022-05-04 08:23:56 2022-05-04 08:54:18 0:30:22 0:18:32 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-05-04T08:42:34.190917+0000 mon.a (mon.0) 192 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6812255 2022-04-29 03:21:44 2022-05-04 08:24:47 2022-05-04 08:57:16 0:32:29 0:18:35 0:13:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-04T08:54:13.090460+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812256 2022-04-29 03:21:45 2022-05-04 08:25:17 2022-05-04 09:16:22 0:51:05 0:38:20 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6812257 2022-04-29 03:21:46 2022-05-04 08:25:57 2022-05-04 08:52:02 0:26:05 0:14:32 0:11:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6812258 2022-04-29 03:21:47 2022-05-04 08:26:28 2022-05-04 08:51:19 0:24:51 0:11:25 0:13:26 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6812259 2022-04-29 03:21:49 2022-05-04 08:27:48 2022-05-04 08:49:39 0:21:51 0:10:56 0:10:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6812260 2022-04-29 03:21:50 2022-05-04 08:28:09 2022-05-04 08:47:38 0:19:29 0:08:28 0:11:01 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_huge_image} 1
pass 6812261 2022-04-29 03:21:51 2022-05-04 08:28:09 2022-05-04 09:03:30 0:35:21 0:23:20 0:12:01 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
fail 6812262 2022-04-29 03:21:52 2022-05-04 08:29:50 2022-05-04 09:00:32 0:30:42 0:18:28 0:12:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-04T08:58:53.521259+0000 mon.a (mon.0) 256 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6812263 2022-04-29 03:21:53 2022-05-04 08:30:10 2022-05-04 08:52:54 0:22:44 0:10:24 0:12:20 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6812264 2022-04-29 03:21:54 2022-05-04 08:31:30 2022-05-04 08:57:26 0:25:56 0:14:50 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6812265 2022-04-29 03:21:55 2022-05-04 08:31:51 2022-05-04 09:01:23 0:29:32 0:18:44 0:10:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-04T08:59:34.935432+0000 mon.a (mon.0) 242 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812266 2022-04-29 03:21:56 2022-05-04 08:32:01 2022-05-04 09:02:56 0:30:55 0:19:02 0:11:53 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6812267 2022-04-29 03:21:57 2022-05-04 08:33:02 2022-05-04 08:55:31 0:22:29 0:10:11 0:12:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_image_read} 3
pass 6812268 2022-04-29 03:21:58 2022-05-04 08:33:32 2022-05-04 08:53:15 0:19:43 0:08:15 0:11:28 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6812269 2022-04-29 03:21:59 2022-05-04 08:33:32 2022-05-04 09:04:16 0:30:44 0:18:34 0:12:10 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6812270 2022-04-29 03:22:00 2022-05-04 08:34:53 2022-05-04 08:57:23 0:22:30 0:09:48 0:12:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
pass 6812271 2022-04-29 03:22:01 2022-05-04 08:37:44 2022-05-04 09:05:43 0:27:59 0:17:09 0:10:50 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6812272 2022-04-29 03:22:02 2022-05-04 08:38:04 2022-05-04 09:00:37 0:22:33 0:09:36 0:12:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6812273 2022-04-29 03:22:03 2022-05-04 08:39:25 2022-05-04 09:06:38 0:27:13 0:14:51 0:12:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6812274 2022-04-29 03:22:04 2022-05-04 08:40:35 2022-05-04 09:03:29 0:22:54 0:11:00 0:11:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
fail 6812275 2022-04-29 03:22:05 2022-05-04 08:41:46 2022-05-04 09:12:01 0:30:15 0:18:35 0:11:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-04T09:09:49.519995+0000 mon.a (mon.0) 284 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812276 2022-04-29 03:22:06 2022-04-29 06:23:08 2022-04-29 06:44:34 0:21:26 0:11:57 0:09:29 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_msgr_segments} 1
pass 6812277 2022-04-29 03:22:07 2022-05-04 08:42:26 2022-05-04 09:17:20 0:34:54 0:22:08 0:12:46 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6812278 2022-04-29 03:22:08 2022-05-04 08:42:56 2022-05-04 09:05:30 0:22:34 0:10:46 0:11:48 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
pass 6812279 2022-04-29 03:22:09 2022-05-04 08:44:17 2022-05-04 09:24:40 0:40:23 0:28:39 0:11:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_fio} 3
fail 6812280 2022-04-29 03:22:10 2022-05-04 08:44:57 2022-05-04 09:14:05 0:29:08 0:18:36 0:10:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-05-04T09:12:27.089504+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6812281 2022-04-29 03:22:11 2022-05-04 08:44:58 2022-05-04 13:47:54 5:02:56 4:45:43 0:17:13 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy-rxbounce} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi141 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude3fse9h20 -r -- '-g auto -g blockdev -x clone'"

pass 6812282 2022-04-29 03:22:12 2022-05-04 08:49:39 2022-05-04 10:07:23 1:17:44 1:06:18 0:11:26 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6812283 2022-04-29 03:22:13 2022-05-04 08:49:49 2022-05-04 09:12:54 0:23:05 0:11:22 0:11:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6812284 2022-04-29 03:22:14 2022-05-04 08:51:20 2022-05-04 09:21:11 0:29:51 0:18:33 0:11:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6812285 2022-04-29 03:22:15 2022-05-04 08:51:40 2022-05-04 09:21:41 0:30:01 0:18:45 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-04T09:19:40.478921+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812286 2022-04-29 03:22:16 2022-05-04 08:52:10 2022-05-04 09:16:38 0:24:28 0:11:56 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6812287 2022-04-29 03:22:17 2022-05-04 08:53:01 2022-05-04 09:42:32 0:49:31 0:38:35 0:10:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6812288 2022-04-29 03:22:18 2022-05-04 08:53:01 2022-05-04 09:18:58 0:25:57 0:14:20 0:11:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6812289 2022-04-29 03:22:19 2022-05-04 08:53:22 2022-05-04 09:13:39 0:20:17 0:09:20 0:10:57 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_parent_overlap} 1
pass 6812290 2022-04-29 03:22:20 2022-05-04 08:54:22 2022-05-04 09:25:27 0:31:05 0:17:28 0:13:37 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6812291 2022-04-29 03:22:21 2022-05-04 08:55:32 2022-05-04 09:17:35 0:22:03 0:10:51 0:11:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6812292 2022-04-29 03:22:22 2022-05-04 08:55:33 2022-05-04 10:05:01 1:09:28 0:56:47 0:12:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6812293 2022-04-29 03:22:23 2022-05-04 08:57:23 2022-05-04 09:51:12 0:53:49 0:41:27 0:12:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6812294 2022-04-29 03:22:24 2022-05-04 08:57:24 2022-05-04 09:23:08 0:25:44 0:14:46 0:10:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6812295 2022-04-29 03:22:25 2022-05-04 08:57:34 2022-05-04 09:32:47 0:35:13 0:18:35 0:16:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-04T09:30:14.718795+0000 mon.a (mon.0) 222 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812296 2022-04-29 03:22:26 2022-05-04 09:01:21 2022-05-04 09:24:34 0:23:13 0:11:33 0:11:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6812297 2022-04-29 03:22:27 2022-05-04 09:01:21 2022-05-04 09:22:39 0:21:18 0:08:59 0:12:19 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6812298 2022-04-29 03:22:28 2022-05-04 09:01:31 2022-05-04 09:44:51 0:43:20 0:26:58 0:16:22 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6812299 2022-04-29 03:22:29 2022-05-04 09:03:02 2022-05-04 09:29:12 0:26:10 0:11:23 0:14:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6812300 2022-04-29 03:22:30 2022-05-04 09:03:32 2022-05-04 09:29:12 0:25:40 0:10:32 0:15:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6812301 2022-04-29 03:22:31 2022-05-04 09:03:33 2022-05-04 09:37:54 0:34:21 0:19:14 0:15:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-04T09:36:04.796536+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812302 2022-04-29 03:22:32 2022-05-04 09:04:23 2022-05-04 09:28:56 0:24:33 0:10:43 0:13:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6812303 2022-04-29 03:22:33 2022-05-04 09:05:34 2022-05-04 09:41:19 0:35:45 0:21:39 0:14:06 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6812304 2022-04-29 03:22:34 2022-05-04 09:05:44 2022-05-04 09:30:54 0:25:10 0:10:39 0:14:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
pass 6812305 2022-04-29 03:22:36 2022-05-04 09:06:45 2022-05-04 09:27:25 0:20:40 0:08:54 0:11:46 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6812306 2022-04-29 03:22:37 2022-05-04 09:06:45 2022-05-04 09:33:59 0:27:14 0:13:33 0:13:41 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6812307 2022-04-29 03:22:38 2022-05-04 09:09:56 2022-05-04 09:59:30 0:49:34 0:36:38 0:12:56 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6812308 2022-04-29 03:22:39 2022-05-04 09:12:06 2022-05-04 09:36:27 0:24:21 0:10:18 0:14:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_kernel} 3
pass 6812309 2022-04-29 03:22:40 2022-05-04 09:12:57 2022-05-04 09:43:17 0:30:20 0:15:32 0:14:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6812310 2022-04-29 03:22:41 2022-05-04 09:14:07 2022-05-04 09:48:49 0:34:42 0:19:19 0:15:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-04T09:36:05.886305+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812311 2022-04-29 03:22:42 2022-05-04 09:16:28 2022-05-04 09:42:22 0:25:54 0:14:50 0:11:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6812312 2022-04-29 03:22:43 2022-05-04 09:16:39 2022-05-04 09:41:48 0:25:09 0:12:25 0:12:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6812313 2022-04-29 03:22:44 2022-05-04 09:17:29 2022-05-04 09:49:36 0:32:07 0:19:06 0:13:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-04T09:36:15.542222+0000 mon.a (mon.0) 121 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812314 2022-04-29 03:22:45 2022-05-04 09:17:39 2022-05-04 09:39:09 0:21:30 0:09:34 0:11:56 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_deep_flatten} 1
pass 6812315 2022-04-29 03:22:46 2022-05-04 09:17:40 2022-05-04 09:54:41 0:37:01 0:24:23 0:12:38 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6812316 2022-04-29 03:22:47 2022-05-04 09:19:00 2022-05-04 10:00:52 0:41:52 0:27:29 0:14:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6812317 2022-04-29 03:22:48 2022-05-04 09:21:21 2022-05-04 09:53:14 0:31:53 0:18:46 0:13:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6812318 2022-04-29 03:22:49 2022-05-04 09:21:51 2022-05-04 09:41:59 0:20:08 0:08:28 0:11:40 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6812319 2022-04-29 03:22:50 2022-05-04 09:25:35 2022-05-04 10:00:32 0:34:57 0:20:24 0:14:33 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6812320 2022-04-29 03:22:51 2022-05-04 09:29:06 2022-05-04 09:53:03 0:23:57 0:11:07 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6812321 2022-04-29 03:22:52 2022-05-04 09:29:16 2022-05-04 09:59:05 0:29:49 0:18:14 0:11:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6812322 2022-04-29 03:22:53 2022-05-04 09:29:17 2022-05-04 09:53:01 0:23:44 0:10:33 0:13:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_fallocate} 3
pass 6812323 2022-04-29 03:22:54 2022-05-04 09:30:57 2022-05-04 09:55:58 0:25:01 0:10:54 0:14:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6812324 2022-04-29 03:22:55 2022-05-04 09:32:28 2022-05-04 10:23:15 0:50:47 0:36:23 0:14:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6812325 2022-04-29 03:22:56 2022-05-04 09:32:58 2022-05-04 10:00:05 0:27:07 0:13:01 0:14:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6812326 2022-04-29 03:22:57 2022-05-04 09:34:09 2022-05-04 09:55:58 0:21:49 0:09:03 0:12:46 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6812327 2022-04-29 03:22:58 2022-05-04 09:36:29 2022-05-04 10:03:27 0:26:58 0:14:35 0:12:23 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6812328 2022-04-29 03:22:59 2022-05-04 09:38:00 2022-05-04 10:08:36 0:30:36 0:18:15 0:12:21 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-05-04T09:56:16.730640+0000 mon.a (mon.0) 190 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6812329 2022-04-29 03:23:00 2022-05-04 09:39:10 2022-05-04 10:09:20 0:30:10 0:18:31 0:11:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-04T10:07:36.056107+0000 mon.a (mon.0) 336 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6812330 2022-04-29 03:23:01 2022-05-04 09:39:41 2022-05-04 10:31:03 0:51:22 0:38:10 0:13:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6812331 2022-04-29 03:23:02 2022-05-04 09:41:21 2022-05-04 10:08:11 0:26:50 0:14:27 0:12:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6812332 2022-04-29 03:23:03 2022-05-04 09:41:52 2022-05-04 10:06:18 0:24:26 0:11:18 0:13:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6812333 2022-04-29 03:23:04 2022-05-04 09:42:32 2022-05-04 10:04:03 0:21:31 0:10:36 0:10:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6812334 2022-04-29 03:23:05 2022-05-04 09:42:32 2022-05-04 10:02:02 0:19:30 0:08:14 0:11:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_huge_image} 1
pass 6812335 2022-04-29 03:23:06 2022-05-04 09:42:33 2022-05-04 10:14:54 0:32:21 0:19:53 0:12:28 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6812336 2022-04-29 03:23:07 2022-05-04 09:43:23 2022-05-04 10:58:50 1:15:27 1:03:30 0:11:57 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6812337 2022-04-29 03:23:08 2022-05-04 09:44:54 2022-05-04 10:13:07 0:28:13 0:11:51 0:16:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6812338 2022-04-29 03:23:09 2022-05-04 09:48:55 2022-05-04 10:11:36 0:22:41 0:10:35 0:12:06 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6812339 2022-04-29 03:23:10 2022-05-04 09:49:45 2022-05-04 10:17:23 0:27:38 0:14:50 0:12:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_concurrent} 3
pass 6812340 2022-04-29 03:23:11 2022-05-04 09:51:16 2022-05-04 10:12:51 0:21:35 0:09:54 0:11:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6812341 2022-04-29 03:23:12 2022-05-04 10:00:35 2022-05-04 10:30:26 0:29:51 0:19:03 0:10:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6812342 2022-04-29 03:23:13 2022-05-04 10:00:55 2022-05-04 10:25:19 0:24:24 0:10:20 0:14:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6812343 2022-04-29 03:23:14 2022-05-04 10:47:11 2022-05-04 11:07:33 0:20:22 0:08:44 0:11:38 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_modprobe} 1
pass 6812344 2022-04-29 03:23:15 2022-05-04 10:47:31 2022-05-04 11:16:01 0:28:30 0:17:11 0:11:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6812345 2022-04-29 03:23:16 2022-05-04 10:48:21 2022-05-04 11:18:28 0:30:07 0:18:57 0:11:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-05-04T11:06:01.202959+0000 mon.a (mon.0) 181 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812346 2022-04-29 03:23:17 2022-05-04 10:49:02 2022-05-04 11:17:01 0:27:59 0:17:07 0:10:52 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6812347 2022-04-29 03:23:18 2022-05-04 10:49:42 2022-05-04 11:09:49 0:20:07 0:09:26 0:10:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6812348 2022-04-29 03:23:19 2022-05-04 10:50:13 2022-05-04 11:16:06 0:25:53 0:14:34 0:11:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6812349 2022-04-29 03:23:20 2022-05-04 10:50:43 2022-05-04 11:13:08 0:22:25 0:11:06 0:11:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6812350 2022-04-29 03:23:21 2022-05-04 10:51:24 2022-05-04 11:25:31 0:34:07 0:23:24 0:10:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_simple_big} 3
pass 6812351 2022-04-29 03:23:22 2022-05-04 10:51:34 2022-05-04 11:12:15 0:20:41 0:08:15 0:12:26 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6812352 2022-04-29 03:23:24 2022-05-04 10:52:54 2022-05-04 11:31:07 0:38:13 0:23:56 0:14:17 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6812353 2022-04-29 03:23:25 2022-05-04 10:54:05 2022-05-04 11:37:13 0:43:08 0:27:40 0:15:28 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_fio} 3
fail 6812354 2022-04-29 03:23:26 2022-05-04 10:56:26 2022-05-04 11:26:37 0:30:11 0:18:20 0:11:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-05-04T11:24:46.541600+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812355 2022-04-29 03:23:27 2022-05-04 10:57:06 2022-05-04 11:19:46 0:22:40 0:10:42 0:11:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6812356 2022-04-29 03:23:28 2022-05-04 10:58:17 2022-05-04 11:28:22 0:30:05 0:17:49 0:12:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6812357 2022-04-29 03:23:29 2022-05-04 10:58:27 2022-05-04 11:28:51 0:30:24 0:19:04 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-04T11:27:38.058467+0000 mon.a (mon.0) 335 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812358 2022-04-29 03:23:30 2022-05-04 10:58:57 2022-05-04 11:21:53 0:22:56 0:10:51 0:12:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6812359 2022-04-29 03:23:31 2022-05-04 11:00:28 2022-05-04 11:50:00 0:49:32 0:36:24 0:13:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6812360 2022-04-29 03:23:32 2022-05-04 11:02:18 2022-05-04 11:28:36 0:26:18 0:13:35 0:12:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6812361 2022-04-29 03:23:33 2022-05-04 11:03:29 2022-05-04 11:25:09 0:21:40 0:08:23 0:13:17 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_parent_overlap} 1
pass 6812362 2022-04-29 03:23:34 2022-05-04 11:05:40 2022-05-04 11:31:30 0:25:50 0:15:05 0:10:45 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6812363 2022-04-29 03:23:35 2022-05-04 11:06:10 2022-05-04 12:01:46 0:55:36 0:42:37 0:12:59 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6812364 2022-04-29 03:23:36 2022-05-04 11:07:41 2022-05-04 11:31:37 0:23:56 0:10:54 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6812365 2022-04-29 03:23:37 2022-05-04 11:09:51 2022-05-04 12:18:24 1:08:33 0:56:35 0:11:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6812366 2022-04-29 03:23:38 2022-05-04 11:10:42 2022-05-04 12:05:38 0:54:56 0:43:54 0:11:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6812367 2022-04-29 03:23:39 2022-05-04 11:11:32 2022-05-04 11:38:58 0:27:26 0:15:29 0:11:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6812368 2022-04-29 03:23:40 2022-05-04 11:13:13 2022-05-04 11:42:54 0:29:41 0:18:38 0:11:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-04T11:30:05.013984+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6812369 2022-04-29 03:23:41 2022-05-04 11:13:23 2022-05-04 11:39:39 0:26:16 0:12:42 0:13:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6812370 2022-04-29 03:23:42 2022-05-04 11:16:04 2022-05-04 11:37:20 0:21:16 0:10:26 0:10:50 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_read_only} 1
pass 6812371 2022-04-29 03:23:43 2022-05-04 11:16:04 2022-05-04 11:49:49 0:33:45 0:20:58 0:12:47 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6812372 2022-04-29 03:23:44 2022-05-04 11:16:14 2022-05-04 11:44:44 0:28:30 0:13:29 0:15:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6812373 2022-04-29 03:23:45 2022-05-04 11:16:25 2022-05-04 11:42:40 0:26:15 0:12:30 0:13:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6812374 2022-04-29 03:23:46 2022-05-04 11:17:05 2022-05-04 11:51:49 0:34:44 0:19:27 0:15:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-04T11:50:09.119746+0000 mon.a (mon.0) 328 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812375 2022-04-29 03:23:47 2022-05-04 11:18:36 2022-05-04 11:45:30 0:26:54 0:11:58 0:14:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6812376 2022-04-29 03:23:48 2022-05-04 11:19:56 2022-05-04 11:59:32 0:39:36 0:22:17 0:17:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6812377 2022-04-29 03:23:49 2022-05-04 11:21:57 2022-05-04 12:00:24 0:38:27 0:18:31 0:19:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-05-04T11:57:48.945074+0000 mon.a (mon.0) 232 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6812378 2022-04-29 03:23:50 2022-05-04 11:22:07 2022-05-04 11:51:18 0:29:11 0:08:47 0:20:24 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6812379 2022-04-29 03:23:51 2022-05-04 11:23:48 2022-05-04 12:02:44 0:38:56 0:16:46 0:22:10 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6812380 2022-04-29 03:23:52 2022-05-04 11:25:08 2022-05-04 11:57:39 0:32:31 0:10:27 0:22:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6812381 2022-04-29 03:23:53 2022-05-04 11:25:39 2022-05-04 12:04:24 0:38:45 0:15:55 0:22:50 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6812382 2022-04-29 03:23:54 2022-05-04 11:25:59 2022-05-04 11:56:10 0:30:11 0:09:46 0:20:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6812383 2022-04-29 03:23:55 2022-05-04 11:26:39 2022-05-04 12:02:16 0:35:37 0:14:43 0:20:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6812384 2022-04-29 03:23:56 2022-05-04 12:00:33 2022-05-04 12:26:46 0:26:13 0:13:18 0:12:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6812385 2022-04-29 03:23:57 2022-05-04 12:00:33 2022-05-04 12:40:03 0:39:30 0:26:45 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3