Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6841869 2022-05-20 03:20:14 2022-05-21 06:32:29 2022-05-21 06:56:19 0:23:50 0:12:40 0:11:10 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_deep_flatten} 1
pass 6841870 2022-05-20 03:20:15 2022-05-21 06:32:59 2022-05-21 07:07:50 0:34:51 0:22:27 0:12:24 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6841871 2022-05-20 03:20:16 2022-05-21 06:34:10 2022-05-21 07:00:21 0:26:11 0:13:11 0:13:00 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6841872 2022-05-20 03:20:18 2022-05-21 06:35:00 2022-05-21 07:20:41 0:45:41 0:31:12 0:14:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6841873 2022-05-20 03:20:19 2022-05-21 06:36:21 2022-05-21 07:09:52 0:33:31 0:21:31 0:12:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_data_pool} 3
fail 6841874 2022-05-20 03:20:20 2022-05-21 06:36:22 2022-05-21 11:41:27 5:05:05 4:50:20 0:14:45 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi099 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude3_9x7dvz -r -- '-g auto -g blockdev -x clone'"

pass 6841875 2022-05-20 03:20:21 2022-05-21 06:38:22 2022-05-21 07:32:58 0:54:36 0:41:39 0:12:57 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6841876 2022-05-20 03:20:22 2022-05-21 06:39:13 2022-05-21 06:46:39 0:07:26 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6841877 2022-05-20 03:20:23 2022-05-21 06:39:13 2022-05-21 07:02:36 0:23:23 0:11:41 0:11:42 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
fail 6841878 2022-05-20 03:20:24 2022-05-21 06:39:13 2022-05-21 07:14:25 0:35:12 0:22:35 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-05-21T07:01:44.471543+0000 mon.a (mon.0) 144 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6841879 2022-05-20 03:20:25 2022-05-21 06:41:04 2022-05-21 07:15:43 0:34:39 0:22:29 0:12:10 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-21T07:12:57.014941+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6841880 2022-05-20 03:20:26 2022-05-21 06:42:25 2022-05-21 07:16:07 0:33:42 0:21:55 0:11:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-21T07:04:23.649657+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6841881 2022-05-20 03:20:27 2022-05-21 06:44:25 2022-05-21 07:10:40 0:26:15 0:14:31 0:11:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6841882 2022-05-20 03:20:28 2022-05-21 06:44:26 2022-05-21 07:35:28 0:51:02 0:39:46 0:11:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6841883 2022-05-20 03:20:29 2022-05-21 06:45:26 2022-05-21 07:13:54 0:28:28 0:16:40 0:11:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6841884 2022-05-20 03:20:31 2022-05-21 06:46:07 2022-05-21 07:09:25 0:23:18 0:12:49 0:10:29 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_discard} 1
pass 6841885 2022-05-20 03:20:32 2022-05-21 06:46:07 2022-05-21 07:19:34 0:33:27 0:20:30 0:12:57 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6841886 2022-05-20 03:20:33 2022-05-21 06:47:38 2022-05-21 07:13:10 0:25:32 0:14:33 0:10:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
fail 6841887 2022-05-20 03:20:34 2022-05-21 06:47:48 2022-05-21 07:22:43 0:34:55 0:22:12 0:12:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-21T07:19:36.064308+0000 mon.a (mon.0) 221 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841888 2022-05-20 03:20:35 2022-05-21 06:49:08 2022-05-21 07:44:31 0:55:23 0:41:49 0:13:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6841889 2022-05-20 03:20:36 2022-05-21 06:52:59 2022-05-21 07:22:41 0:29:42 0:17:52 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6841890 2022-05-20 03:20:37 2022-05-21 06:55:30 2022-05-21 07:21:49 0:26:19 0:15:06 0:11:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6841891 2022-05-20 03:20:38 2022-05-21 06:55:50 2022-05-21 07:22:11 0:26:21 0:14:26 0:11:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6841892 2022-05-20 03:20:39 2022-05-21 06:56:41 2022-05-21 07:17:58 0:21:17 0:11:38 0:09:39 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
fail 6841893 2022-05-20 03:20:40 2022-05-21 06:57:01 2022-05-21 07:29:07 0:32:06 0:22:03 0:10:03 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2022-05-21T07:27:56.667822+0000 mon.a (mon.0) 251 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6841894 2022-05-20 03:20:41 2022-05-21 06:57:42 2022-05-21 07:31:47 0:34:05 0:22:02 0:12:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-21T07:20:09.983230+0000 mon.a (mon.0) 149 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6841895 2022-05-20 03:20:42 2022-05-21 07:00:23 2022-05-21 07:35:41 0:35:18 0:22:00 0:13:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-21T07:32:35.107663+0000 mon.a (mon.0) 234 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6841896 2022-05-20 03:20:43 2022-05-21 07:01:53 2022-05-21 07:39:38 0:37:45 0:22:00 0:15:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-21T07:38:19.786892+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841897 2022-05-20 03:20:45 2022-05-21 07:07:54 2022-05-21 07:32:55 0:25:01 0:13:36 0:11:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6841898 2022-05-20 03:20:46 2022-05-21 07:09:25 2022-05-21 07:43:19 0:33:54 0:22:28 0:11:26 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6841899 2022-05-20 03:20:47 2022-05-21 07:09:56 2022-05-21 07:34:50 0:24:54 0:13:52 0:11:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6841900 2022-05-20 03:20:48 2022-05-21 07:10:46 2022-05-21 07:31:57 0:21:11 0:11:36 0:09:35 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_modprobe} 1
pass 6841901 2022-05-20 03:20:49 2022-05-21 07:10:46 2022-05-21 07:40:44 0:29:58 0:18:21 0:11:37 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6841902 2022-05-20 03:20:50 2022-05-21 07:11:07 2022-05-21 08:22:40 1:11:33 0:59:47 0:11:46 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6841903 2022-05-20 03:20:51 2022-05-21 07:13:17 2022-05-21 07:37:40 0:24:23 0:13:26 0:10:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
pass 6841904 2022-05-20 03:20:52 2022-05-21 07:13:58 2022-05-21 07:45:32 0:31:34 0:21:46 0:09:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6841905 2022-05-20 03:20:53 2022-05-21 07:14:08 2022-05-21 07:37:44 0:23:36 0:13:12 0:10:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6841906 2022-05-20 03:20:54 2022-05-21 07:14:29 2022-05-21 07:49:02 0:34:33 0:22:04 0:12:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-05-21T07:46:08.767374+0000 mon.a (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841907 2022-05-20 03:20:55 2022-05-21 07:15:49 2022-05-21 07:41:38 0:25:49 0:14:58 0:10:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6841908 2022-05-20 03:20:56 2022-05-21 07:16:10 2022-05-21 07:53:18 0:37:08 0:25:43 0:11:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
pass 6841909 2022-05-20 03:20:57 2022-05-21 07:18:00 2022-05-21 07:39:13 0:21:13 0:11:42 0:09:31 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_msgr_segments} 1
pass 6841910 2022-05-20 03:20:59 2022-05-21 07:18:01 2022-05-21 07:57:42 0:39:41 0:26:45 0:12:56 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6841911 2022-05-20 03:21:00 2022-05-21 07:19:41 2022-05-21 08:02:52 0:43:11 0:30:16 0:12:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6841912 2022-05-20 03:21:01 2022-05-21 07:21:57 2022-05-21 07:59:50 0:37:53 0:26:08 0:11:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6841913 2022-05-20 03:21:02 2022-05-21 07:22:18 2022-05-21 07:48:16 0:25:58 0:14:19 0:11:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6841914 2022-05-20 03:21:03 2022-05-21 07:22:48 2022-05-21 07:54:06 0:31:18 0:21:30 0:09:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6841915 2022-05-20 03:21:04 2022-05-21 07:22:49 2022-05-21 07:52:58 0:30:09 0:14:22 0:15:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
pass 6841916 2022-05-20 03:21:05 2022-05-21 07:27:00 2022-05-21 07:52:23 0:25:23 0:14:08 0:11:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6841917 2022-05-20 03:21:06 2022-05-21 07:29:10 2022-05-21 08:05:06 0:35:56 0:22:09 0:13:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-21T08:02:54.412360+0000 mon.a (mon.0) 291 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6841918 2022-05-20 03:21:07 2022-05-21 07:31:51 2022-05-21 08:07:11 0:35:20 0:22:31 0:12:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-21T08:05:40.621072+0000 mon.a (mon.0) 280 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841919 2022-05-20 03:21:08 2022-05-21 07:32:32 2022-05-21 07:53:25 0:20:53 0:11:55 0:08:58 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6841920 2022-05-20 03:21:09 2022-05-21 07:32:32 2022-05-21 08:04:50 0:32:18 0:20:31 0:11:47 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6841921 2022-05-20 03:21:10 2022-05-21 07:33:02 2022-05-21 07:58:37 0:25:35 0:14:07 0:11:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_namespaces} 3
pass 6841922 2022-05-20 03:21:11 2022-05-21 07:33:03 2022-05-21 08:43:46 1:10:43 0:58:04 0:12:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6841923 2022-05-20 03:21:12 2022-05-21 07:34:03 2022-05-21 08:09:17 0:35:14 0:22:20 0:12:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-05-21T07:56:38.658536+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6841924 2022-05-20 03:21:14 2022-05-21 07:34:54 2022-05-21 08:05:35 0:30:41 0:18:13 0:12:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6841925 2022-05-20 03:21:15 2022-05-21 07:35:34 2022-05-21 08:03:26 0:27:52 0:16:03 0:11:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6841926 2022-05-20 03:21:16 2022-05-21 07:35:45 2022-05-21 08:11:11 0:35:26 0:22:07 0:13:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-21T07:58:40.642228+0000 mon.a (mon.0) 122 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6841927 2022-05-20 03:21:17 2022-05-21 07:37:45 2022-05-21 07:59:13 0:21:28 0:12:13 0:09:15 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_read_only} 1
pass 6841928 2022-05-20 03:21:18 2022-05-21 07:37:46 2022-05-21 08:09:45 0:31:59 0:21:23 0:10:36 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6841929 2022-05-20 03:21:19 2022-05-21 07:38:16 2022-05-21 08:34:51 0:56:35 0:44:17 0:12:18 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6841930 2022-05-20 03:21:20 2022-05-21 07:39:07 2022-05-21 08:04:56 0:25:49 0:14:44 0:11:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6841931 2022-05-20 03:21:21 2022-05-21 07:39:27 2022-05-21 08:05:18 0:25:51 0:14:14 0:11:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6841932 2022-05-20 03:21:22 2022-05-21 07:39:47 2022-05-21 08:14:00 0:34:13 0:22:25 0:11:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-21T08:11:14.774576+0000 mon.a (mon.0) 269 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6841933 2022-05-20 03:21:23 2022-05-21 07:39:58 2022-05-21 08:04:30 0:24:32 0:13:40 0:10:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6841934 2022-05-20 03:21:24 2022-05-21 07:40:48 2022-05-21 08:15:24 0:34:36 0:22:16 0:12:20 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-21T08:12:20.773511+0000 mon.a (mon.0) 223 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841935 2022-05-20 03:21:25 2022-05-21 07:41:39 2022-05-21 08:08:59 0:27:20 0:13:50 0:13:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6841936 2022-05-20 03:21:26 2022-05-21 07:43:29 2022-05-21 08:06:54 0:23:25 0:12:07 0:11:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_whole_object_zeroout} 1
pass 6841937 2022-05-20 03:21:27 2022-05-21 07:43:30 2022-05-21 08:18:07 0:34:37 0:22:51 0:11:46 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6841938 2022-05-20 03:21:28 2022-05-21 07:44:40 2022-05-21 08:09:05 0:24:25 0:13:05 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6841939 2022-05-20 03:21:29 2022-05-21 07:45:41 2022-05-21 08:18:00 0:32:19 0:18:56 0:13:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6841940 2022-05-20 03:21:30 2022-05-21 07:48:22 2022-05-21 08:13:40 0:25:18 0:13:18 0:12:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6841941 2022-05-20 03:21:31 2022-05-21 07:49:12 2022-05-21 08:21:05 0:31:53 0:18:28 0:13:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
fail 6841942 2022-05-20 03:21:32 2022-05-21 07:51:23 2022-05-21 08:25:46 0:34:23 0:22:34 0:11:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2022-05-21T08:12:54.700316+0000 mon.a (mon.0) 184 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6841943 2022-05-20 03:21:33 2022-05-21 07:52:24 2022-05-21 08:27:02 0:34:38 0:22:19 0:12:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-21T08:24:22.328935+0000 mon.a (mon.0) 283 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841944 2022-05-20 03:21:34 2022-05-21 07:53:04 2022-05-21 08:16:15 0:23:11 0:12:50 0:10:21 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6841945 2022-05-20 03:21:35 2022-05-21 07:53:04 2022-05-21 08:28:40 0:35:36 0:24:31 0:11:05 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6841946 2022-05-20 03:21:36 2022-05-21 07:53:25 2022-05-21 08:36:55 0:43:30 0:31:26 0:12:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6841947 2022-05-20 03:21:37 2022-05-21 07:53:35 2022-05-21 08:25:31 0:31:56 0:21:59 0:09:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_data_pool} 3
pass 6841948 2022-05-20 03:21:39 2022-05-21 07:54:16 2022-05-21 08:18:03 0:23:47 0:12:07 0:11:40 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
fail 6841949 2022-05-20 03:21:40 2022-05-21 07:54:26 2022-05-21 08:30:29 0:36:03 0:22:27 0:13:36 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
Failure Reason:

"2022-05-21T08:27:59.460851+0000 mon.a (mon.0) 239 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841950 2022-05-20 03:21:41 2022-05-21 07:57:07 2022-05-21 08:23:48 0:26:41 0:15:31 0:11:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6841951 2022-05-20 03:21:42 2022-05-21 07:57:47 2022-05-21 08:32:08 0:34:21 0:22:14 0:12:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6841952 2022-05-20 03:21:43 2022-05-21 07:58:38 2022-05-21 08:25:32 0:26:54 0:15:03 0:11:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
fail 6841953 2022-05-20 03:21:44 2022-05-21 07:59:59 2022-05-21 08:33:49 0:33:50 0:22:27 0:11:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-05-21T08:31:19.723135+0000 mon.a (mon.0) 262 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841954 2022-05-20 03:21:45 2022-05-21 08:00:19 2022-05-21 08:55:22 0:55:03 0:40:48 0:14:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6841955 2022-05-20 03:21:46 2022-05-21 08:34:58 2022-05-21 09:03:12 0:28:14 0:16:41 0:11:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6841956 2022-05-20 03:21:47 2022-05-21 08:35:38 2022-05-21 08:59:14 0:23:36 0:12:51 0:10:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_discard} 1
pass 6841957 2022-05-20 03:21:48 2022-05-21 08:35:39 2022-05-21 09:08:24 0:32:45 0:20:48 0:11:57 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6841958 2022-05-20 03:21:49 2022-05-21 08:36:59 2022-05-21 09:51:07 1:14:08 1:01:26 0:12:42 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6841959 2022-05-20 03:21:50 2022-05-21 08:37:00 2022-05-21 09:04:29 0:27:29 0:14:43 0:12:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
fail 6841960 2022-05-20 03:21:51 2022-05-21 08:38:51 2022-05-21 09:13:53 0:35:02 0:22:18 0:12:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-21T09:12:28.467004+0000 mon.a (mon.0) 279 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841961 2022-05-20 03:21:52 2022-05-21 08:39:51 2022-05-21 09:34:34 0:54:43 0:41:45 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6841962 2022-05-20 03:21:53 2022-05-21 08:40:52 2022-05-21 09:16:09 0:35:17 0:22:02 0:13:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-05-21T09:13:45.767024+0000 mon.a (mon.0) 279 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6841963 2022-05-20 03:21:54 2022-05-21 08:42:42 2022-05-21 09:09:37 0:26:55 0:14:39 0:12:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6841964 2022-05-20 03:21:55 2022-05-21 08:43:53 2022-05-21 09:10:32 0:26:39 0:13:58 0:12:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6841965 2022-05-20 03:21:56 2022-05-21 08:44:54 2022-05-21 09:07:16 0:22:22 0:11:43 0:10:39 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_huge_image} 1
pass 6841966 2022-05-20 03:21:57 2022-05-21 08:45:54 2022-05-21 09:21:59 0:36:05 0:25:21 0:10:44 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6841967 2022-05-20 03:21:58 2022-05-21 08:46:04 2022-05-21 09:12:00 0:25:56 0:15:03 0:10:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6841968 2022-05-20 03:21:59 2022-05-21 08:46:15 2022-05-21 09:13:08 0:26:53 0:14:10 0:12:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6841969 2022-05-20 03:22:00 2022-05-21 08:47:25 2022-05-21 09:17:45 0:30:20 0:18:39 0:11:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
pass 6841970 2022-05-20 03:22:01 2022-05-21 08:47:46 2022-05-21 09:11:33 0:23:47 0:13:52 0:09:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6841971 2022-05-20 03:22:02 2022-05-21 08:48:16 2022-05-21 09:22:44 0:34:28 0:21:55 0:12:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6841972 2022-05-20 03:22:03 2022-05-21 08:48:37 2022-05-21 09:14:33 0:25:56 0:14:19 0:11:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_image_read} 3
pass 6841973 2022-05-20 03:22:04 2022-05-21 08:49:07 2022-05-21 09:13:04 0:23:57 0:11:50 0:12:07 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
fail 6841974 2022-05-20 03:22:05 2022-05-21 08:49:48 2022-05-21 09:24:56 0:35:08 0:22:21 0:12:47 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2022-05-21T09:11:58.518981+0000 mon.a (mon.0) 125 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6841975 2022-05-20 03:22:06 2022-05-21 08:50:58 2022-05-21 09:16:40 0:25:42 0:13:49 0:11:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
pass 6841976 2022-05-20 03:22:07 2022-05-21 08:51:09 2022-05-21 09:23:17 0:32:08 0:20:25 0:11:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6841977 2022-05-20 03:22:09 2022-05-21 08:51:39 2022-05-21 09:16:22 0:24:43 0:13:05 0:11:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6841978 2022-05-20 03:22:10 2022-05-21 09:24:20 1100 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6841979 2022-05-20 03:22:11 2022-05-21 08:54:40 2022-05-21 09:21:49 0:27:09 0:14:23 0:12:46 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6841980 2022-05-20 03:22:12 2022-05-21 08:55:31 2022-05-21 09:36:43 0:41:12 0:27:48 0:13:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6841981 2022-05-20 03:22:13 2022-05-21 08:56:51 2022-05-21 09:18:21 0:21:30 0:12:01 0:09:29 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_msgr_segments} 1
pass 6841982 2022-05-20 03:22:14 2022-05-21 08:56:52 2022-05-21 09:34:20 0:37:28 0:25:25 0:12:03 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6841983 2022-05-20 03:22:15 2022-05-21 08:59:02 2022-05-21 09:25:58 0:26:56 0:14:29 0:12:27 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
pass 6841984 2022-05-20 03:22:16 2022-05-21 09:00:13 2022-05-21 09:42:24 0:42:11 0:30:58 0:11:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_fio} 3
pass 6841985 2022-05-20 03:22:17 2022-05-21 09:01:03 2022-05-21 09:38:51 0:37:48 0:25:53 0:11:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_data_pool} 3
fail 6841986 2022-05-20 03:22:18 2022-05-21 09:03:14 2022-05-21 09:37:44 0:34:30 0:21:40 0:12:50 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc-rxbounce} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

"2022-05-21T09:24:47.740924+0000 mon.a (mon.0) 102 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6841987 2022-05-20 03:22:19 2022-05-21 09:03:15 2022-05-21 10:22:12 1:18:57 1:07:58 0:10:59 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6841988 2022-05-20 03:22:20 2022-05-21 09:04:35 2022-05-21 09:31:22 0:26:47 0:14:25 0:12:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6841989 2022-05-20 03:22:21 2022-05-21 09:05:56 2022-05-21 09:39:43 0:33:47 0:21:37 0:12:10 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6841990 2022-05-20 03:22:22 2022-05-21 09:08:27 2022-05-21 09:35:28 0:27:01 0:14:44 0:12:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
pass 6841991 2022-05-20 03:22:24 2022-05-21 09:09:47 2022-05-21 09:36:17 0:26:30 0:14:55 0:11:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6841992 2022-05-20 03:22:25 2022-05-21 09:10:38 2022-05-21 09:46:44 0:36:06 0:22:35 0:13:31 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-21T09:33:41.653684+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6841993 2022-05-20 03:22:26 2022-05-21 09:11:08 2022-05-21 09:41:05 0:29:57 0:17:54 0:12:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6841994 2022-05-20 03:22:27 2022-05-21 09:11:29 2022-05-21 09:34:54 0:23:25 0:11:40 0:11:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6841995 2022-05-20 03:22:28 2022-05-21 09:11:39 2022-05-21 09:47:01 0:35:22 0:23:16 0:12:06 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6841996 2022-05-20 03:22:29 2022-05-21 09:11:39 2022-05-21 09:37:51 0:26:12 0:14:24 0:11:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6841997 2022-05-20 03:22:30 2022-05-21 09:12:10 2022-05-21 10:22:38 1:10:28 0:59:14 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6841998 2022-05-20 03:22:31 2022-05-21 09:13:10 2022-05-21 10:13:51 1:00:41 0:48:22 0:12:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6841999 2022-05-20 03:22:32 2022-05-21 09:14:01 2022-05-21 09:41:59 0:27:58 0:18:01 0:09:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6842000 2022-05-20 03:22:33 2022-05-21 09:14:41 2022-05-21 09:41:29 0:26:48 0:15:34 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6842001 2022-05-20 03:22:34 2022-05-21 09:16:12 2022-05-21 09:48:38 0:32:26 0:21:54 0:10:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-21T09:37:18.363744+0000 mon.a (mon.0) 131 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6842002 2022-05-20 03:22:36 2022-05-21 09:16:32 2022-05-21 09:37:49 0:21:17 0:12:30 0:08:47 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6842003 2022-05-20 03:22:37 2022-05-21 09:16:33 2022-05-21 09:56:10 0:39:37 0:29:21 0:10:16 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6842004 2022-05-20 03:22:38 2022-05-21 09:16:43 2022-05-21 09:43:51 0:27:08 0:14:42 0:12:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6842005 2022-05-20 03:22:39 2022-05-21 09:17:53 2022-05-21 09:48:10 0:30:17 0:13:31 0:16:46 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6842006 2022-05-20 03:22:40 2022-05-21 09:21:54 2022-05-21 09:52:05 0:30:11 0:18:36 0:11:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
pass 6842007 2022-05-20 03:22:41 2022-05-21 09:22:05 2022-05-21 09:48:57 0:26:52 0:13:52 0:13:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6842008 2022-05-20 03:22:42 2022-05-21 09:22:45 2022-05-21 09:59:02 0:36:17 0:23:54 0:12:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6842009 2022-05-20 03:22:43 2022-05-21 09:23:26 2022-05-21 09:50:08 0:26:42 0:13:57 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
pass 6842010 2022-05-20 03:22:44 2022-05-21 09:24:26 2022-05-21 09:47:56 0:23:30 0:12:26 0:11:04 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_whole_object_zeroout} 1
pass 6842011 2022-05-20 03:22:45 2022-05-21 09:24:27 2022-05-21 09:52:46 0:28:19 0:17:19 0:11:00 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6842012 2022-05-20 03:22:47 2022-05-21 09:24:57 2022-05-21 10:15:52 0:50:55 0:40:13 0:10:42 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6842013 2022-05-20 03:22:48 2022-05-21 09:26:08 2022-05-21 09:54:52 0:28:44 0:13:11 0:15:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6842014 2022-05-20 03:22:49 2022-05-21 09:31:29 2022-05-21 10:03:47 0:32:18 0:19:00 0:13:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6842015 2022-05-20 03:22:50 2022-05-21 09:34:30 2022-05-21 09:58:29 0:23:59 0:13:20 0:10:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6842016 2022-05-20 03:22:51 2022-05-21 09:34:40 2022-05-21 10:03:15 0:28:35 0:18:19 0:10:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_unmap} 3
fail 6842017 2022-05-20 03:22:52 2022-05-21 09:35:31 2022-05-21 10:10:03 0:34:32 0:22:17 0:12:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2022-05-21T09:56:37.491583+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6842018 2022-05-20 03:22:53 2022-05-21 09:36:21 2022-05-21 10:14:38 0:38:17 0:26:44 0:11:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6842019 2022-05-20 03:22:54 2022-05-21 09:36:52 2022-05-21 09:58:13 0:21:21 0:12:21 0:09:00 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_deep_flatten} 1
pass 6842020 2022-05-20 03:22:55 2022-05-21 09:36:52 2022-05-21 10:15:42 0:38:50 0:27:22 0:11:28 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6842021 2022-05-20 03:22:56 2022-05-21 09:37:53 2022-05-21 10:19:18 0:41:25 0:31:00 0:10:25 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6842022 2022-05-20 03:22:57 2022-05-21 09:37:53 2022-05-21 10:12:25 0:34:32 0:22:43 0:11:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6842023 2022-05-20 03:22:58 2022-05-21 09:37:53 2022-05-21 10:02:23 0:24:30 0:12:16 0:12:14 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6842024 2022-05-20 03:22:59 2022-05-21 09:38:54 2022-05-21 10:13:06 0:34:12 0:24:15 0:09:57 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6842025 2022-05-20 03:23:00 2022-05-21 09:39:44 2022-05-21 10:06:50 0:27:06 0:14:34 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6842026 2022-05-20 03:23:02 2022-05-21 09:41:15 2022-05-21 10:13:05 0:31:50 0:21:56 0:09:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6842027 2022-05-20 03:23:03 2022-05-21 09:41:35 2022-05-21 10:05:32 0:23:57 0:14:02 0:09:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
pass 6842028 2022-05-20 03:23:04 2022-05-21 09:42:06 2022-05-21 10:07:55 0:25:49 0:14:27 0:11:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6842029 2022-05-20 03:23:05 2022-05-21 09:42:26 2022-05-21 10:33:55 0:51:29 0:40:02 0:11:27 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6842030 2022-05-20 03:23:06 2022-05-21 09:43:57 2022-05-21 10:14:08 0:30:11 0:17:18 0:12:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6842031 2022-05-20 03:23:07 2022-05-21 09:46:48 2022-05-21 10:08:20 0:21:32 0:12:35 0:08:57 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6842032 2022-05-20 03:23:08 2022-05-21 09:46:48 2022-05-21 10:16:32 0:29:44 0:19:12 0:10:32 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6842033 2022-05-20 03:23:09 2022-05-21 09:47:08 2022-05-21 10:14:31 0:27:23 0:14:56 0:12:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_namespaces} 3
pass 6842034 2022-05-20 03:23:10 2022-05-21 09:48:19 2022-05-21 11:11:51 1:23:32 1:13:33 0:09:59 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6842035 2022-05-20 03:23:11 2022-05-21 09:48:19 2022-05-21 10:42:46 0:54:27 0:42:07 0:12:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6842036 2022-05-20 03:23:12 2022-05-21 09:48:40 2022-05-21 10:19:13 0:30:33 0:18:03 0:12:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6842037 2022-05-20 03:23:14 2022-05-21 09:49:00 2022-05-21 10:15:47 0:26:47 0:14:45 0:12:02 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6842038 2022-05-20 03:23:15 2022-05-21 09:50:11 2022-05-21 10:17:20 0:27:09 0:13:57 0:13:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6842039 2022-05-20 03:23:16 2022-05-21 09:51:11 2022-05-21 10:13:40 0:22:29 0:11:54 0:10:35 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_huge_image} 1
pass 6842040 2022-05-20 03:23:17 2022-05-21 09:52:12 2022-05-21 10:28:50 0:36:38 0:25:19 0:11:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6842041 2022-05-20 03:23:18 2022-05-21 09:52:52 2022-05-21 11:10:18 1:17:26 1:05:07 0:12:19 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6842042 2022-05-20 03:23:19 2022-05-21 09:54:53 2022-05-21 10:21:40 0:26:47 0:15:05 0:11:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6842043 2022-05-20 03:23:20 2022-05-21 09:56:14 2022-05-21 10:31:03 0:34:49 0:22:18 0:12:31 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-21T10:27:58.211955+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6842044 2022-05-20 03:23:21 2022-05-21 09:57:24 2022-05-21 10:28:01 0:30:37 0:18:19 0:12:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_concurrent} 3
fail 6842045 2022-05-20 03:23:22 2022-05-21 09:58:05 2022-05-21 10:33:11 0:35:06 0:22:12 0:12:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-21T10:30:09.610132+0000 mon.a (mon.0) 262 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6842046 2022-05-20 03:23:24 2022-05-21 09:58:35 2022-05-21 10:30:39 0:32:04 0:22:17 0:09:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6842047 2022-05-20 03:23:25 2022-05-21 09:59:06 2022-05-21 10:24:19 0:25:13 0:13:43 0:11:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6842048 2022-05-20 03:23:26 2022-05-21 10:00:16 2022-05-21 10:23:23 0:23:07 0:12:00 0:11:07 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_modprobe} 1
pass 6842049 2022-05-20 03:23:27 2022-05-21 10:02:07 2022-05-21 10:33:54 0:31:47 0:21:48 0:09:59 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6842050 2022-05-20 03:23:28 2022-05-21 10:02:27 2022-05-21 10:26:55 0:24:28 0:13:20 0:11:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
pass 6842051 2022-05-20 03:23:29 2022-05-21 10:03:18 2022-05-21 10:33:13 0:29:55 0:20:04 0:09:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6842052 2022-05-20 03:23:30 2022-05-21 10:03:48 2022-05-21 10:29:05 0:25:17 0:13:28 0:11:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6842053 2022-05-20 03:23:31 2022-05-21 10:05:39 2022-05-21 10:40:34 0:34:55 0:22:20 0:12:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-05-21T10:27:05.368494+0000 mon.a (mon.0) 170 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6842054 2022-05-20 03:23:33 2022-05-21 10:06:59 2022-05-21 10:31:29 0:24:30 0:14:23 0:10:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6842055 2022-05-20 03:23:34 2022-05-21 10:08:00 2022-05-21 10:46:02 0:38:02 0:26:41 0:11:21 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6842056 2022-05-20 03:23:35 2022-05-21 10:08:30 2022-05-21 10:30:45 0:22:15 0:11:59 0:10:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6842057 2022-05-20 03:23:36 2022-05-21 10:09:11 2022-05-21 10:48:03 0:38:52 0:26:44 0:12:08 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6842058 2022-05-20 03:23:37 2022-05-21 10:10:11 2022-05-21 10:49:18 0:39:07 0:22:24 0:16:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-05-21T10:46:36.250531+0000 mon.a (mon.0) 262 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6842059 2022-05-20 03:23:38 2022-05-21 10:12:32 2022-05-21 10:50:38 0:38:06 0:26:02 0:12:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
pass 6842060 2022-05-20 03:23:39 2022-05-21 10:13:13 2022-05-21 10:38:34 0:25:21 0:14:19 0:11:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6842061 2022-05-20 03:23:40 2022-05-21 10:13:13 2022-05-21 10:47:57 0:34:44 0:21:46 0:12:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6842062 2022-05-20 03:23:41 2022-05-21 10:13:54 2022-05-21 10:41:46 0:27:52 0:14:51 0:13:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_fallocate} 3
pass 6842063 2022-05-20 03:23:42 2022-05-21 10:14:14 2022-05-21 10:42:52 0:28:38 0:15:01 0:13:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6842064 2022-05-20 03:23:43 2022-05-21 10:14:35 2022-05-21 11:08:35 0:54:00 0:41:05 0:12:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6842065 2022-05-20 03:23:44 2022-05-21 10:14:45 2022-05-21 10:49:33 0:34:48 0:20:33 0:14:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6842066 2022-05-20 03:23:46 2022-05-21 10:15:45 2022-05-21 10:39:13 0:23:28 0:12:15 0:11:13 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_parent_overlap} 1
pass 6842067 2022-05-20 03:23:47 2022-05-21 10:15:46 2022-05-21 10:47:43 0:31:57 0:20:04 0:11:53 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6842068 2022-05-20 03:23:48 2022-05-21 10:15:56 2022-05-21 11:13:39 0:57:43 0:45:21 0:12:22 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6842069 2022-05-20 03:23:49 2022-05-21 10:15:57 2022-05-21 10:44:07 0:28:10 0:15:21 0:12:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6842070 2022-05-20 03:23:50 2022-05-21 10:16:27 2022-05-21 11:27:57 1:11:30 1:00:01 0:11:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6842071 2022-05-20 03:23:51 2022-05-21 10:16:37 2022-05-21 11:15:39 0:59:02 0:47:20 0:11:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6842072 2022-05-20 03:23:52 2022-05-21 10:17:28 2022-05-21 10:49:30 0:32:02 0:18:17 0:13:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6842073 2022-05-20 03:23:53 2022-05-21 10:19:19 2022-05-21 10:52:41 0:33:22 0:22:26 0:10:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-21T10:39:59.990753+0000 mon.a (mon.0) 201 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6842074 2022-05-20 03:23:55 2022-05-21 10:19:19 2022-05-21 10:47:21 0:28:02 0:14:46 0:13:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6842075 2022-05-20 03:23:56 2022-05-21 10:21:50 2022-05-21 10:45:14 0:23:24 0:12:39 0:10:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_read_only} 1
pass 6842076 2022-05-20 03:23:57 2022-05-21 10:21:50 2022-05-21 10:57:52 0:36:02 0:24:13 0:11:49 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6842077 2022-05-20 03:23:58 2022-05-21 10:22:21 2022-05-21 10:47:32 0:25:11 0:15:09 0:10:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6842078 2022-05-20 03:23:59 2022-05-21 10:22:31 2022-05-21 10:48:05 0:25:34 0:13:32 0:12:02 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6842079 2022-05-20 03:24:00 2022-05-21 10:22:41 2022-05-21 10:54:27 0:31:46 0:18:38 0:13:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
fail 6842080 2022-05-20 03:24:01 2022-05-21 10:24:22 2022-05-21 11:00:49 0:36:27 0:22:09 0:14:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-21T10:47:41.786181+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6842081 2022-05-20 03:24:02 2022-05-21 10:26:13 2022-05-21 11:00:45 0:34:32 0:23:40 0:10:52 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6842082 2022-05-20 03:24:03 2022-05-21 10:27:03 2022-05-21 10:51:59 0:24:56 0:13:56 0:11:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_image_read} 3
pass 6842083 2022-05-20 03:24:05 2022-05-21 10:28:04 2022-05-21 10:49:41 0:21:37 0:12:19 0:09:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6842084 2022-05-20 03:24:06 2022-05-21 10:28:14 2022-05-21 11:00:54 0:32:40 0:20:35 0:12:05 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6842085 2022-05-20 03:24:07 2022-05-21 10:28:55 2022-05-21 10:52:38 0:23:43 0:13:18 0:10:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6842086 2022-05-20 03:24:08 2022-05-21 10:29:15 2022-05-21 11:00:20 0:31:05 0:19:17 0:11:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6842087 2022-05-20 03:24:09 2022-05-21 10:30:46 2022-05-21 11:04:38 0:33:52 0:22:26 0:11:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-21T11:02:01.194485+0000 mon.a (mon.0) 234 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6842088 2022-05-20 03:24:10 2022-05-21 10:31:07 2022-05-21 10:59:12 0:28:05 0:18:16 0:09:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6842089 2022-05-20 03:24:11 2022-05-21 10:31:37 2022-05-21 10:58:40 0:27:03 0:15:02 0:12:01 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6842090 2022-05-20 03:24:12 2022-05-21 10:33:18 2022-05-21 11:06:56 0:33:38 0:21:59 0:11:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2022-05-21T11:04:07.588085+0000 mon.a (mon.0) 277 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log