Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6730868 2022-03-11 03:20:49 2022-03-11 03:21:32 2022-03-11 03:42:54 0:21:22 0:11:53 0:09:29 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_deep_flatten} 1
pass 6730869 2022-03-11 03:20:50 2022-03-11 03:21:32 2022-03-11 04:07:54 0:46:22 0:24:17 0:22:05 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6730870 2022-03-11 03:20:51 2022-03-11 03:21:32 2022-03-11 03:51:48 0:30:16 0:11:27 0:18:49 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6730871 2022-03-11 03:20:51 2022-03-11 03:21:33 2022-03-11 04:15:35 0:54:02 0:32:15 0:21:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6730872 2022-03-11 03:20:52 2022-03-11 03:21:33 2022-03-11 04:07:43 0:46:10 0:23:19 0:22:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
fail 6730873 2022-03-11 03:20:53 2022-03-11 03:21:33 2022-03-11 08:08:06 4:46:33 4:25:16 0:21:17 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi145 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeuassqe0b -r -- '-g auto -g blockdev -x clone'"

pass 6730874 2022-03-11 03:20:54 2022-03-11 03:21:34 2022-03-11 04:26:38 1:05:04 0:42:39 0:22:25 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6730875 2022-03-11 03:20:54 2022-03-11 03:21:34 2022-03-11 03:33:12 0:11:38 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6730876 2022-03-11 03:20:55 2022-03-11 03:21:34 2022-03-11 03:50:28 0:28:54 0:09:29 0:19:25 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
fail 6730877 2022-03-11 03:20:56 2022-03-11 03:21:35 2022-03-11 04:02:03 0:40:28 0:20:13 0:20:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-03-11T03:59:18.634826+0000 mon.a (mon.0) 250 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730878 2022-03-11 03:20:57 2022-03-11 03:21:35 2022-03-11 03:57:26 0:35:51 0:19:57 0:15:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6730879 2022-03-11 03:20:57 2022-03-11 03:21:35 2022-03-11 03:46:37 0:25:02 0:12:01 0:13:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6730880 2022-03-11 03:20:58 2022-03-11 03:21:36 2022-03-11 04:09:37 0:48:01 0:24:08 0:23:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-03-11T03:56:03.442731+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6730881 2022-03-11 03:20:59 2022-03-11 03:21:36 2022-03-11 04:10:28 0:48:52 0:37:38 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6730882 2022-03-11 03:21:00 2022-03-11 03:21:36 2022-03-11 04:01:07 0:39:31 0:17:54 0:21:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6730883 2022-03-11 03:21:00 2022-03-11 03:21:37 2022-03-11 03:55:40 0:34:03 0:10:56 0:23:07 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_discard} 1
pass 6730884 2022-03-11 03:21:01 2022-03-11 03:21:37 2022-03-11 04:05:48 0:44:11 0:20:58 0:23:13 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6730885 2022-03-11 03:21:02 2022-03-11 03:21:37 2022-03-11 03:46:59 0:25:22 0:12:44 0:12:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
fail 6730886 2022-03-11 03:21:03 2022-03-11 03:21:38 2022-03-11 04:02:15 0:40:37 0:20:17 0:20:20 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-03-11T04:00:22.557300+0000 mon.a (mon.0) 331 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730887 2022-03-11 03:21:04 2022-03-11 03:21:38 2022-03-11 04:15:46 0:54:08 0:37:44 0:16:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6730888 2022-03-11 03:21:04 2022-03-11 03:21:38 2022-03-11 04:05:44 0:44:06 0:20:16 0:23:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6730889 2022-03-11 03:21:05 2022-03-11 03:21:39 2022-03-11 03:59:04 0:37:25 0:14:35 0:22:50 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6730890 2022-03-11 03:21:06 2022-03-11 03:21:39 2022-03-11 03:45:10 0:23:31 0:13:14 0:10:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6730891 2022-03-11 03:21:07 2022-03-11 03:21:39 2022-03-11 03:52:01 0:30:22 0:10:04 0:20:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6730892 2022-03-11 03:21:08 2022-03-11 03:21:40 2022-03-11 04:11:53 0:50:13 0:27:15 0:22:58 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6730893 2022-03-11 03:21:08 2022-03-11 03:21:40 2022-03-11 03:59:28 0:37:48 0:19:09 0:18:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-03-11T03:57:21.058791+0000 mon.a (mon.0) 284 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6730894 2022-03-11 03:21:09 2022-03-11 03:21:40 2022-03-11 04:09:22 0:47:42 0:24:18 0:23:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-03-11T04:06:08.760165+0000 mon.a (mon.0) 305 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730895 2022-03-11 03:21:10 2022-03-11 03:21:41 2022-03-11 04:05:23 0:43:42 0:20:08 0:23:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
fail 6730896 2022-03-11 03:21:11 2022-03-11 03:21:41 2022-03-11 04:07:40 0:45:59 0:23:50 0:22:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-03-11T04:06:20.259193+0000 mon.a (mon.0) 251 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730897 2022-03-11 03:21:11 2022-03-11 03:21:41 2022-03-11 04:02:18 0:40:37 0:20:04 0:20:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6730898 2022-03-11 03:21:12 2022-03-11 03:21:42 2022-03-11 03:59:10 0:37:28 0:14:55 0:22:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6730899 2022-03-11 03:21:13 2022-03-11 03:21:42 2022-03-11 03:45:54 0:24:12 0:09:03 0:15:09 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_modprobe} 1
pass 6730900 2022-03-11 03:21:14 2022-03-11 03:21:42 2022-03-11 04:03:57 0:42:15 0:19:05 0:23:10 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6730901 2022-03-11 03:21:14 2022-03-11 03:21:43 2022-03-11 04:58:17 1:36:34 1:16:08 0:20:26 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6730902 2022-03-11 03:21:15 2022-03-11 03:21:43 2022-03-11 03:59:35 0:37:52 0:15:32 0:22:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
fail 6730903 2022-03-11 03:21:16 2022-03-11 03:21:44 2022-03-11 04:08:21 0:46:37 0:23:41 0:22:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-03-11T04:06:40.802510+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730904 2022-03-11 03:21:17 2022-03-11 03:21:44 2022-03-11 03:59:42 0:37:58 0:14:27 0:23:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6730905 2022-03-11 03:21:17 2022-03-11 03:21:44 2022-03-11 03:58:13 0:36:29 0:16:33 0:19:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6730906 2022-03-11 03:21:18 2022-03-11 03:21:45 2022-03-11 04:01:00 0:39:15 0:15:15 0:24:00 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6730907 2022-03-11 03:21:19 2022-03-11 03:29:36 2022-03-11 04:13:08 0:43:32 0:26:15 0:17:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
pass 6730908 2022-03-11 03:21:20 2022-03-11 03:35:37 2022-03-11 03:57:05 0:21:28 0:12:12 0:09:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_msgr_segments} 1
pass 6730909 2022-03-11 03:21:20 2022-03-11 03:35:37 2022-03-11 04:18:24 0:42:47 0:23:20 0:19:27 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6730910 2022-03-11 03:21:21 2022-03-11 03:42:59 2022-03-11 04:24:46 0:41:47 0:28:47 0:13:00 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6730911 2022-03-11 03:21:22 2022-03-11 03:45:20 2022-03-11 04:20:10 0:34:50 0:23:26 0:11:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6730912 2022-03-11 03:21:23 2022-03-11 03:46:41 2022-03-11 04:08:45 0:22:04 0:11:16 0:10:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6730913 2022-03-11 03:21:23 2022-03-11 03:47:01 2022-03-11 04:21:24 0:34:23 0:18:12 0:16:11 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6730914 2022-03-11 03:21:24 2022-03-11 03:51:52 2022-03-11 04:15:45 0:23:53 0:11:30 0:12:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_fallocate} 3
pass 6730915 2022-03-11 03:21:25 2022-03-11 03:52:03 2022-03-11 04:19:27 0:27:24 0:11:44 0:15:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6730916 2022-03-11 03:21:25 2022-03-11 03:57:04 2022-03-11 04:44:57 0:47:53 0:37:41 0:10:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6730917 2022-03-11 03:21:26 2022-03-11 03:57:34 2022-03-11 04:27:54 0:30:20 0:19:44 0:10:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-03-11T04:26:30.421470+0000 mon.a (mon.0) 307 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730918 2022-03-11 03:21:27 2022-03-11 03:58:15 2022-03-11 04:17:47 0:19:32 0:09:11 0:10:21 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6730919 2022-03-11 03:21:28 2022-03-11 03:58:15 2022-03-11 04:29:22 0:31:07 0:18:47 0:12:20 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6730920 2022-03-11 03:21:28 2022-03-11 03:59:05 2022-03-11 04:20:44 0:21:39 0:11:50 0:09:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_namespaces} 3
pass 6730921 2022-03-11 03:21:29 2022-03-11 03:59:16 2022-03-11 05:25:25 1:26:09 1:14:18 0:11:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6730922 2022-03-11 03:21:30 2022-03-11 03:59:36 2022-03-11 04:28:52 0:29:16 0:19:36 0:09:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-03-11T04:27:38.770606+0000 mon.a (mon.0) 221 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730923 2022-03-11 03:21:31 2022-03-11 03:59:36 2022-03-11 04:28:07 0:28:31 0:15:25 0:13:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6730924 2022-03-11 03:21:31 2022-03-11 03:59:47 2022-03-11 04:32:28 0:32:41 0:19:37 0:13:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-03-11T04:19:01.758168+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6730925 2022-03-11 03:21:32 2022-03-11 04:24:51 743 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6730926 2022-03-11 03:21:33 2022-03-11 04:01:08 2022-03-11 04:20:24 0:19:16 0:09:28 0:09:48 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_read_only} 1
pass 6730927 2022-03-11 03:21:33 2022-03-11 04:01:08 2022-03-11 04:32:22 0:31:14 0:19:40 0:11:34 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6730928 2022-03-11 03:21:34 2022-03-11 04:55:48 2534 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6730929 2022-03-11 03:21:35 2022-03-11 04:02:19 2022-03-11 04:25:40 0:23:21 0:11:57 0:11:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6730930 2022-03-11 03:21:36 2022-03-11 04:02:19 2022-03-11 04:25:55 0:23:36 0:10:39 0:12:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6730931 2022-03-11 03:21:37 2022-03-11 04:04:00 2022-03-11 04:33:14 0:29:14 0:16:25 0:12:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
fail 6730932 2022-03-11 03:21:38 2022-03-11 04:37:33 1174 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-03-11T04:35:03.445016+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6730933 2022-03-11 03:21:38 2022-03-11 04:37:22 1190 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-03-11T04:24:43.852076+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6730934 2022-03-11 03:21:39 2022-03-11 04:31:17 679 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6730935 2022-03-11 03:21:40 2022-03-11 04:07:51 2022-03-11 04:29:07 0:21:16 0:10:20 0:10:56 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_whole_object_zeroout} 1
pass 6730936 2022-03-11 03:21:41 2022-03-11 04:37:43 1126 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6730937 2022-03-11 03:21:42 2022-03-11 04:08:02 2022-03-11 04:32:16 0:24:14 0:11:13 0:13:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6730938 2022-03-11 03:21:43 2022-03-11 04:08:22 2022-03-11 04:36:37 0:28:15 0:16:17 0:11:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6730939 2022-03-11 03:21:44 2022-03-11 04:43:07 1211 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-03-11T04:39:52.235851+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730940 2022-03-11 03:21:44 2022-03-11 04:09:23 2022-03-11 04:37:06 0:27:43 0:16:06 0:11:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6730941 2022-03-11 03:21:45 2022-03-11 04:09:44 2022-03-11 04:33:58 0:24:14 0:12:55 0:11:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6730942 2022-03-11 03:21:46 2022-03-11 04:10:34 2022-03-11 04:47:32 0:36:58 0:25:02 0:11:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_simple_big} 3
pass 6730943 2022-03-11 03:21:47 2022-03-11 04:11:55 2022-03-11 04:33:02 0:21:07 0:09:51 0:11:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6730944 2022-03-11 03:21:48 2022-03-11 04:11:55 2022-03-11 04:42:59 0:31:04 0:20:07 0:10:57 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6730945 2022-03-11 03:21:48 2022-03-11 04:13:16 2022-03-11 04:57:02 0:43:46 0:28:34 0:15:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_fio} 3
pass 6730946 2022-03-11 03:21:49 2022-03-11 04:15:36 2022-03-11 04:47:07 0:31:31 0:18:38 0:12:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6730947 2022-03-11 03:21:50 2022-03-11 04:15:47 2022-03-11 04:39:43 0:23:56 0:09:24 0:14:32 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6730948 2022-03-11 03:21:51 2022-03-11 04:15:47 2022-03-11 04:53:37 0:37:50 0:21:01 0:16:49 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
pass 6730949 2022-03-11 03:21:51 2022-03-11 04:17:57 2022-03-11 04:43:47 0:25:50 0:11:53 0:13:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6730950 2022-03-11 03:21:52 2022-03-11 04:18:28 2022-03-11 04:53:51 0:35:23 0:19:50 0:15:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-03-11T04:51:11.434219+0000 mon.a (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730951 2022-03-11 03:21:53 2022-03-11 04:19:28 2022-03-11 04:43:54 0:24:26 0:11:10 0:13:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
pass 6730952 2022-03-11 03:21:53 2022-03-11 04:20:19 2022-03-11 04:44:15 0:23:56 0:11:42 0:12:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6730953 2022-03-11 03:21:54 2022-03-11 04:20:49 2022-03-11 05:11:12 0:50:23 0:37:13 0:13:10 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6730954 2022-03-11 03:21:55 2022-03-11 04:21:30 2022-03-11 04:50:10 0:28:40 0:14:00 0:14:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6730955 2022-03-11 03:21:56 2022-03-11 04:24:50 2022-03-11 04:44:10 0:19:20 0:09:43 0:09:37 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_discard} 1
pass 6730956 2022-03-11 03:21:56 2022-03-11 04:24:51 2022-03-11 04:54:41 0:29:50 0:17:29 0:12:21 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6730957 2022-03-11 03:21:57 2022-03-11 04:25:01 2022-03-11 05:55:16 1:30:15 1:18:57 0:11:18 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6730958 2022-03-11 03:21:58 2022-03-11 04:25:42 2022-03-11 04:50:01 0:24:19 0:11:47 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
pass 6730959 2022-03-11 03:21:58 2022-03-11 04:26:02 2022-03-11 06:00:58 1:34:56 1:22:53 0:12:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6730960 2022-03-11 03:21:59 2022-03-11 04:26:42 2022-03-11 05:17:34 0:50:52 0:37:46 0:13:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6730961 2022-03-11 03:22:00 2022-03-11 04:28:03 2022-03-11 04:54:29 0:26:26 0:15:28 0:10:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6730962 2022-03-11 03:22:01 2022-03-11 04:28:13 2022-03-11 04:52:11 0:23:58 0:12:05 0:11:53 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6730963 2022-03-11 03:22:01 2022-03-11 04:28:54 2022-03-11 04:51:45 0:22:51 0:11:06 0:11:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6730964 2022-03-11 03:22:02 2022-03-11 04:29:24 2022-03-11 04:49:00 0:19:36 0:08:47 0:10:49 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_huge_image} 1
pass 6730965 2022-03-11 03:22:03 2022-03-11 04:29:24 2022-03-11 05:04:51 0:35:27 0:22:32 0:12:55 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6730966 2022-03-11 03:22:04 2022-03-11 04:31:25 2022-03-11 04:56:07 0:24:42 0:12:34 0:12:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6730967 2022-03-11 03:22:04 2022-03-11 04:32:25 2022-03-11 05:04:33 0:32:08 0:19:37 0:12:31 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-03-11T05:01:38.438752+0000 mon.a (mon.0) 272 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730968 2022-03-11 03:22:05 2022-03-11 04:32:26 2022-03-11 04:57:47 0:25:21 0:15:52 0:09:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
pass 6730969 2022-03-11 03:22:06 2022-03-11 04:32:36 2022-03-11 04:54:55 0:22:19 0:10:33 0:11:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6730970 2022-03-11 03:22:06 2022-03-11 04:33:17 2022-03-11 05:03:39 0:30:22 0:18:55 0:11:27 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6730971 2022-03-11 03:22:07 2022-03-11 04:34:07 2022-03-11 04:58:17 0:24:10 0:11:05 0:13:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_image_read} 3
pass 6730972 2022-03-11 03:22:08 2022-03-11 04:36:38 2022-03-11 04:58:21 0:21:43 0:09:19 0:12:24 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6730973 2022-03-11 03:22:09 2022-03-11 04:36:38 2022-03-11 05:08:44 0:32:06 0:20:40 0:11:26 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6730974 2022-03-11 03:22:09 2022-03-11 04:37:09 2022-03-11 05:08:47 0:31:38 0:20:14 0:11:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-03-11T05:06:29.755347+0000 mon.a (mon.0) 246 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730975 2022-03-11 03:22:10 2022-03-11 04:37:29 2022-03-11 05:07:12 0:29:43 0:17:33 0:12:10 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6730976 2022-03-11 03:22:11 2022-03-11 04:37:39 2022-03-11 04:59:24 0:21:45 0:10:34 0:11:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6730977 2022-03-11 03:22:11 2022-03-11 04:37:50 2022-03-11 05:07:43 0:29:53 0:15:23 0:14:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6730978 2022-03-11 03:22:12 2022-03-11 04:39:50 2022-03-11 05:04:43 0:24:53 0:11:52 0:13:01 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
fail 6730979 2022-03-11 03:22:13 2022-03-11 04:43:01 2022-03-11 05:12:55 0:29:54 0:19:22 0:10:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

"2022-03-11T05:01:07.434948+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6730980 2022-03-11 03:22:13 2022-03-11 04:43:11 2022-03-11 05:05:02 0:21:51 0:09:45 0:12:06 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_msgr_segments} 1
pass 6730981 2022-03-11 03:22:14 2022-03-11 04:43:52 2022-03-11 05:17:33 0:33:41 0:21:56 0:11:45 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6730982 2022-03-11 03:22:15 2022-03-11 04:44:02 2022-03-11 05:07:33 0:23:31 0:11:37 0:11:54 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
pass 6730983 2022-03-11 03:22:16 2022-03-11 04:44:13 2022-03-11 05:23:49 0:39:36 0:27:24 0:12:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_fio} 3
pass 6730984 2022-03-11 03:22:16 2022-03-11 04:44:23 2022-03-11 05:20:19 0:35:56 0:23:06 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_data_pool} 3
fail 6730985 2022-03-11 03:22:17 2022-03-11 04:45:03 2022-03-11 09:55:30 5:10:27 4:55:53 0:14:34 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi197 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude2uzl879e -r -- '-g auto -g blockdev -x clone'"

pass 6730986 2022-03-11 03:22:18 2022-03-11 04:47:34 2022-03-11 06:20:45 1:33:11 1:22:15 0:10:56 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6730987 2022-03-11 03:22:18 2022-03-11 04:49:05 2022-03-11 05:14:06 0:25:01 0:11:28 0:13:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6730988 2022-03-11 03:22:19 2022-03-11 04:50:05 2022-03-11 05:17:37 0:27:32 0:18:15 0:09:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6730989 2022-03-11 03:22:20 2022-03-11 04:50:16 2022-03-11 05:23:58 0:33:42 0:19:26 0:14:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-03-11T05:21:08.876184+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730990 2022-03-11 03:22:21 2022-03-11 04:51:46 2022-03-11 05:13:39 0:21:53 0:11:25 0:10:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6730991 2022-03-11 03:22:21 2022-03-11 04:52:17 2022-03-11 05:43:26 0:51:09 0:37:44 0:13:25 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6730992 2022-03-11 03:22:22 2022-03-11 04:53:47 2022-03-11 05:22:11 0:28:24 0:14:25 0:13:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6730993 2022-03-11 03:22:23 2022-03-11 04:53:58 2022-03-11 05:13:36 0:19:38 0:09:14 0:10:24 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6730994 2022-03-11 03:22:23 2022-03-11 04:53:58 2022-03-11 05:26:51 0:32:53 0:19:04 0:13:49 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6730995 2022-03-11 03:22:24 2022-03-11 04:54:38 2022-03-11 05:18:38 0:24:00 0:11:34 0:12:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6730996 2022-03-11 03:22:25 2022-03-11 04:54:49 2022-03-11 06:23:13 1:28:24 1:15:04 0:13:20 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6730997 2022-03-11 03:22:26 2022-03-11 04:54:59 2022-03-11 05:27:23 0:32:24 0:19:34 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-03-11T05:26:02.342178+0000 mon.a (mon.0) 261 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6730998 2022-03-11 03:22:26 2022-03-11 04:55:49 2022-03-11 05:23:34 0:27:45 0:15:21 0:12:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6730999 2022-03-11 03:22:27 2022-03-11 04:56:10 2022-03-11 05:22:37 0:26:27 0:13:24 0:13:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6731000 2022-03-11 03:22:28 2022-03-11 04:57:10 2022-03-11 05:29:16 0:32:06 0:20:07 0:11:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-03-11T05:27:15.366073+0000 mon.a (mon.0) 318 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731001 2022-03-11 03:22:28 2022-03-11 04:57:51 2022-03-11 05:20:06 0:22:15 0:09:54 0:12:21 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6731002 2022-03-11 03:22:29 2022-03-11 04:58:21 2022-03-11 05:35:49 0:37:28 0:26:35 0:10:53 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6731003 2022-03-11 03:22:30 2022-03-11 04:58:22 2022-03-11 05:22:02 0:23:40 0:11:54 0:11:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6731004 2022-03-11 03:22:31 2022-03-11 04:58:22 2022-03-11 05:21:10 0:22:48 0:11:06 0:11:42 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6731005 2022-03-11 03:22:31 2022-03-11 04:59:33 2022-03-11 05:33:15 0:33:42 0:19:48 0:13:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-03-11T05:31:48.295002+0000 mon.a (mon.0) 285 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731006 2022-03-11 03:22:32 2022-03-11 05:03:43 2022-03-11 05:26:27 0:22:44 0:10:57 0:11:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6731007 2022-03-11 03:22:33 2022-03-11 05:04:34 2022-03-11 05:40:24 0:35:50 0:24:03 0:11:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6731008 2022-03-11 03:22:33 2022-03-11 05:04:44 2022-03-11 05:26:23 0:21:39 0:10:53 0:10:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_image_read} 3
pass 6731009 2022-03-11 03:22:34 2022-03-11 05:04:55 2022-03-11 05:26:29 0:21:34 0:09:27 0:12:07 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_whole_object_zeroout} 1
dead 6731010 2022-03-11 03:22:35 2022-03-11 05:05:05 2022-03-11 05:22:55 0:17:50 0:04:23 0:13:27 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

{'smithi153.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'only_upgrade': False, 'package': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'policy_rc_d': None, 'purge': False, 'state': 'absent', 'update_cache': None, 'upgrade': None}}, 'msg': "'apt-get remove 'librbd1' 'librados2'' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 7388 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 7388 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 7388 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}}

pass 6731011 2022-03-11 03:22:35 2022-03-11 05:07:15 2022-03-11 05:57:06 0:49:51 0:38:16 0:11:35 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6731012 2022-03-11 03:22:36 2022-03-11 05:07:36 2022-03-11 05:31:36 0:24:00 0:10:55 0:13:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6731013 2022-03-11 03:22:37 2022-03-11 05:07:46 2022-03-11 05:36:23 0:28:37 0:17:03 0:11:34 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6731014 2022-03-11 03:22:38 2022-03-11 05:08:47 2022-03-11 05:30:19 0:21:32 0:11:05 0:10:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6731015 2022-03-11 03:22:38 2022-03-11 05:08:57 2022-03-11 05:38:53 0:29:56 0:15:59 0:13:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6731016 2022-03-11 03:22:39 2022-03-11 05:11:18 2022-03-11 05:36:41 0:25:23 0:12:22 0:13:01 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6731017 2022-03-11 03:22:40 2022-03-11 05:12:58 2022-03-11 05:49:12 0:36:14 0:24:17 0:11:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_simple_big} 3
pass 6731018 2022-03-11 03:22:40 2022-03-11 05:13:49 2022-03-11 05:33:31 0:19:42 0:09:48 0:09:54 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_deep_flatten} 1
pass 6731019 2022-03-11 03:22:41 2022-03-11 05:13:49 2022-03-11 05:50:06 0:36:17 0:25:10 0:11:07 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6731020 2022-03-11 03:22:42 2022-03-11 05:14:10 2022-03-11 05:55:01 0:40:51 0:27:46 0:13:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6731021 2022-03-11 03:22:42 2022-03-11 08:32:04 2022-03-11 09:07:39 0:35:35 0:19:18 0:16:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6731022 2022-03-11 03:22:43 2022-03-11 08:32:44 2022-03-11 08:54:08 0:21:24 0:09:35 0:11:49 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6731023 2022-03-11 03:22:44 2022-03-11 08:32:45 2022-03-11 09:08:17 0:35:32 0:21:36 0:13:56 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
fail 6731024 2022-03-11 03:22:45 2022-03-11 08:33:55 2022-03-11 09:06:24 0:32:29 0:19:40 0:12:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-03-11T09:04:45.188070+0000 mon.a (mon.0) 212 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731025 2022-03-11 03:22:45 2022-03-11 08:34:46 2022-03-11 09:12:50 0:38:04 0:22:19 0:15:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
fail 6731026 2022-03-11 03:22:46 2022-03-11 08:37:16 2022-03-11 09:09:40 0:32:24 0:19:52 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

"2022-03-11T08:57:29.474257+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6731027 2022-03-11 03:22:47 2022-03-11 08:38:17 2022-03-11 09:09:11 0:30:54 0:19:48 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-03-11T09:08:08.049558+0000 mon.a (mon.0) 272 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731028 2022-03-11 03:22:47 2022-03-11 08:39:27 2022-03-11 09:30:47 0:51:20 0:38:51 0:12:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6731029 2022-03-11 03:22:48 2022-03-11 08:41:08 2022-03-11 09:07:02 0:25:54 0:15:20 0:10:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6731030 2022-03-11 03:22:49 2022-03-11 08:41:18 2022-03-11 09:06:12 0:24:54 0:11:57 0:12:57 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6731031 2022-03-11 03:22:50 2022-03-11 08:42:49 2022-03-11 09:12:52 0:30:03 0:17:29 0:12:34 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6731032 2022-03-11 03:22:50 2022-03-11 08:43:09 2022-03-11 09:17:12 0:34:03 0:20:27 0:13:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-03-11T09:14:38.122824+0000 mon.a (mon.0) 312 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731033 2022-03-11 03:22:51 2022-03-11 08:43:40 2022-03-11 10:20:44 1:37:04 1:25:00 0:12:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6731034 2022-03-11 03:22:52 2022-03-11 08:45:00 2022-03-11 09:35:53 0:50:53 0:38:40 0:12:13 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6731035 2022-03-11 03:22:52 2022-03-11 08:46:11 2022-03-11 09:19:51 0:33:40 0:15:43 0:17:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6731036 2022-03-11 03:22:53 2022-03-11 08:47:51 2022-03-11 09:17:44 0:29:53 0:13:07 0:16:46 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6731037 2022-03-11 03:22:54 2022-03-11 08:48:12 2022-03-11 09:15:48 0:27:36 0:11:40 0:15:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6731038 2022-03-11 03:22:55 2022-03-11 08:48:22 2022-03-11 09:14:45 0:26:23 0:09:19 0:17:04 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_huge_image} 1
pass 6731039 2022-03-11 03:22:55 2022-03-11 08:48:32 2022-03-11 09:25:05 0:36:33 0:20:02 0:16:31 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6731040 2022-03-11 03:22:56 2022-03-11 08:48:53 2022-03-11 10:26:37 1:37:44 1:22:11 0:15:33 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6731041 2022-03-11 03:22:57 2022-03-11 08:49:03 2022-03-11 09:17:12 0:28:09 0:12:17 0:15:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6731042 2022-03-11 03:22:58 2022-03-11 08:49:24 2022-03-11 09:16:54 0:27:30 0:12:01 0:15:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6731043 2022-03-11 03:22:58 2022-03-11 08:49:24 2022-03-11 09:22:16 0:32:52 0:16:15 0:16:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
pass 6731044 2022-03-11 03:22:59 2022-03-11 08:50:14 2022-03-11 09:18:33 0:28:19 0:12:22 0:15:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6731045 2022-03-11 03:23:00 2022-03-11 08:50:35 2022-03-11 09:26:53 0:36:18 0:20:23 0:15:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6731046 2022-03-11 03:23:00 2022-03-11 08:51:15 2022-03-11 09:19:17 0:28:02 0:12:08 0:15:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6731047 2022-03-11 03:23:01 2022-03-11 08:51:46 2022-03-11 09:15:24 0:23:38 0:09:25 0:14:13 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_modprobe} 1
pass 6731048 2022-03-11 03:23:02 2022-03-11 08:51:56 2022-03-11 09:25:13 0:33:17 0:18:48 0:14:29 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6731049 2022-03-11 03:23:02 2022-03-11 08:51:56 2022-03-11 09:19:57 0:28:01 0:12:34 0:15:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
fail 6731050 2022-03-11 03:23:03 2022-03-11 08:52:07 2022-03-11 09:27:48 0:35:41 0:20:20 0:15:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-03-11T09:26:08.332130+0000 mon.a (mon.0) 214 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731051 2022-03-11 03:23:04 2022-03-11 08:52:17 2022-03-11 09:20:25 0:28:08 0:11:53 0:16:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6731052 2022-03-11 03:23:05 2022-03-11 08:52:58 2022-03-11 09:25:37 0:32:39 0:17:50 0:14:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6731053 2022-03-11 03:23:05 2022-03-11 08:54:08 2022-03-11 09:21:49 0:27:41 0:13:29 0:14:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6731054 2022-03-11 03:23:06 2022-03-11 08:54:19 2022-03-11 09:34:27 0:40:08 0:24:32 0:15:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_simple_big} 3
pass 6731055 2022-03-11 03:23:07 2022-03-11 08:54:59 2022-03-11 09:21:18 0:26:19 0:10:42 0:15:37 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6731056 2022-03-11 03:23:08 2022-03-11 08:55:40 2022-03-11 09:35:00 0:39:20 0:25:53 0:13:27 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6731057 2022-03-11 03:23:08 2022-03-11 08:55:40 2022-03-11 09:31:48 0:36:08 0:20:20 0:15:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-03-11T09:28:43.846412+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731058 2022-03-11 03:23:09 2022-03-11 08:57:41 2022-03-11 09:34:42 0:37:01 0:24:15 0:12:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_data_pool} 3
pass 6731059 2022-03-11 03:23:10 2022-03-11 08:58:31 2022-03-11 09:23:51 0:25:20 0:13:08 0:12:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6731060 2022-03-11 03:23:10 2022-03-11 08:58:32 2022-03-11 09:29:43 0:31:11 0:18:57 0:12:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6731061 2022-03-11 03:23:11 2022-03-11 08:59:22 2022-03-11 09:31:41 0:32:19 0:12:01 0:20:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_fallocate} 3
pass 6731062 2022-03-11 03:23:12 2022-03-11 09:06:14 2022-03-11 09:32:04 0:25:50 0:11:42 0:14:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6731063 2022-03-11 03:23:13 2022-03-11 09:06:34 2022-03-11 09:40:46 0:34:12 0:19:32 0:14:40 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-03-11T09:28:19.889566+0000 mon.a (mon.0) 194 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6731064 2022-03-11 03:23:13 2022-03-11 09:07:05 2022-03-11 09:44:50 0:37:45 0:19:40 0:18:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-03-11T09:42:12.626520+0000 mon.a (mon.0) 214 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6731065 2022-03-11 03:23:14 2022-03-11 09:07:45 2022-03-11 09:31:55 0:24:10 0:09:18 0:14:52 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_parent_overlap} 1
pass 6731066 2022-03-11 03:23:15 2022-03-11 09:08:26 2022-03-11 09:38:52 0:30:26 0:15:38 0:14:48 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6731067 2022-03-11 03:23:15 2022-03-11 09:09:16 2022-03-11 10:07:02 0:57:46 0:42:10 0:15:36 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6731068 2022-03-11 03:23:16 2022-03-11 09:09:47 2022-03-11 09:37:15 0:27:28 0:12:22 0:15:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6731069 2022-03-11 03:23:17 2022-03-11 09:11:37 2022-03-11 10:40:47 1:29:10 1:14:33 0:14:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6731070 2022-03-11 03:23:18 2022-03-11 09:12:58 2022-03-11 10:08:29 0:55:31 0:42:14 0:13:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6731071 2022-03-11 03:23:18 2022-03-11 09:12:58 2022-03-11 09:41:11 0:28:13 0:15:56 0:12:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6731072 2022-03-11 03:23:19 2022-03-11 09:13:39 2022-03-11 09:41:00 0:27:21 0:13:37 0:13:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6731073 2022-03-11 03:23:20 2022-03-11 09:14:49 2022-03-11 09:45:19 0:30:30 0:19:27 0:11:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-03-11T09:34:32.055991+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6731074 2022-03-11 03:23:20 2022-03-11 09:15:50 2022-03-11 09:35:09 0:19:19 0:09:47 0:09:32 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_read_only} 1
pass 6731075 2022-03-11 03:23:21 2022-03-11 09:15:50 2022-03-11 09:52:37 0:36:47 0:24:49 0:11:58 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6731076 2022-03-11 03:23:22 2022-03-11 09:17:01 2022-03-11 09:41:12 0:24:11 0:12:00 0:12:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6731077 2022-03-11 03:23:23 2022-03-11 09:17:21 2022-03-11 09:40:37 0:23:16 0:11:17 0:11:59 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6731078 2022-03-11 03:23:23 2022-03-11 09:17:21 2022-03-11 09:48:43 0:31:22 0:19:48 0:11:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-03-11T09:36:23.756731+0000 mon.a (mon.0) 180 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6731079 2022-03-11 03:23:24 2022-03-11 09:17:22 2022-03-11 09:41:25 0:24:03 0:11:00 0:13:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6731080 2022-03-11 03:23:25 2022-03-11 09:17:52 2022-03-11 09:50:50 0:32:58 0:19:53 0:13:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-03-11T09:38:16.215468+0000 mon.a (mon.0) 139 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6731081 2022-03-11 03:23:25 2022-03-11 09:18:42 2022-03-11 09:42:49 0:24:07 0:11:34 0:12:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_image_read} 3
pass 6731082 2022-03-11 03:23:26 2022-03-11 09:19:23 2022-03-11 09:41:16 0:21:53 0:10:08 0:11:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6731083 2022-03-11 03:23:27 2022-03-11 09:19:53 2022-03-11 09:49:54 0:30:01 0:17:38 0:12:23 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6731084 2022-03-11 03:23:27 2022-03-11 09:20:04 2022-03-11 09:41:58 0:21:54 0:10:40 0:11:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6731085 2022-03-11 03:23:28 2022-03-11 09:20:34 2022-03-11 09:49:21 0:28:47 0:16:30 0:12:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6731086 2022-03-11 03:23:29 2022-03-11 09:21:24 2022-03-11 09:45:24 0:24:00 0:10:57 0:13:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6731087 2022-03-11 03:23:30 2022-03-11 09:21:55 2022-03-11 09:50:26 0:28:31 0:15:41 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6731088 2022-03-11 03:23:30 2022-03-11 09:22:25 2022-03-11 09:47:21 0:24:56 0:12:50 0:12:06 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6731089 2022-03-11 03:23:31 2022-03-11 09:23:56 2022-03-11 10:05:21 0:41:25 0:27:05 0:14:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3