Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6582635 2021-12-24 03:21:10 2021-12-26 17:05:51 2021-12-26 17:25:47 0:19:56 0:09:41 0:10:15 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} 1
pass 6582636 2021-12-24 03:21:11 2021-12-26 17:05:51 2021-12-26 17:36:08 0:30:17 0:19:36 0:10:41 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6582637 2021-12-24 03:21:11 2021-12-26 17:05:52 2021-12-26 17:47:08 0:41:16 0:28:25 0:12:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} 3
pass 6582638 2021-12-24 03:21:12 2021-12-26 17:06:42 2021-12-26 17:37:20 0:30:38 0:18:55 0:11:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} 3
fail 6582639 2021-12-24 03:21:13 2021-12-26 17:07:12 2021-12-26 22:00:56 4:53:44 4:38:35 0:15:09 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi194 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludecxe4upfi -r -- '-g auto -g blockdev -x clone'"

pass 6582640 2021-12-24 03:21:14 2021-12-26 17:10:33 2021-12-26 18:03:02 0:52:29 0:40:57 0:11:32 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6582641 2021-12-24 03:21:14 2021-12-26 17:11:14 2021-12-26 17:18:55 0:07:41 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6582642 2021-12-24 03:21:15 2021-12-26 17:11:14 2021-12-26 17:31:15 0:20:01 0:09:41 0:10:20 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6582643 2021-12-24 03:21:16 2021-12-26 17:11:35 2021-12-26 17:34:20 0:22:45 0:11:48 0:10:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6582644 2021-12-24 03:21:17 2021-12-26 17:11:55 2021-12-26 17:42:54 0:30:59 0:19:41 0:11:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-12-26T17:31:10.557297+0000 mon.a (mon.0) 123 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582645 2021-12-24 03:21:18 2021-12-26 17:12:45 2021-12-26 17:35:13 0:22:28 0:11:39 0:10:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
pass 6582646 2021-12-24 03:21:18 2021-12-26 17:13:16 2021-12-26 17:39:18 0:26:02 0:12:04 0:13:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6582647 2021-12-24 03:21:19 2021-12-26 17:15:06 2021-12-26 18:03:10 0:48:04 0:37:00 0:11:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6582648 2021-12-24 03:21:20 2021-12-26 17:15:27 2021-12-26 17:41:27 0:26:00 0:13:56 0:12:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6582649 2021-12-24 03:21:21 2021-12-26 17:17:17 2021-12-26 17:40:21 0:23:04 0:10:11 0:12:53 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} 1
pass 6582650 2021-12-24 03:21:22 2021-12-26 17:18:38 2021-12-26 17:46:40 0:28:02 0:17:55 0:10:07 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6582651 2021-12-24 03:21:22 2021-12-26 17:18:38 2021-12-26 17:49:08 0:30:30 0:19:28 0:11:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2021-12-26T17:47:46.261385+0000 mon.a (mon.0) 296 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6582652 2021-12-24 03:21:23 2021-12-26 17:18:58 2021-12-26 17:50:00 0:31:02 0:19:32 0:11:30 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-12-26T17:48:24.165275+0000 mon.a (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582653 2021-12-24 03:21:24 2021-12-26 17:19:59 2021-12-26 18:12:29 0:52:30 0:37:46 0:14:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6582654 2021-12-24 03:21:25 2021-12-26 17:21:09 2021-12-26 17:52:17 0:31:08 0:19:14 0:11:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2021-12-26T17:51:02.732971+0000 mon.a (mon.0) 222 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582655 2021-12-24 03:21:25 2021-12-26 17:22:20 2021-12-26 17:54:27 0:32:07 0:19:43 0:12:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-12-26T17:41:08.004193+0000 mon.a (mon.0) 182 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582656 2021-12-24 03:21:26 2021-12-26 17:22:30 2021-12-26 17:47:18 0:24:48 0:11:50 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6582657 2021-12-24 03:21:27 2021-12-26 17:22:51 2021-12-26 17:42:46 0:19:55 0:09:28 0:10:27 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
fail 6582658 2021-12-24 03:21:28 2021-12-26 17:23:01 2021-12-26 17:55:24 0:32:23 0:19:48 0:12:35 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2021-12-26T17:52:26.178998+0000 mon.a (mon.0) 276 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582659 2021-12-24 03:21:29 2021-12-26 17:23:11 2021-12-26 17:47:42 0:24:31 0:12:32 0:11:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6582660 2021-12-24 03:21:29 2021-12-26 17:23:32 2021-12-26 17:55:52 0:32:20 0:19:42 0:12:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-12-26T17:54:33.110580+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582661 2021-12-24 03:21:30 2021-12-26 17:25:52 2021-12-26 17:54:45 0:28:53 0:15:55 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} 3
pass 6582662 2021-12-24 03:21:31 2021-12-26 17:28:33 2021-12-26 17:53:20 0:24:47 0:10:51 0:13:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6582663 2021-12-24 03:21:32 2021-12-26 17:31:24 2021-12-26 18:01:55 0:30:31 0:18:59 0:11:32 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6582664 2021-12-24 03:21:32 2021-12-26 17:31:54 2021-12-26 17:54:46 0:22:52 0:10:52 0:12:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} 3
pass 6582665 2021-12-24 03:21:33 2021-12-26 17:32:35 2021-12-26 17:52:15 0:19:40 0:09:24 0:10:16 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_modprobe} 1
pass 6582666 2021-12-24 03:21:34 2021-12-26 17:32:35 2021-12-26 18:00:51 0:28:16 0:15:26 0:12:50 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6582667 2021-12-24 03:21:35 2021-12-26 17:34:25 2021-12-26 19:03:17 1:28:52 1:16:23 0:12:29 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6582668 2021-12-24 03:21:36 2021-12-26 17:35:16 2021-12-26 17:58:44 0:23:28 0:10:44 0:12:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} 3
fail 6582669 2021-12-24 03:21:36 2021-12-26 17:36:16 2021-12-26 18:07:39 0:31:23 0:19:39 0:11:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2021-12-26T17:55:58.297708+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582670 2021-12-24 03:21:37 2021-12-26 17:37:27 2021-12-26 17:59:24 0:21:57 0:10:33 0:11:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6582671 2021-12-24 03:21:38 2021-12-26 17:37:27 2021-12-26 18:05:44 0:28:17 0:15:29 0:12:48 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} 3
pass 6582672 2021-12-24 03:21:39 2021-12-26 17:39:28 2021-12-26 18:02:14 0:22:46 0:11:49 0:10:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6582673 2021-12-24 03:21:40 2021-12-26 17:39:58 2021-12-26 18:19:41 0:39:43 0:26:32 0:13:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} 3
pass 6582674 2021-12-24 03:21:40 2021-12-26 17:41:29 2021-12-26 18:01:39 0:20:10 0:09:24 0:10:46 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} 1
pass 6582675 2021-12-24 03:21:41 2021-12-26 17:41:29 2021-12-26 18:16:37 0:35:08 0:23:50 0:11:18 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6582676 2021-12-24 03:21:42 2021-12-26 17:42:39 2021-12-26 18:23:00 0:40:21 0:28:17 0:12:04 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
fail 6582677 2021-12-24 03:21:43 2021-12-26 17:43:00 2021-12-26 18:16:39 0:33:39 0:19:24 0:14:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2021-12-26T18:15:23.945563+0000 mon.a (mon.0) 285 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6582678 2021-12-24 03:21:44 2021-12-26 17:46:41 2021-12-26 18:09:27 0:22:46 0:12:07 0:10:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6582679 2021-12-24 03:21:44 2021-12-26 17:47:11 2021-12-26 18:18:07 0:30:56 0:18:51 0:12:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6582680 2021-12-24 03:21:45 2021-12-26 17:47:21 2021-12-26 18:20:01 0:32:40 0:19:35 0:13:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2021-12-26T18:17:07.572427+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582681 2021-12-24 03:21:46 2021-12-26 17:47:52 2021-12-26 18:11:01 0:23:09 0:11:35 0:11:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6582682 2021-12-24 03:21:47 2021-12-26 17:49:12 2021-12-26 18:19:53 0:30:41 0:19:29 0:11:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-12-26T18:18:06.109214+0000 mon.a (mon.0) 239 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582683 2021-12-24 03:21:48 2021-12-26 17:50:03 2021-12-26 18:22:20 0:32:17 0:19:22 0:12:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2021-12-26T18:10:40.229932+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582684 2021-12-24 03:21:48 2021-12-26 17:52:23 2021-12-26 18:12:20 0:19:57 0:09:19 0:10:38 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
fail 6582685 2021-12-24 03:21:49 2021-12-26 17:52:24 2021-12-26 18:23:26 0:31:02 0:18:42 0:12:20 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6582686 2021-12-24 03:21:50 2021-12-26 17:53:24 2021-12-26 18:17:35 0:24:11 0:11:47 0:12:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} 3
pass 6582687 2021-12-24 03:21:51 2021-12-26 17:53:35 2021-12-26 19:01:10 1:07:35 0:53:40 0:13:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6582688 2021-12-24 03:21:51 2021-12-26 17:54:35 2021-12-26 18:47:13 0:52:38 0:40:27 0:12:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6582689 2021-12-24 03:21:52 2021-12-26 17:54:46 2021-12-26 18:21:05 0:26:19 0:15:14 0:11:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6582690 2021-12-24 03:21:53 2021-12-26 17:54:56 2021-12-26 18:21:25 0:26:29 0:13:48 0:12:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6582691 2021-12-24 03:21:54 2021-12-26 17:55:26 2021-12-26 18:19:56 0:24:30 0:12:01 0:12:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} 3
pass 6582692 2021-12-24 03:21:54 2021-12-26 17:55:57 2021-12-26 18:17:53 0:21:56 0:10:06 0:11:50 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} 1
pass 6582693 2021-12-24 03:21:55 2021-12-26 17:55:57 2021-12-26 18:29:00 0:33:03 0:18:14 0:14:49 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6582694 2021-12-24 03:21:56 2021-12-26 17:58:48 2021-12-26 18:53:39 0:54:51 0:42:25 0:12:26 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6582695 2021-12-24 03:21:57 2021-12-26 17:59:28 2021-12-26 18:23:08 0:23:40 0:11:46 0:11:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6582696 2021-12-24 03:21:58 2021-12-26 18:00:59 2021-12-26 18:24:00 0:23:01 0:10:44 0:12:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6582697 2021-12-24 03:21:58 2021-12-26 18:01:59 2021-12-26 18:30:35 0:28:36 0:15:59 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} 3
pass 6582698 2021-12-24 03:21:59 2021-12-26 18:02:19 2021-12-26 18:25:07 0:22:48 0:11:02 0:11:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6582699 2021-12-24 03:22:00 2021-12-26 18:03:10 2021-12-26 18:33:21 0:30:11 0:19:21 0:10:50 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2021-12-26T18:32:00.393709+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582700 2021-12-24 03:22:01 2021-12-26 18:03:20 2021-12-26 18:28:06 0:24:46 0:11:01 0:13:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6582701 2021-12-24 03:22:01 2021-12-26 18:05:51 2021-12-26 18:26:01 0:20:10 0:09:31 0:10:39 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} 1
pass 6582702 2021-12-24 03:22:02 2021-12-26 18:05:51 2021-12-26 18:39:58 0:34:07 0:20:08 0:13:59 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6582703 2021-12-24 03:22:03 2021-12-26 18:07:42 2021-12-26 18:31:44 0:24:02 0:10:27 0:13:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} 3
pass 6582704 2021-12-24 03:22:04 2021-12-26 18:09:32 2021-12-26 18:39:12 0:29:40 0:15:56 0:13:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6582705 2021-12-24 03:22:05 2021-12-26 18:11:03 2021-12-26 18:43:29 0:32:26 0:19:24 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2021-12-26T18:42:12.650582+0000 mon.a (mon.0) 242 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582706 2021-12-24 03:22:05 2021-12-26 18:12:33 2021-12-26 18:44:41 0:32:08 0:16:01 0:16:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6582707 2021-12-24 03:22:06 2021-12-26 18:16:44 2021-12-26 18:40:47 0:24:03 0:13:28 0:10:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6582708 2021-12-24 03:22:07 2021-12-26 18:16:45 2021-12-26 18:56:28 0:39:43 0:26:56 0:12:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} 3
pass 6582709 2021-12-24 03:22:08 2021-12-26 18:17:35 2021-12-26 18:37:31 0:19:56 0:10:05 0:09:51 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6582710 2021-12-24 03:22:08 2021-12-26 18:17:35 2021-12-26 18:50:59 0:33:24 0:20:54 0:12:30 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6582711 2021-12-24 03:22:09 2021-12-26 18:18:16 2021-12-26 19:00:00 0:41:44 0:28:53 0:12:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} 3
pass 6582712 2021-12-24 03:22:10 2021-12-26 18:19:46 2021-12-26 18:50:07 0:30:21 0:19:15 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} 3
pass 6582713 2021-12-24 03:22:11 2021-12-26 18:19:57 2021-12-26 18:39:45 0:19:48 0:09:50 0:09:58 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6582714 2021-12-24 03:22:11 2021-12-26 18:19:57 2021-12-26 18:54:08 0:34:11 0:22:00 0:12:11 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
fail 6582715 2021-12-24 03:22:12 2021-12-26 18:19:57 2021-12-26 18:52:04 0:32:07 0:20:00 0:12:07 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2021-12-26T18:49:44.119417+0000 mon.a (mon.0) 308 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582716 2021-12-24 03:22:13 2021-12-26 18:20:08 2021-12-26 18:53:48 0:33:40 0:20:09 0:13:31 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-12-26T18:51:02.645290+0000 mon.a (mon.0) 232 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582717 2021-12-24 03:22:14 2021-12-26 18:21:08 2021-12-26 18:45:41 0:24:33 0:11:59 0:12:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} 3
fail 6582718 2021-12-24 03:22:15 2021-12-26 18:21:29 2021-12-26 18:52:26 0:30:57 0:19:37 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2021-12-26T18:51:06.401471+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582719 2021-12-24 03:22:15 2021-12-26 18:22:29 2021-12-26 19:11:25 0:48:56 0:37:44 0:11:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6582720 2021-12-24 03:22:16 2021-12-26 18:23:10 2021-12-26 18:47:19 0:24:09 0:13:58 0:10:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6582721 2021-12-24 03:22:17 2021-12-26 18:23:10 2021-12-26 18:45:27 0:22:17 0:10:22 0:11:55 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} 1
pass 6582722 2021-12-24 03:22:18 2021-12-26 18:23:30 2021-12-26 18:54:02 0:30:32 0:18:19 0:12:13 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6582723 2021-12-24 03:22:19 2021-12-26 18:24:01 2021-12-26 20:01:32 1:37:31 1:25:37 0:11:54 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6582724 2021-12-24 03:22:19 2021-12-26 18:25:11 2021-12-26 18:48:32 0:23:21 0:11:43 0:11:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} 3
pass 6582725 2021-12-24 03:22:20 2021-12-26 18:26:02 2021-12-26 20:04:29 1:38:27 1:25:20 0:13:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6582726 2021-12-24 03:22:21 2021-12-26 18:28:12 2021-12-26 19:17:31 0:49:19 0:37:22 0:11:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6582727 2021-12-24 03:22:22 2021-12-26 18:29:03 2021-12-26 18:58:55 0:29:52 0:15:33 0:14:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6582728 2021-12-24 03:22:22 2021-12-26 18:30:43 2021-12-26 18:55:50 0:25:07 0:12:13 0:12:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6582729 2021-12-24 03:22:23 2021-12-26 18:31:54 2021-12-26 18:55:23 0:23:29 0:11:24 0:12:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} 3
pass 6582730 2021-12-24 03:22:24 2021-12-26 18:33:24 2021-12-26 18:53:00 0:19:36 0:09:05 0:10:31 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} 1
pass 6582731 2021-12-24 03:22:25 2021-12-26 18:33:45 2021-12-26 19:10:31 0:36:46 0:22:21 0:14:25 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6582732 2021-12-24 03:22:25 2021-12-26 18:36:05 2021-12-26 19:01:26 0:25:21 0:12:07 0:13:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6582733 2021-12-24 03:22:26 2021-12-26 18:39:16 2021-12-26 19:09:57 0:30:41 0:19:29 0:11:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-12-26T19:08:51.382394+0000 mon.a (mon.0) 262 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582734 2021-12-24 03:22:27 2021-12-26 18:39:47 2021-12-26 19:06:23 0:26:36 0:15:40 0:10:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6582735 2021-12-24 03:22:28 2021-12-26 18:40:07 2021-12-26 19:11:23 0:31:16 0:19:24 0:11:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2021-12-26T19:09:56.399676+0000 mon.a (mon.0) 313 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582736 2021-12-24 03:22:28 2021-12-26 18:40:57 2021-12-26 19:14:20 0:33:23 0:18:54 0:14:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6582737 2021-12-24 03:22:29 2021-12-26 18:43:38 2021-12-26 19:07:02 0:23:24 0:11:14 0:12:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} 3
pass 6582738 2021-12-24 03:22:30 2021-12-26 18:44:48 2021-12-26 19:05:29 0:20:41 0:09:18 0:11:23 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
fail 6582739 2021-12-24 03:22:31 2021-12-26 18:45:29 2021-12-26 19:16:32 0:31:03 0:19:41 0:11:22 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2021-12-26T19:15:21.132546+0000 mon.a (mon.0) 265 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582740 2021-12-24 03:22:32 2021-12-26 18:45:49 2021-12-26 19:18:19 0:32:30 0:19:43 0:12:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2021-12-26T19:17:13.955762+0000 mon.a (mon.0) 308 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582741 2021-12-24 03:22:32 2021-12-26 18:47:20 2021-12-26 19:17:33 0:30:13 0:19:51 0:10:22 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2021-12-26T19:16:25.396630+0000 mon.a (mon.0) 319 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582742 2021-12-24 03:22:33 2021-12-26 18:47:20 2021-12-26 19:10:49 0:23:29 0:10:53 0:12:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6582743 2021-12-24 03:22:34 2021-12-26 18:48:41 2021-12-26 19:16:19 0:27:38 0:15:28 0:12:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} 3
pass 6582744 2021-12-24 03:22:35 2021-12-26 18:50:11 2021-12-26 19:16:14 0:26:03 0:12:09 0:13:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6582745 2021-12-24 03:22:35 2021-12-26 18:51:02 2021-12-26 19:29:53 0:38:51 0:27:46 0:11:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6582746 2021-12-24 03:22:36 2021-12-26 18:52:12 2021-12-26 19:12:14 0:20:02 0:09:15 0:10:47 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} 1
pass 6582747 2021-12-24 03:22:37 2021-12-26 18:52:12 2021-12-26 19:28:26 0:36:14 0:25:11 0:11:03 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6582748 2021-12-24 03:22:38 2021-12-26 18:52:33 2021-12-26 19:33:40 0:41:07 0:29:18 0:11:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} 3
fail 6582749 2021-12-24 03:22:38 2021-12-26 18:53:43 2021-12-26 19:27:04 0:33:21 0:19:42 0:13:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2021-12-26T19:24:44.336404+0000 mon.a (mon.0) 279 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582750 2021-12-24 03:22:39 2021-12-26 18:53:54 2021-12-26 19:26:13 0:32:19 0:19:20 0:12:59 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{prefer-crc} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

"2021-12-26T19:23:28.761176+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582751 2021-12-24 03:22:40 2021-12-26 18:54:04 2021-12-26 20:46:25 1:52:21 1:37:33 0:14:48 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6582752 2021-12-24 03:22:41 2021-12-26 18:54:14 2021-12-26 19:19:31 0:25:17 0:11:45 0:13:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6582753 2021-12-24 03:22:42 2021-12-26 18:55:25 2021-12-26 19:25:23 0:29:58 0:18:37 0:11:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6582754 2021-12-24 03:22:42 2021-12-26 18:55:25 2021-12-26 19:28:08 0:32:43 0:19:23 0:13:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2021-12-26T19:25:32.293127+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582755 2021-12-24 03:22:43 2021-12-26 18:55:56 2021-12-26 19:20:43 0:24:47 0:11:51 0:12:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6582756 2021-12-24 03:22:44 2021-12-26 18:56:36 2021-12-26 19:29:08 0:32:32 0:19:32 0:13:00 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-12-26T19:17:14.255777+0000 mon.a (mon.0) 111 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582757 2021-12-24 03:22:45 2021-12-26 18:58:57 2021-12-26 19:26:18 0:27:21 0:14:23 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6582758 2021-12-24 03:22:45 2021-12-26 19:00:07 2021-12-26 19:20:19 0:20:12 0:09:11 0:11:01 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} 1
pass 6582759 2021-12-24 03:22:46 2021-12-26 19:00:08 2021-12-26 19:31:32 0:31:24 0:17:58 0:13:26 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6582760 2021-12-24 03:22:47 2021-12-26 19:01:18 2021-12-26 19:23:40 0:22:22 0:11:08 0:11:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6582761 2021-12-24 03:22:47 2021-12-26 19:01:28 2021-12-26 20:31:20 1:29:52 1:16:17 0:13:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6582762 2021-12-24 03:22:48 2021-12-26 19:03:19 2021-12-26 19:58:48 0:55:29 0:39:39 0:15:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6582763 2021-12-24 03:22:49 2021-12-26 19:06:30 2021-12-26 19:33:20 0:26:50 0:15:03 0:11:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6582764 2021-12-24 03:22:50 2021-12-26 19:07:10 2021-12-26 19:39:50 0:32:40 0:19:24 0:13:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-12-26T19:28:21.167347+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582765 2021-12-24 03:22:51 2021-12-26 19:10:01 2021-12-26 19:32:53 0:22:52 0:11:46 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6582766 2021-12-24 03:22:51 2021-12-26 19:10:41 2021-12-26 19:30:35 0:19:54 0:09:36 0:10:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
fail 6582767 2021-12-24 03:22:52 2021-12-26 19:10:42 2021-12-26 19:41:06 0:30:24 0:18:03 0:12:21 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Command failed on smithi161 with status 151: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0'

pass 6582768 2021-12-24 03:22:53 2021-12-26 19:10:52 2021-12-26 19:33:29 0:22:37 0:11:44 0:10:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6582769 2021-12-24 03:22:54 2021-12-26 19:11:32 2021-12-26 19:33:21 0:21:49 0:11:03 0:10:46 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6582770 2021-12-24 03:22:54 2021-12-26 19:11:33 2021-12-26 19:40:15 0:28:42 0:15:49 0:12:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} 3
pass 6582771 2021-12-24 03:22:55 2021-12-26 19:14:03 2021-12-26 19:37:45 0:23:42 0:10:39 0:13:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6582772 2021-12-24 03:22:56 2021-12-26 19:14:24 2021-12-26 19:48:22 0:33:58 0:19:29 0:14:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2021-12-26T19:45:44.559566+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582773 2021-12-24 03:22:57 2021-12-26 19:16:24 2021-12-26 19:39:07 0:22:43 0:11:12 0:11:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} 3
pass 6582774 2021-12-24 03:22:58 2021-12-26 19:16:25 2021-12-26 19:38:38 0:22:13 0:10:01 0:12:12 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} 1
pass 6582775 2021-12-24 03:22:58 2021-12-26 19:16:35 2021-12-26 19:43:48 0:27:13 0:15:18 0:11:55 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6582776 2021-12-24 03:22:59 2021-12-26 19:17:36 2021-12-26 20:06:01 0:48:25 0:37:41 0:10:44 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6582777 2021-12-24 03:23:00 2021-12-26 19:17:36 2021-12-26 19:40:57 0:23:21 0:10:42 0:12:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} 3
pass 6582778 2021-12-24 03:23:01 2021-12-26 19:18:26 2021-12-26 19:47:38 0:29:12 0:16:36 0:12:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6582779 2021-12-24 03:23:01 2021-12-26 19:19:37 2021-12-26 19:42:51 0:23:14 0:10:59 0:12:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6582780 2021-12-24 03:23:02 2021-12-26 19:20:27 2021-12-26 19:49:00 0:28:33 0:16:01 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} 3
fail 6582781 2021-12-24 03:23:03 2021-12-26 19:20:48 2021-12-26 19:54:00 0:33:12 0:19:35 0:13:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2021-12-26T19:52:29.334149+0000 mon.a (mon.0) 219 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582782 2021-12-24 03:23:04 2021-12-26 19:23:49 2021-12-26 20:03:39 0:39:50 0:26:58 0:12:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} 3
pass 6582783 2021-12-24 03:23:05 2021-12-26 19:25:29 2021-12-26 19:48:13 0:22:44 0:09:56 0:12:48 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} 1
fail 6582784 2021-12-24 03:23:05 2021-12-26 19:26:19 2021-12-26 19:58:37 0:32:18 0:19:16 0:13:02 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
Failure Reason:

"2021-12-26T19:55:40.541618+0000 mon.a (mon.0) 193 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582785 2021-12-24 03:23:06 2021-12-26 19:26:20 2021-12-26 20:06:06 0:39:46 0:28:22 0:11:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6582786 2021-12-24 03:23:07 2021-12-26 19:26:20 2021-12-26 19:57:10 0:30:50 0:19:28 0:11:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6582787 2021-12-24 03:23:08 2021-12-26 19:27:11 2021-12-26 19:50:28 0:23:17 0:10:00 0:13:17 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6582788 2021-12-24 03:23:09 2021-12-26 19:28:11 2021-12-26 20:00:28 0:32:17 0:21:41 0:10:36 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
fail 6582789 2021-12-24 03:23:09 2021-12-26 19:28:32 2021-12-26 19:59:15 0:30:43 0:19:29 0:11:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2021-12-26T19:47:27.432511+0000 mon.a (mon.0) 164 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582790 2021-12-24 03:23:10 2021-12-26 19:29:12 2021-12-26 20:00:00 0:30:48 0:19:03 0:11:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6582791 2021-12-24 03:23:11 2021-12-26 19:30:03 2021-12-26 19:53:30 0:23:27 0:11:33 0:11:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} 3
fail 6582792 2021-12-24 03:23:12 2021-12-26 19:31:33 2021-12-26 20:02:48 0:31:15 0:19:38 0:11:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2021-12-26T20:01:27.884773+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582793 2021-12-24 03:23:13 2021-12-26 19:32:13 2021-12-26 20:21:08 0:48:55 0:37:11 0:11:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6582794 2021-12-24 03:23:13 2021-12-26 19:32:54 2021-12-26 20:00:28 0:27:34 0:13:54 0:13:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6582795 2021-12-24 03:23:14 2021-12-26 19:33:24 2021-12-26 19:55:35 0:22:11 0:10:26 0:11:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
fail 6582796 2021-12-24 03:23:15 2021-12-26 19:33:25 2021-12-26 20:03:21 0:29:56 0:18:28 0:11:28 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2021-12-26T19:51:54.055250+0000 mon.a (mon.0) 122 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582797 2021-12-24 03:23:16 2021-12-26 19:33:25 2021-12-26 19:57:36 0:24:11 0:11:46 0:12:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} 3
fail 6582798 2021-12-24 03:23:16 2021-12-26 19:33:35 2021-12-26 20:05:46 0:32:11 0:19:22 0:12:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-12-26T20:02:57.508160+0000 mon.a (mon.0) 228 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6582799 2021-12-24 03:23:17 2021-12-26 19:33:46 2021-12-26 20:27:06 0:53:20 0:37:06 0:16:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6582800 2021-12-24 03:23:18 2021-12-26 19:37:46 2021-12-26 20:12:00 0:34:14 0:19:56 0:14:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2021-12-26T20:09:31.627153+0000 mon.a (mon.0) 296 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6582801 2021-12-24 03:23:19 2021-12-26 19:39:17 2021-12-26 20:03:58 0:24:41 0:12:18 0:12:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6582802 2021-12-24 03:23:19 2021-12-26 19:39:57 2021-12-26 20:04:30 0:24:33 0:11:31 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} 3
pass 6582803 2021-12-24 03:23:20 2021-12-26 19:40:18 2021-12-26 20:02:30 0:22:12 0:09:54 0:12:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} 1
pass 6582804 2021-12-24 03:23:21 2021-12-26 19:40:18 2021-12-26 20:13:35 0:33:17 0:21:30 0:11:47 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6582805 2021-12-24 03:23:22 2021-12-26 19:40:59 2021-12-26 21:17:15 1:36:16 1:24:57 0:11:19 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6582806 2021-12-24 03:23:23 2021-12-26 19:41:09 2021-12-26 20:13:25 0:32:16 0:19:23 0:12:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2021-12-26T20:01:46.448143+0000 mon.a (mon.0) 128 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6582807 2021-12-24 03:23:23 2021-12-26 19:43:00 2021-12-26 20:16:03 0:33:03 0:19:41 0:13:22 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-12-26T20:02:37.649292+0000 mon.a (mon.0) 202 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6582808 2021-12-24 03:23:24 2021-12-26 19:43:50 2021-12-26 20:17:38 0:33:48 0:19:29 0:14:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} 3
Failure Reason:

"2021-12-26T20:16:16.227839+0000 mon.a (mon.0) 205 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582809 2021-12-24 03:23:25 2021-12-26 19:47:41 2021-12-26 20:11:31 0:23:50 0:11:03 0:12:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6582810 2021-12-24 03:23:26 2021-12-26 19:48:31 2021-12-26 20:19:15 0:30:44 0:19:30 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6582811 2021-12-24 03:23:26 2021-12-26 19:49:02 2021-12-26 20:12:46 0:23:44 0:11:18 0:12:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6582812 2021-12-24 03:23:27 2021-12-26 19:50:12 2021-12-26 20:10:06 0:19:54 0:08:58 0:10:56 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_modprobe} 1
pass 6582813 2021-12-24 03:23:28 2021-12-26 19:50:13 2021-12-26 20:20:58 0:30:45 0:19:21 0:11:24 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6582814 2021-12-24 03:23:29 2021-12-26 19:50:33 2021-12-26 20:23:49 0:33:16 0:19:31 0:13:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2021-12-26T20:22:44.809766+0000 mon.a (mon.0) 252 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582815 2021-12-24 03:23:30 2021-12-26 19:53:34 2021-12-26 20:21:52 0:28:18 0:17:15 0:11:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
fail 6582816 2021-12-24 03:23:30 2021-12-26 19:54:04 2021-12-26 20:29:38 0:35:34 0:19:31 0:16:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2021-12-26T20:27:01.489979+0000 mon.a (mon.0) 288 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582817 2021-12-24 03:23:31 2021-12-26 19:57:05 2021-12-26 20:29:28 0:32:23 0:19:45 0:12:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2021-12-26T20:26:26.456637+0000 mon.a (mon.0) 300 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582818 2021-12-24 03:23:32 2021-12-26 19:57:15 2021-12-26 20:21:59 0:24:44 0:12:06 0:12:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6582819 2021-12-24 03:23:33 2021-12-26 19:57:46 2021-12-26 20:36:36 0:38:50 0:26:19 0:12:31 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} 3
pass 6582820 2021-12-24 03:23:33 2021-12-26 19:58:46 2021-12-26 20:19:07 0:20:21 0:09:08 0:11:13 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6582821 2021-12-24 03:23:34 2021-12-26 19:58:56 2021-12-26 20:35:17 0:36:21 0:25:17 0:11:04 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6582822 2021-12-24 03:23:35 2021-12-26 19:59:17 2021-12-26 20:30:17 0:31:00 0:19:24 0:11:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2021-12-26T20:28:49.327668+0000 mon.a (mon.0) 281 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582823 2021-12-24 03:23:36 2021-12-26 20:00:07 2021-12-26 20:34:40 0:34:33 0:22:29 0:12:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} 3
pass 6582824 2021-12-24 03:23:37 2021-12-26 20:00:38 2021-12-26 20:23:05 0:22:27 0:11:31 0:10:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6582825 2021-12-24 03:23:37 2021-12-26 20:00:38 2021-12-26 20:31:40 0:31:02 0:18:54 0:12:08 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6582826 2021-12-24 03:23:38 2021-12-26 20:34:43 2021-12-26 21:06:04 0:31:21 0:19:31 0:11:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2021-12-26T21:04:47.111569+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582827 2021-12-24 03:23:39 2021-12-26 20:35:24 2021-12-26 20:59:04 0:23:40 0:11:28 0:12:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6582828 2021-12-24 03:23:40 2021-12-26 20:36:44 2021-12-26 21:16:44 0:40:00 0:19:27 0:20:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-12-26T21:15:14.790010+0000 mon.a (mon.0) 217 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582829 2021-12-24 03:23:40 2021-12-26 20:46:36 2021-12-26 21:23:26 0:36:50 0:14:09 0:22:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6582830 2021-12-24 03:23:41 2021-12-26 20:57:07 2021-12-26 21:17:10 0:20:03 0:09:21 0:10:42 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} 1
pass 6582831 2021-12-24 03:23:42 2021-12-26 20:57:08 2021-12-26 21:25:24 0:28:16 0:15:40 0:12:36 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6582832 2021-12-24 03:23:43 2021-12-26 20:59:08 2021-12-26 22:01:26 1:02:18 0:45:05 0:17:13 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6582833 2021-12-24 03:23:44 2021-12-26 21:05:29 2021-12-26 21:28:17 0:22:48 0:11:26 0:11:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} 3
pass 6582834 2021-12-24 03:23:44 2021-12-26 21:06:10 2021-12-26 22:44:52 1:38:42 1:16:58 0:21:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6582835 2021-12-24 03:23:45 2021-12-26 21:16:51 2021-12-26 22:09:39 0:52:48 0:41:07 0:11:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6582836 2021-12-24 03:23:46 2021-12-26 21:17:22 2021-12-26 21:50:00 0:32:38 0:15:09 0:17:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6582837 2021-12-24 03:23:47 2021-12-26 21:23:33 2021-12-26 21:55:48 0:32:15 0:19:22 0:12:53 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-12-26T21:43:41.968398+0000 mon.a (mon.0) 110 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6582838 2021-12-24 03:23:47 2021-12-26 21:25:33 2021-12-26 21:58:51 0:33:18 0:19:23 0:13:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2021-12-26T21:57:31.285986+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582839 2021-12-24 03:23:48 2021-12-26 21:28:24 2021-12-26 21:48:39 0:20:15 0:09:45 0:10:30 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} 1
pass 6582840 2021-12-24 03:23:49 2021-12-26 21:28:24 2021-12-26 22:23:52 0:55:28 0:23:20 0:32:08 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6582841 2021-12-24 03:23:50 2021-12-26 21:49:27 2021-12-26 22:12:35 0:23:08 0:11:45 0:11:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6582842 2021-12-24 03:23:50 2021-12-26 21:50:08 2021-12-26 22:12:26 0:22:18 0:10:53 0:11:25 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6582843 2021-12-24 03:23:51 2021-12-26 21:50:38 2021-12-26 22:21:30 0:30:52 0:19:47 0:11:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2021-12-26T22:09:52.735863+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6582844 2021-12-24 03:23:52 2021-12-26 21:51:18 2021-12-26 22:16:52 0:25:34 0:10:41 0:14:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6582845 2021-12-24 03:23:53 2021-12-26 21:54:19 2021-12-26 22:29:37 0:35:18 0:23:07 0:12:11 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2021-12-26T22:18:35.472258+0000 mon.a (mon.0) 234 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 31 sec, osd.7 has slow ops (SLOW_OPS)" in cluster log

fail 6582846 2021-12-24 03:23:54 2021-12-26 21:55:30 2021-12-26 22:26:02 0:30:32 0:19:43 0:10:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2021-12-26T22:24:43.439731+0000 mon.a (mon.0) 254 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6582847 2021-12-24 03:23:54 2021-12-26 21:55:50 2021-12-26 22:15:51 0:20:01 0:09:57 0:10:04 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6582848 2021-12-24 03:23:55 2021-12-26 21:55:50 2021-12-26 22:26:46 0:30:56 0:18:11 0:12:45 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6582849 2021-12-24 03:23:56 2021-12-26 21:56:01 2021-12-26 22:21:09 0:25:08 0:10:56 0:14:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6582850 2021-12-24 03:23:57 2021-12-26 21:59:01 2021-12-26 22:27:23 0:28:22 0:16:11 0:12:11 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6582851 2021-12-24 03:23:57 2021-12-26 21:59:22 2021-12-26 22:21:42 0:22:20 0:11:10 0:11:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6582852 2021-12-24 03:23:58 2021-12-26 21:59:42 2021-12-26 22:26:25 0:26:43 0:15:51 0:10:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} 3
fail 6582853 2021-12-24 03:23:59 2021-12-26 22:00:12 2021-12-26 22:30:45 0:30:33 0:19:26 0:11:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2021-12-26T22:29:37.590594+0000 mon.a (mon.0) 275 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6582854 2021-12-24 03:24:00 2021-12-26 22:00:43 2021-12-26 22:33:39 0:32:56 0:19:44 0:13:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2021-12-26T22:30:34.190384+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log