Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6831579 2022-05-13 03:20:13 2022-05-15 18:52:18 2022-05-15 19:16:53 0:24:35 0:12:06 0:12:29 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_deep_flatten} 1
pass 6831580 2022-05-13 03:20:13 2022-05-15 18:52:18 2022-05-15 19:26:09 0:33:51 0:22:45 0:11:06 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6831581 2022-05-13 03:20:14 2022-05-15 18:52:39 2022-05-15 19:15:40 0:23:01 0:13:08 0:09:53 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_default_map_options} 3
pass 6831582 2022-05-13 03:20:15 2022-05-15 18:52:39 2022-05-15 19:38:19 0:45:40 0:30:19 0:15:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6831583 2022-05-13 03:20:16 2022-05-15 18:56:10 2022-05-15 19:28:35 0:32:25 0:21:44 0:10:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_data_pool} 3
fail 6831584 2022-05-13 03:20:17 2022-05-15 18:56:21 2022-05-15 23:32:15 4:35:54 4:21:25 0:14:29 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi174 with status 2: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeuwcepmug -r -- '-g auto -g blockdev -x clone'"

pass 6831585 2022-05-13 03:20:18 2022-05-15 19:00:22 2022-05-15 19:58:43 0:58:21 0:43:52 0:14:29 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6831586 2022-05-13 03:20:19 2022-05-15 19:02:03 2022-05-15 19:11:08 0:09:05 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6831587 2022-05-13 03:20:20 2022-05-15 19:03:43 2022-05-15 19:24:40 0:20:57 0:11:54 0:09:03 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6831588 2022-05-13 03:20:21 2022-05-15 19:03:44 2022-05-15 19:33:14 0:29:30 0:14:43 0:14:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6831589 2022-05-13 03:20:22 2022-05-15 19:05:54 2022-05-15 19:37:09 0:31:15 0:22:00 0:09:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-15T19:35:33.678753+0000 mon.a (mon.0) 223 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831590 2022-05-13 03:20:23 2022-05-15 19:06:05 2022-05-15 19:32:27 0:26:22 0:13:49 0:12:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6831591 2022-05-13 03:20:24 2022-05-15 19:06:45 2022-05-15 19:42:42 0:35:57 0:22:24 0:13:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-05-15T19:29:33.827311+0000 mon.a (mon.0) 161 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831592 2022-05-13 03:20:25 2022-05-15 19:06:46 2022-05-15 19:56:11 0:49:25 0:39:45 0:09:40 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6831593 2022-05-13 03:20:26 2022-05-15 19:06:46 2022-05-15 19:37:11 0:30:25 0:17:07 0:13:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6831594 2022-05-13 03:20:26 2022-05-15 19:07:16 2022-05-15 19:28:15 0:20:59 0:12:21 0:08:38 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_discard} 1
pass 6831595 2022-05-13 03:20:27 2022-05-15 19:07:17 2022-05-15 19:37:15 0:29:58 0:18:56 0:11:02 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6831596 2022-05-13 03:20:28 2022-05-15 19:07:47 2022-05-15 19:34:08 0:26:21 0:15:04 0:11:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
fail 6831597 2022-05-13 03:20:29 2022-05-15 19:08:28 2022-05-15 19:43:22 0:34:54 0:22:26 0:12:28 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-15T19:40:22.845304+0000 mon.a (mon.0) 341 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6831598 2022-05-13 03:20:30 2022-05-15 19:09:38 2022-05-15 20:02:33 0:52:55 0:41:55 0:11:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6831599 2022-05-13 03:20:31 2022-05-15 19:11:09 2022-05-15 19:41:03 0:29:54 0:17:57 0:11:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6831600 2022-05-13 03:20:32 2022-05-15 19:13:40 2022-05-15 19:39:17 0:25:37 0:14:53 0:10:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6831601 2022-05-13 03:20:33 2022-05-15 19:13:40 2022-05-15 19:40:58 0:27:18 0:13:56 0:13:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6831602 2022-05-13 03:20:34 2022-05-15 19:15:31 2022-05-15 19:36:29 0:20:58 0:12:06 0:08:52 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
fail 6831603 2022-05-13 03:20:35 2022-05-15 19:15:31 2022-05-15 19:48:44 0:33:13 0:21:58 0:11:15 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2022-05-15T19:45:37.402193+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831604 2022-05-13 03:20:36 2022-05-15 19:15:42 2022-05-15 19:48:25 0:32:43 0:21:48 0:10:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-15T19:47:06.038604+0000 mon.a (mon.0) 235 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831605 2022-05-13 03:20:37 2022-05-15 19:16:42 2022-05-15 19:51:32 0:34:50 0:21:41 0:13:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-15T19:48:37.718084+0000 mon.a (mon.0) 313 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831606 2022-05-13 03:20:38 2022-05-15 19:17:03 2022-05-15 19:53:03 0:36:00 0:18:05 0:17:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
pass 6831607 2022-05-13 03:20:39 2022-05-15 19:23:34 2022-05-15 19:46:53 0:23:19 0:13:35 0:09:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6831608 2022-05-13 03:20:40 2022-05-15 19:23:34 2022-05-15 19:59:40 0:36:06 0:22:01 0:14:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6831609 2022-05-13 03:20:41 2022-05-15 19:26:15 2022-05-15 19:50:05 0:23:50 0:13:25 0:10:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6831610 2022-05-13 03:20:41 2022-05-15 19:26:36 2022-05-15 19:47:37 0:21:01 0:11:20 0:09:41 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_modprobe} 1
pass 6831611 2022-05-13 03:20:42 2022-05-15 19:26:36 2022-05-15 19:58:46 0:32:10 0:18:12 0:13:58 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6831612 2022-05-13 03:20:43 2022-05-15 19:28:37 2022-05-15 20:40:18 1:11:41 0:59:31 0:12:10 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6831613 2022-05-13 03:20:44 2022-05-15 19:28:47 2022-05-15 20:02:15 0:33:28 0:21:45 0:11:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-05-15T20:00:09.388614+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831614 2022-05-13 03:20:45 2022-05-15 19:28:48 2022-05-15 20:02:03 0:33:15 0:19:54 0:13:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6831615 2022-05-13 03:20:46 2022-05-15 19:29:28 2022-05-15 19:56:07 0:26:39 0:12:54 0:13:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6831616 2022-05-13 03:20:47 2022-05-15 19:32:29 2022-05-15 20:03:24 0:30:55 0:18:08 0:12:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6831617 2022-05-13 03:20:48 2022-05-15 19:33:19 2022-05-15 19:59:50 0:26:31 0:14:22 0:12:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6831618 2022-05-13 03:20:49 2022-05-15 19:34:10 2022-05-15 20:11:53 0:37:43 0:26:06 0:11:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_simple_big} 3
pass 6831619 2022-05-13 03:20:50 2022-05-15 19:36:31 2022-05-15 19:57:56 0:21:25 0:11:39 0:09:46 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_msgr_segments} 1
pass 6831620 2022-05-13 03:20:51 2022-05-15 19:37:11 2022-05-15 20:16:45 0:39:34 0:29:39 0:09:55 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
fail 6831621 2022-05-13 03:20:51 2022-05-15 19:37:21 2022-05-15 20:11:19 0:33:58 0:22:03 0:11:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-05-15T20:09:01.811298+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831622 2022-05-13 03:20:52 2022-05-15 19:37:22 2022-05-15 20:16:29 0:39:07 0:26:35 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6831623 2022-05-13 03:20:53 2022-05-15 19:38:22 2022-05-15 20:02:58 0:24:36 0:13:58 0:10:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6831624 2022-05-13 03:20:54 2022-05-15 19:39:23 2022-05-15 20:14:36 0:35:13 0:21:58 0:13:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6831625 2022-05-13 03:20:55 2022-05-15 19:41:03 2022-05-15 20:14:35 0:33:32 0:22:12 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-15T20:01:18.776283+0000 mon.a (mon.0) 192 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831626 2022-05-13 03:20:56 2022-05-15 19:41:14 2022-05-15 20:08:27 0:27:13 0:14:23 0:12:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6831627 2022-05-13 03:20:57 2022-05-15 19:42:44 2022-05-15 20:16:53 0:34:09 0:21:56 0:12:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-15T20:13:54.749262+0000 mon.a (mon.0) 251 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831628 2022-05-13 03:20:58 2022-05-15 19:43:25 2022-05-15 20:15:32 0:32:07 0:21:51 0:10:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-15T20:14:17.859901+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831629 2022-05-13 03:20:59 2022-05-15 19:43:55 2022-05-15 20:04:57 0:21:02 0:12:00 0:09:02 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6831630 2022-05-13 03:21:00 2022-05-15 19:43:56 2022-05-15 20:18:28 0:34:32 0:20:38 0:13:54 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6831631 2022-05-13 03:21:01 2022-05-15 19:46:56 2022-05-15 20:18:50 0:31:54 0:21:44 0:10:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

"2022-05-15T20:17:44.397427+0000 mon.a (mon.0) 216 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831632 2022-05-13 03:21:02 2022-05-15 19:47:47 2022-05-15 20:58:01 1:10:14 0:58:37 0:11:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6831633 2022-05-13 03:21:02 2022-05-15 19:48:28 2022-05-15 20:45:40 0:57:12 0:45:43 0:11:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6831634 2022-05-13 03:21:03 2022-05-15 19:48:48 2022-05-15 20:17:35 0:28:47 0:17:24 0:11:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6831635 2022-05-13 03:21:04 2022-05-15 19:50:09 2022-05-15 20:20:08 0:29:59 0:15:25 0:14:34 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6831636 2022-05-13 03:21:05 2022-05-15 19:51:39 2022-05-15 20:16:36 0:24:57 0:13:52 0:11:05 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6831637 2022-05-13 03:21:06 2022-05-15 19:53:10 2022-05-15 20:17:30 0:24:20 0:12:09 0:12:11 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_read_only} 1
pass 6831638 2022-05-13 03:21:07 2022-05-15 19:54:10 2022-05-15 20:27:54 0:33:44 0:20:01 0:13:43 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6831639 2022-05-13 03:21:08 2022-05-15 19:56:11 2022-05-15 20:54:01 0:57:50 0:46:30 0:11:20 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
fail 6831640 2022-05-13 03:21:09 2022-05-15 19:56:21 2022-05-15 20:35:05 0:38:44 0:22:35 0:16:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-15T20:31:51.624354+0000 mon.a (mon.0) 285 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831641 2022-05-13 03:21:10 2022-05-15 19:58:52 2022-05-15 20:25:30 0:26:38 0:14:09 0:12:29 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6831642 2022-05-13 03:21:11 2022-05-15 19:58:53 2022-05-15 20:33:07 0:34:14 0:22:18 0:11:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-15T20:31:23.998166+0000 mon.a (mon.0) 295 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6831643 2022-05-13 03:21:12 2022-05-15 19:59:43 2022-05-15 20:25:38 0:25:55 0:13:56 0:11:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6831644 2022-05-13 03:21:13 2022-05-15 19:59:54 2022-05-15 20:38:42 0:38:48 0:24:41 0:14:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6831645 2022-05-13 03:21:14 2022-05-15 20:02:04 2022-05-15 20:35:43 0:33:39 0:22:01 0:11:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-05-15T20:23:23.506889+0000 mon.a (mon.0) 192 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6831646 2022-05-13 03:21:14 2022-05-15 20:02:25 2022-05-15 20:23:02 0:20:37 0:12:40 0:07:57 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_whole_object_zeroout} 1
pass 6831647 2022-05-13 03:21:15 2022-05-15 20:02:25 2022-05-15 20:33:58 0:31:33 0:21:29 0:10:04 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6831648 2022-05-13 03:21:16 2022-05-15 20:02:35 2022-05-15 20:28:41 0:26:06 0:13:41 0:12:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6831649 2022-05-13 03:21:17 2022-05-15 20:03:06 2022-05-15 20:33:31 0:30:25 0:19:11 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6831650 2022-05-13 03:21:18 2022-05-15 20:03:26 2022-05-15 20:37:16 0:33:50 0:22:34 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-15T20:35:11.202809+0000 mon.a (mon.0) 232 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831651 2022-05-13 03:21:19 2022-05-15 20:03:57 2022-05-15 20:38:22 0:34:25 0:17:52 0:16:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6831652 2022-05-13 03:21:20 2022-05-15 20:08:28 2022-05-15 20:39:22 0:30:54 0:15:37 0:15:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6831653 2022-05-13 03:21:21 2022-05-15 20:11:28 2022-05-15 20:49:22 0:37:54 0:27:42 0:10:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6831654 2022-05-13 03:21:22 2022-05-15 20:11:59 2022-05-15 20:35:30 0:23:31 0:12:18 0:11:13 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6831655 2022-05-13 03:21:23 2022-05-15 20:11:59 2022-05-15 20:50:13 0:38:14 0:24:07 0:14:07 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6831656 2022-05-13 03:21:24 2022-05-15 20:14:40 2022-05-15 20:55:51 0:41:11 0:30:34 0:10:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_fio} 3
pass 6831657 2022-05-13 03:21:25 2022-05-15 20:14:41 2022-05-15 20:49:11 0:34:30 0:21:22 0:13:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_data_pool} 3
pass 6831658 2022-05-13 03:21:26 2022-05-15 20:15:41 2022-05-15 20:42:45 0:27:04 0:12:53 0:14:11 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6831659 2022-05-13 03:21:27 2022-05-15 20:16:32 2022-05-15 20:52:30 0:35:58 0:24:06 0:11:52 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
fail 6831660 2022-05-13 03:21:28 2022-05-15 20:16:42 2022-05-15 20:50:14 0:33:32 0:22:04 0:11:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

"2022-05-15T20:48:46.355514+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831661 2022-05-13 03:21:29 2022-05-15 20:16:52 2022-05-15 20:52:36 0:35:44 0:23:21 0:12:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-05-15T20:50:53.560565+0000 mon.a (mon.0) 245 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831662 2022-05-13 03:21:30 2022-05-15 20:17:03 2022-05-15 20:45:03 0:28:00 0:14:49 0:13:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
fail 6831663 2022-05-13 03:21:30 2022-05-15 20:17:33 2022-05-15 20:53:14 0:35:41 0:23:03 0:12:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-05-15T20:50:42.030795+0000 mon.a (mon.0) 233 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831664 2022-05-13 03:21:31 2022-05-15 20:17:44 2022-05-15 21:10:08 0:52:24 0:40:44 0:11:40 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6831665 2022-05-13 03:21:32 2022-05-15 20:18:34 2022-05-15 20:47:41 0:29:07 0:17:45 0:11:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6831666 2022-05-13 03:21:33 2022-05-15 20:18:55 2022-05-15 20:43:40 0:24:45 0:13:17 0:11:28 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_discard} 1
pass 6831667 2022-05-13 03:21:34 2022-05-15 20:20:15 2022-05-15 20:55:34 0:35:19 0:19:53 0:15:26 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6831668 2022-05-13 03:21:35 2022-05-15 20:23:06 2022-05-15 21:44:22 1:21:16 1:07:26 0:13:50 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6831669 2022-05-13 03:21:36 2022-05-15 20:25:37 2022-05-15 20:51:32 0:25:55 0:14:39 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_namespaces} 3
fail 6831670 2022-05-13 03:21:37 2022-05-15 20:25:47 2022-05-15 21:45:39 1:19:52 1:06:09 0:13:43 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-05-15T20:50:17.301809+0000 osd.1 (osd.1) 3 : cluster [WRN] 1 slow requests (by type [ 'started' : 1 ] most affected pool [ 'rbd' : 1 ])" in cluster log

pass 6831671 2022-05-13 03:21:38 2022-05-15 20:27:58 2022-05-15 21:22:27 0:54:29 0:41:50 0:12:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6831672 2022-05-13 03:21:39 2022-05-15 20:28:48 2022-05-15 21:06:47 0:37:59 0:21:42 0:16:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-05-15T21:04:06.994252+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831673 2022-05-13 03:21:40 2022-05-15 20:33:09 2022-05-15 20:59:51 0:26:42 0:14:39 0:12:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6831674 2022-05-13 03:21:41 2022-05-15 20:33:40 2022-05-15 20:59:30 0:25:50 0:13:56 0:11:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6831675 2022-05-13 03:21:42 2022-05-15 20:34:00 2022-05-15 20:57:06 0:23:06 0:11:20 0:11:46 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_huge_image} 1
pass 6831676 2022-05-13 03:21:43 2022-05-15 20:35:11 2022-05-15 21:10:58 0:35:47 0:24:44 0:11:03 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
fail 6831677 2022-05-13 03:21:44 2022-05-15 20:35:31 2022-05-15 21:09:09 0:33:38 0:22:11 0:11:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-15T21:07:08.658927+0000 mon.a (mon.0) 232 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831678 2022-05-13 03:21:45 2022-05-15 20:35:51 2022-05-15 21:08:40 0:32:49 0:21:44 0:11:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2022-05-15T20:57:00.682223+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831679 2022-05-13 03:21:45 2022-05-15 20:37:22 2022-05-15 21:08:10 0:30:48 0:18:08 0:12:40 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6831680 2022-05-13 03:21:46 2022-05-15 20:38:22 2022-05-15 21:13:16 0:34:54 0:21:52 0:13:02 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-05-15T21:10:37.297484+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831681 2022-05-13 03:21:47 2022-05-15 20:38:43 2022-05-15 21:13:23 0:34:40 0:22:01 0:12:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6831682 2022-05-13 03:21:48 2022-05-15 20:39:23 2022-05-15 21:03:45 0:24:22 0:13:11 0:11:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_image_read} 3
pass 6831683 2022-05-13 03:21:49 2022-05-15 20:40:24 2022-05-15 21:04:22 0:23:58 0:11:44 0:12:14 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6831684 2022-05-13 03:21:50 2022-05-15 20:42:54 2022-05-15 21:18:36 0:35:42 0:22:27 0:13:15 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6831685 2022-05-13 03:21:51 2022-05-15 20:45:05 2022-05-15 21:16:57 0:31:52 0:22:09 0:09:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-05-15T21:15:24.947184+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831686 2022-05-13 03:21:52 2022-05-15 20:45:46 2022-05-15 21:20:49 0:35:03 0:22:05 0:12:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-05-15T21:17:33.650037+0000 mon.a (mon.0) 319 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831687 2022-05-13 03:21:53 2022-05-15 20:49:16 2022-05-15 21:12:45 0:23:29 0:12:55 0:10:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6831688 2022-05-13 03:21:54 2022-05-15 20:49:27 2022-05-15 21:21:31 0:32:04 0:21:54 0:10:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-05-15T21:20:12.479456+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831689 2022-05-13 03:21:55 2022-05-15 20:50:17 2022-05-15 21:13:45 0:23:28 0:14:06 0:09:22 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6831690 2022-05-13 03:21:56 2022-05-15 20:50:18 2022-05-15 21:31:21 0:41:03 0:28:38 0:12:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6831691 2022-05-13 03:21:57 2022-05-15 20:51:38 2022-05-15 21:13:05 0:21:27 0:11:24 0:10:03 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_msgr_segments} 1
pass 6831692 2022-05-13 03:21:57 2022-05-15 20:51:39 2022-05-15 21:28:08 0:36:29 0:25:57 0:10:32 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6831693 2022-05-13 03:21:58 2022-05-15 20:52:39 2022-05-15 21:18:02 0:25:23 0:14:04 0:11:19 smithi master krbd/ms_modeless/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
fail 6831694 2022-05-13 03:21:59 2022-05-15 20:52:40 2022-05-15 21:27:14 0:34:34 0:21:58 0:12:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2022-05-15T21:24:24.863395+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831695 2022-05-13 03:22:00 2022-05-15 20:53:20 2022-05-15 21:28:13 0:34:53 0:21:49 0:13:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-05-15T21:25:45.865994+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831696 2022-05-13 03:22:01 2022-05-15 20:54:11 2022-05-16 01:56:18 5:02:07 4:47:33 0:14:34 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi078 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude0__6fiwd -r -- '-g auto -g blockdev -x clone'"

pass 6831697 2022-05-13 03:22:02 2022-05-15 20:55:41 2022-05-15 22:09:24 1:13:43 1:01:45 0:11:58 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6831698 2022-05-13 03:22:03 2022-05-15 20:56:02 2022-05-15 21:21:41 0:25:39 0:14:05 0:11:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6831699 2022-05-13 03:22:04 2022-05-15 20:58:02 2022-05-15 21:31:10 0:33:08 0:20:51 0:12:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6831700 2022-05-13 03:22:05 2022-05-15 20:59:33 2022-05-15 21:31:59 0:32:26 0:21:35 0:10:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-15T21:30:27.096404+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831701 2022-05-13 03:22:06 2022-05-15 20:59:53 2022-05-15 21:27:22 0:27:29 0:14:06 0:13:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6831702 2022-05-13 03:22:07 2022-05-15 21:03:54 2022-05-15 21:58:12 0:54:18 0:40:48 0:13:30 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6831703 2022-05-13 03:22:08 2022-05-15 21:06:55 2022-05-15 21:35:05 0:28:10 0:17:04 0:11:06 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6831704 2022-05-13 03:22:09 2022-05-15 21:06:56 2022-05-15 21:29:52 0:22:56 0:11:38 0:11:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6831705 2022-05-13 03:22:10 2022-05-15 21:08:16 2022-05-15 21:40:05 0:31:49 0:20:56 0:10:53 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6831706 2022-05-13 03:22:11 2022-05-15 21:08:47 2022-05-15 21:34:31 0:25:44 0:14:12 0:11:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6831707 2022-05-13 03:22:12 2022-05-15 21:09:17 2022-05-15 22:19:19 1:10:02 0:58:20 0:11:42 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6831708 2022-05-13 03:22:13 2022-05-15 21:09:27 2022-05-15 22:05:54 0:56:27 0:44:07 0:12:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6831709 2022-05-13 03:22:14 2022-05-15 21:10:18 2022-05-15 21:38:40 0:28:22 0:17:46 0:10:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6831710 2022-05-13 03:22:14 2022-05-15 21:11:08 2022-05-15 21:44:10 0:33:02 0:21:53 0:11:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-15T21:42:34.503045+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6831711 2022-05-13 03:22:15 2022-05-15 21:12:49 2022-05-15 21:47:17 0:34:28 0:21:56 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-15T21:35:09.388054+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831712 2022-05-13 03:22:16 2022-05-15 21:13:09 2022-05-15 21:34:29 0:21:20 0:12:21 0:08:59 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6831713 2022-05-13 03:22:17 2022-05-15 21:13:20 2022-05-15 21:53:32 0:40:12 0:29:00 0:11:12 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6831714 2022-05-13 03:22:18 2022-05-15 21:13:30 2022-05-15 21:39:20 0:25:50 0:14:17 0:11:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6831715 2022-05-13 03:22:19 2022-05-15 21:13:51 2022-05-15 21:40:00 0:26:09 0:13:28 0:12:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6831716 2022-05-13 03:22:20 2022-05-15 21:17:01 2022-05-15 21:45:33 0:28:32 0:18:17 0:10:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_concurrent} 3
pass 6831717 2022-05-13 03:22:21 2022-05-15 21:18:12 2022-05-15 21:42:09 0:23:57 0:13:07 0:10:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6831718 2022-05-13 03:22:22 2022-05-15 21:18:42 2022-05-15 21:55:49 0:37:07 0:25:29 0:11:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6831719 2022-05-13 03:22:23 2022-05-15 21:20:53 2022-05-15 21:44:49 0:23:56 0:13:29 0:10:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
pass 6831720 2022-05-13 03:22:24 2022-05-15 21:21:34 2022-05-15 21:45:57 0:24:23 0:12:29 0:11:54 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6831721 2022-05-13 03:22:25 2022-05-15 21:21:34 2022-05-15 21:49:31 0:27:57 0:16:54 0:11:03 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6831722 2022-05-13 03:22:26 2022-05-15 21:21:44 2022-05-15 22:14:16 0:52:32 0:40:38 0:11:54 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6831723 2022-05-13 03:22:26 2022-05-15 21:22:35 2022-05-15 21:47:19 0:24:44 0:12:50 0:11:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_kernel} 3
pass 6831724 2022-05-13 03:22:27 2022-05-15 21:22:55 2022-05-15 21:56:35 0:33:40 0:19:05 0:14:35 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6831725 2022-05-13 03:22:28 2022-05-15 21:27:16 2022-05-15 21:50:50 0:23:34 0:12:52 0:10:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6831726 2022-05-13 03:22:29 2022-05-15 21:27:27 2022-05-15 21:55:28 0:28:01 0:17:52 0:10:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
fail 6831727 2022-05-13 03:22:30 2022-05-15 21:28:17 2022-05-15 21:59:54 0:31:37 0:21:42 0:09:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2022-05-15T21:48:02.328771+0000 mon.a (mon.0) 162 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831728 2022-05-13 03:22:31 2022-05-15 21:28:18 2022-05-15 22:08:49 0:40:31 0:27:14 0:13:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6831729 2022-05-13 03:22:32 2022-05-15 21:31:18 2022-05-15 21:52:53 0:21:35 0:11:57 0:09:38 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_deep_flatten} 1
pass 6831730 2022-05-13 03:22:33 2022-05-15 21:31:19 2022-05-15 22:11:11 0:39:52 0:28:50 0:11:02 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6831731 2022-05-13 03:22:34 2022-05-15 21:31:29 2022-05-15 22:14:07 0:42:38 0:30:37 0:12:01 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6831732 2022-05-13 03:22:35 2022-05-15 21:32:10 2022-05-15 22:06:11 0:34:01 0:22:01 0:12:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6831733 2022-05-13 03:22:36 2022-05-15 21:34:40 2022-05-15 21:56:00 0:21:20 0:12:01 0:09:19 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6831734 2022-05-13 03:22:37 2022-05-15 21:34:41 2022-05-15 22:09:21 0:34:40 0:23:54 0:10:46 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6831735 2022-05-13 03:22:38 2022-05-15 21:35:11 2022-05-15 22:02:11 0:27:00 0:14:16 0:12:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6831736 2022-05-13 03:22:39 2022-05-15 21:38:42 2022-05-15 22:10:57 0:32:15 0:21:43 0:10:32 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
fail 6831737 2022-05-13 03:22:40 2022-05-15 21:39:23 2022-05-15 22:13:19 0:33:56 0:22:22 0:11:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-15T22:10:48.165164+0000 mon.a (mon.0) 292 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831738 2022-05-13 03:22:41 2022-05-15 21:40:03 2022-05-15 22:05:43 0:25:40 0:14:43 0:10:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6831739 2022-05-13 03:22:41 2022-05-15 21:40:13 2022-05-15 22:30:13 0:50:00 0:40:07 0:09:53 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
fail 6831740 2022-05-13 03:22:42 2022-05-15 21:40:54 2022-05-15 22:13:42 0:32:48 0:21:55 0:10:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-15T22:12:16.157731+0000 mon.a (mon.0) 235 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6831741 2022-05-13 03:22:43 2022-05-15 21:42:15 2022-05-15 22:03:13 0:20:58 0:12:46 0:08:12 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6831742 2022-05-13 03:22:44 2022-05-15 21:42:15 2022-05-15 22:11:28 0:29:13 0:18:12 0:11:01 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6831743 2022-05-13 03:22:45 2022-05-15 21:44:16 2022-05-15 22:12:55 0:28:39 0:15:04 0:13:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_namespaces} 3
pass 6831744 2022-05-13 03:22:46 2022-05-15 21:44:26 2022-05-15 23:04:28 1:20:02 1:07:05 0:12:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6831745 2022-05-13 03:22:47 2022-05-15 21:44:56 2022-05-15 22:41:06 0:56:10 0:41:37 0:14:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6831746 2022-05-13 03:22:48 2022-05-15 21:45:37 2022-05-15 22:15:17 0:29:40 0:17:25 0:12:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6831747 2022-05-13 03:22:49 2022-05-15 21:45:47 2022-05-15 22:17:18 0:31:31 0:14:55 0:16:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6831748 2022-05-13 03:22:50 2022-05-15 21:47:18 2022-05-15 22:13:42 0:26:24 0:13:54 0:12:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6831749 2022-05-13 03:22:51 2022-05-15 21:47:28 2022-05-15 22:14:01 0:26:33 0:11:39 0:14:54 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_huge_image} 1
pass 6831750 2022-05-13 03:22:52 2022-05-15 21:47:29 2022-05-15 22:25:05 0:37:36 0:24:10 0:13:26 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6831751 2022-05-13 03:22:53 2022-05-15 21:47:39 2022-05-15 23:05:25 1:17:46 1:03:22 0:14:24 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6831752 2022-05-13 03:22:53 2022-05-15 21:49:40 2022-05-15 22:24:19 0:34:39 0:22:00 0:12:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-05-15T22:22:04.867862+0000 mon.a (mon.0) 213 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831753 2022-05-13 03:22:54 2022-05-15 21:50:50 2022-05-15 22:16:47 0:25:57 0:13:38 0:12:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6831754 2022-05-13 03:22:55 2022-05-16 01:22:58 2022-05-16 01:54:47 0:31:49 0:18:14 0:13:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_concurrent} 3
pass 6831755 2022-05-13 03:22:56 2022-05-16 01:24:48 2022-05-16 01:48:18 0:23:30 0:13:18 0:10:12 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6831756 2022-05-13 03:22:57 2022-05-16 01:24:49 2022-05-16 02:00:41 0:35:52 0:22:28 0:13:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6831757 2022-05-13 03:22:58 2022-05-16 01:27:19 2022-05-16 01:52:54 0:25:35 0:13:38 0:11:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6831758 2022-05-13 03:22:59 2022-05-16 01:27:40 2022-05-16 01:51:03 0:23:23 0:11:17 0:12:06 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_modprobe} 1
fail 6831759 2022-05-13 03:23:00 2022-05-16 01:27:40 2022-05-16 02:01:40 0:34:00 0:21:48 0:12:12 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
Failure Reason:

"2022-05-16T01:48:33.767245+0000 mon.a (mon.0) 197 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831760 2022-05-13 03:23:01 2022-05-16 01:28:01 2022-05-16 01:55:32 0:27:31 0:13:01 0:14:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
pass 6831761 2022-05-13 03:23:02 2022-05-16 01:31:32 2022-05-16 02:01:03 0:29:31 0:19:58 0:09:33 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6831762 2022-05-13 03:23:03 2022-05-16 01:31:32 2022-05-16 01:55:32 0:24:00 0:13:00 0:11:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6831763 2022-05-13 03:23:04 2022-05-16 01:32:13 2022-05-16 02:06:24 0:34:11 0:22:10 0:12:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-05-16T02:03:43.248395+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831764 2022-05-13 03:23:05 2022-05-16 01:32:13 2022-05-16 01:58:51 0:26:38 0:14:02 0:12:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6831765 2022-05-13 03:23:05 2022-05-16 01:32:54 2022-05-16 02:10:28 0:37:34 0:26:35 0:10:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6831766 2022-05-13 03:23:06 2022-05-16 01:33:14 2022-05-16 01:54:25 0:21:11 0:11:33 0:09:38 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6831767 2022-05-13 03:23:07 2022-05-16 01:33:14 2022-05-16 02:14:46 0:41:32 0:29:46 0:11:46 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6831768 2022-05-13 03:23:08 2022-05-16 01:34:45 2022-05-16 02:17:08 0:42:23 0:30:53 0:11:30 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_fio} 3
fail 6831769 2022-05-13 03:23:09 2022-05-16 01:35:25 2022-05-16 02:11:06 0:35:41 0:22:18 0:13:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-05-16T01:58:09.784661+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6831770 2022-05-13 03:23:10 2022-05-16 01:37:26 2022-05-16 02:03:17 0:25:51 0:14:21 0:11:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6831771 2022-05-13 03:23:11 2022-05-16 01:37:36 2022-05-16 02:10:49 0:33:13 0:21:14 0:11:59 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6831772 2022-05-13 03:23:12 2022-05-16 01:39:27 2022-05-16 02:12:07 0:32:40 0:21:42 0:10:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-05-16T02:10:43.206884+0000 mon.a (mon.0) 263 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6831773 2022-05-13 03:23:13 2022-05-16 01:40:38 2022-05-16 02:04:32 0:23:54 0:13:59 0:09:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6831774 2022-05-13 03:23:14 2022-05-16 01:41:08 2022-05-16 02:18:36 0:37:28 0:21:41 0:15:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-05-16T02:06:28.272789+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6831775 2022-05-13 03:23:15 2022-05-16 01:47:09 2022-05-16 02:21:39 0:34:30 0:23:02 0:11:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-05-16T02:20:18.035810+0000 mon.a (mon.0) 270 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831776 2022-05-13 03:23:16 2022-05-16 01:47:40 2022-05-16 02:11:57 0:24:17 0:12:54 0:11:23 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_parent_overlap} 1
pass 6831777 2022-05-13 03:23:17 2022-05-16 01:47:40 2022-05-16 02:19:49 0:32:09 0:19:57 0:12:12 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6831778 2022-05-13 03:23:18 2022-05-16 01:48:21 2022-05-16 02:39:45 0:51:24 0:41:06 0:10:18 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6831779 2022-05-13 03:23:19 2022-05-16 01:48:21 2022-05-16 02:14:00 0:25:39 0:14:50 0:10:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6831780 2022-05-13 03:23:19 2022-05-16 01:48:42 2022-05-16 03:00:18 1:11:36 1:00:21 0:11:15 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6831781 2022-05-13 03:23:20 2022-05-16 01:49:02 2022-05-16 02:47:58 0:58:56 0:46:11 0:12:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6831782 2022-05-13 03:23:22 2022-05-16 01:49:22 2022-05-16 02:20:27 0:31:05 0:18:09 0:12:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6831783 2022-05-13 03:23:22 2022-05-16 01:51:03 2022-05-16 02:24:24 0:33:21 0:22:24 0:10:57 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-05-16T02:11:22.133980+0000 mon.a (mon.0) 150 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6831784 2022-05-13 03:23:23 2022-05-16 01:51:03 2022-05-16 02:25:57 0:34:54 0:22:13 0:12:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-05-16T02:23:33.443623+0000 mon.a (mon.0) 264 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831785 2022-05-13 03:23:24 2022-05-16 01:51:24 2022-05-16 02:14:50 0:23:26 0:12:37 0:10:49 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_read_only} 1
pass 6831786 2022-05-13 03:23:25 2022-05-16 01:51:24 2022-05-16 02:28:18 0:36:54 0:24:16 0:12:38 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6831787 2022-05-13 03:23:26 2022-05-16 01:52:55 2022-05-16 02:19:20 0:26:25 0:14:27 0:11:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6831788 2022-05-13 03:23:27 2022-05-16 01:53:36 2022-05-16 02:17:26 0:23:50 0:13:41 0:10:09 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6831789 2022-05-13 03:23:28 2022-05-16 01:53:56 2022-05-16 03:39:10 1:45:14 1:31:17 0:13:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-05-16T03:21:00.041932+0000 osd.3 (osd.3) 3 : cluster [WRN] 1 slow requests (by type [ 'started' : 1 ] most affected pool [ 'rbd' : 1 ])" in cluster log

pass 6831790 2022-05-13 03:23:29 2022-05-16 01:54:56 2022-05-16 02:19:46 0:24:50 0:13:34 0:11:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6831791 2022-05-13 03:23:30 2022-05-16 01:55:37 2022-05-16 02:29:05 0:33:28 0:21:49 0:11:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-05-16T02:26:12.633684+0000 mon.a (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6831792 2022-05-13 03:23:31 2022-05-16 01:55:37 2022-05-16 02:22:54 0:27:17 0:13:39 0:13:38 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_image_read} 3
pass 6831793 2022-05-13 03:23:32 2022-05-16 01:56:28 2022-05-16 02:18:56 0:22:28 0:12:32 0:09:56 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6831794 2022-05-13 03:23:33 2022-05-16 01:56:28 2022-05-16 02:30:57 0:34:29 0:20:39 0:13:50 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6831795 2022-05-13 03:23:34 2022-05-16 01:58:59 2022-05-16 02:24:37 0:25:38 0:13:11 0:12:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6831796 2022-05-13 03:23:35 2022-05-16 02:00:49 2022-05-16 02:30:41 0:29:52 0:19:04 0:10:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6831797 2022-05-13 03:23:36 2022-05-16 02:01:10 2022-05-16 02:34:53 0:33:43 0:21:51 0:11:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-05-16T02:32:43.194556+0000 mon.a (mon.0) 284 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6831798 2022-05-13 03:23:37 2022-05-16 02:01:10 2022-05-16 02:31:25 0:30:15 0:18:20 0:11:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6831799 2022-05-13 03:23:37 2022-05-16 02:01:51 2022-05-16 02:27:58 0:26:07 0:14:54 0:11:13 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6831800 2022-05-13 03:23:38 2022-05-16 02:02:31 2022-05-16 02:43:02 0:40:31 0:29:30 0:11:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3