Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6655383 2022-02-02 15:31:50 2022-02-02 15:48:00 2022-02-02 16:12:01 0:24:01 0:11:57 0:12:04 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_deep_flatten} 1
pass 6655384 2022-02-02 15:31:51 2022-02-02 15:48:00 2022-02-02 16:23:25 0:35:25 0:24:06 0:11:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6655385 2022-02-02 15:31:52 2022-02-02 15:48:01 2022-02-02 16:29:52 0:41:51 0:29:57 0:11:54 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_fio} 3
pass 6655386 2022-02-02 15:31:53 2022-02-02 15:48:21 2022-02-02 16:24:37 0:36:16 0:23:44 0:12:32 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
fail 6655387 2022-02-02 15:31:54 2022-02-02 15:48:41 2022-02-02 16:26:11 0:37:30 0:19:38 0:17:52 smithi master krbd/rxbounce/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/krbd_rxbounce} 3
Failure Reason:

Command failed (workunit test rbd/krbd_rxbounce.sh) on smithi167 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=239f3b058ecd10b05894b75aab82b58c444cda60 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/krbd_rxbounce.sh'

fail 6655388 2022-02-02 15:31:55 2022-02-02 15:51:32 2022-02-02 20:40:00 4:48:28 4:35:18 0:13:10 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi161 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludefktduxcc -r -- '-g auto -g blockdev -x clone'"

pass 6655389 2022-02-02 15:31:56 2022-02-02 15:52:03 2022-02-02 16:49:39 0:57:36 0:44:36 0:13:00 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6655390 2022-02-02 15:31:57 2022-02-02 15:52:13 2022-02-02 16:00:00 0:07:47 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6655391 2022-02-02 15:31:58 2022-02-02 15:52:23 2022-02-02 16:15:57 0:23:34 0:12:01 0:11:33 smithi master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6655392 2022-02-02 15:31:59 2022-02-02 15:52:34 2022-02-02 16:26:04 0:33:30 0:20:00 0:13:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6655393 2022-02-02 15:32:01 2022-02-02 15:52:34 2022-02-02 16:41:04 0:48:30 0:33:12 0:15:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6655394 2022-02-02 15:32:02 2022-02-02 15:53:14 2022-02-02 16:34:07 0:40:53 0:24:07 0:16:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
pass 6655395 2022-02-02 15:32:03 2022-02-02 15:53:45 2022-02-02 16:25:28 0:31:43 0:15:43 0:16:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6655396 2022-02-02 15:32:04 2022-02-02 15:54:05 2022-02-02 16:52:00 0:57:55 0:42:17 0:15:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6655397 2022-02-02 15:32:05 2022-02-02 15:54:26 2022-02-02 16:46:31 0:52:05 0:36:54 0:15:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6655398 2022-02-02 15:32:06 2022-02-02 15:55:06 2022-02-02 16:20:16 0:25:10 0:10:39 0:14:31 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_discard} 1
pass 6655399 2022-02-02 15:32:07 2022-02-02 15:55:07 2022-02-02 16:38:26 0:43:19 0:25:44 0:17:35 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6655400 2022-02-02 15:32:08 2022-02-02 15:55:47 2022-02-02 16:29:27 0:33:40 0:18:18 0:15:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
pass 6655401 2022-02-02 15:32:09 2022-02-02 15:55:48 2022-02-02 17:37:35 1:41:47 1:25:58 0:15:49 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6655402 2022-02-02 15:32:10 2022-02-02 15:57:48 2022-02-02 16:58:27 1:00:39 0:44:52 0:15:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6655403 2022-02-02 15:32:11 2022-02-02 15:58:19 2022-02-02 16:36:17 0:37:58 0:23:29 0:14:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6655404 2022-02-02 15:32:12 2022-02-02 15:58:39 2022-02-02 16:47:51 0:49:12 0:34:27 0:14:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6655405 2022-02-02 15:32:13 2022-02-02 16:00:10 2022-02-02 16:31:10 0:31:00 0:17:56 0:13:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6655406 2022-02-02 15:32:14 2022-02-02 16:01:10 2022-02-02 16:27:34 0:26:24 0:13:39 0:12:45 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6655407 2022-02-02 15:32:15 2022-02-02 16:04:01 2022-02-02 16:45:51 0:41:50 0:31:55 0:09:55 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6655408 2022-02-02 15:32:16 2022-02-02 16:04:01 2022-02-02 16:39:37 0:35:36 0:19:26 0:16:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6655409 2022-02-02 15:32:18 2022-02-02 16:08:02 2022-02-02 16:35:58 0:27:56 0:15:39 0:12:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6655410 2022-02-02 15:32:19 2022-02-02 16:08:23 2022-02-02 16:38:58 0:30:35 0:19:17 0:11:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_concurrent} 3
fail 6655411 2022-02-02 15:32:20 2022-02-02 16:09:23 2022-02-02 16:44:32 0:35:09 0:23:54 0:11:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-02-02T16:43:00.162992+0000 mon.a (mon.0) 236 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655412 2022-02-02 15:32:21 2022-02-02 16:11:04 2022-02-02 16:48:05 0:37:01 0:25:47 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6655413 2022-02-02 15:32:22 2022-02-02 16:12:04 2022-02-02 16:43:27 0:31:23 0:18:59 0:12:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_image_read} 3
pass 6655414 2022-02-02 15:32:23 2022-02-02 16:13:45 2022-02-02 16:40:44 0:26:59 0:16:32 0:10:27 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_modprobe} 1
pass 6655415 2022-02-02 15:32:24 2022-02-02 16:13:45 2022-02-02 16:49:07 0:35:22 0:20:20 0:15:02 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6655416 2022-02-02 15:32:25 2022-02-02 16:16:46 2022-02-02 17:53:09 1:36:23 1:23:57 0:12:26 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6655417 2022-02-02 15:32:26 2022-02-02 16:16:56 2022-02-02 16:48:59 0:32:03 0:20:14 0:11:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_kernel} 3
pass 6655418 2022-02-02 15:32:27 2022-02-02 16:17:37 2022-02-02 16:54:41 0:37:04 0:23:40 0:13:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6655419 2022-02-02 15:32:28 2022-02-02 16:18:17 2022-02-02 16:49:47 0:31:30 0:17:01 0:14:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6655420 2022-02-02 15:32:29 2022-02-02 16:19:38 2022-02-02 16:54:03 0:34:25 0:22:05 0:12:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6655421 2022-02-02 15:32:30 2022-02-02 16:20:18 2022-02-02 16:50:12 0:29:54 0:17:59 0:11:55 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6655422 2022-02-02 15:32:31 2022-02-02 16:20:49 2022-02-02 16:58:54 0:38:05 0:27:10 0:10:55 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_simple_big} 3
pass 6655423 2022-02-02 15:32:32 2022-02-02 16:21:09 2022-02-02 16:43:50 0:22:41 0:11:49 0:10:52 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_msgr_segments} 1
pass 6655424 2022-02-02 15:32:34 2022-02-02 16:23:00 2022-02-02 17:01:11 0:38:11 0:27:43 0:10:28 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6655425 2022-02-02 15:32:35 2022-02-02 16:23:30 2022-02-02 17:12:34 0:49:04 0:35:23 0:13:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6655426 2022-02-02 15:32:36 2022-02-02 16:24:41 2022-02-02 17:09:05 0:44:24 0:31:59 0:12:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
pass 6655427 2022-02-02 15:32:37 2022-02-02 16:25:31 2022-02-02 16:57:09 0:31:38 0:18:57 0:12:41 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6655428 2022-02-02 15:32:38 2022-02-02 16:25:42 2022-02-02 17:04:49 0:39:07 0:24:20 0:14:47 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6655429 2022-02-02 15:32:39 2022-02-02 16:26:12 2022-02-02 16:55:54 0:29:42 0:18:22 0:11:20 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_fallocate} 3
pass 6655430 2022-02-02 15:32:40 2022-02-02 16:26:12 2022-02-02 16:58:24 0:32:12 0:19:15 0:12:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6655431 2022-02-02 15:32:41 2022-02-02 16:26:53 2022-02-02 17:25:21 0:58:28 0:45:11 0:13:17 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6655432 2022-02-02 15:32:42 2022-02-02 16:27:43 2022-02-02 17:08:47 0:41:04 0:28:00 0:13:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2022-02-02T16:56:24.876081+0000 mon.a (mon.0) 162 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655433 2022-02-02 15:32:43 2022-02-02 16:29:24 2022-02-02 16:57:02 0:27:38 0:14:37 0:13:01 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
pass 6655434 2022-02-02 15:32:44 2022-02-02 16:29:24 2022-02-02 17:09:27 0:40:03 0:27:27 0:12:36 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6655435 2022-02-02 15:32:45 2022-02-02 16:29:35 2022-02-02 17:01:30 0:31:55 0:18:37 0:13:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_namespaces} 3
pass 6655436 2022-02-02 15:32:46 2022-02-02 16:29:55 2022-02-02 18:01:35 1:31:40 1:18:28 0:13:12 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6655437 2022-02-02 15:32:47 2022-02-02 16:31:15 2022-02-02 17:20:48 0:49:33 0:34:56 0:14:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2022-02-02T17:18:48.542438+0000 mon.a (mon.0) 243 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6655438 2022-02-02 15:32:48 2022-02-02 16:34:16 2022-02-02 17:15:47 0:41:31 0:28:08 0:13:23 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-02-02T17:03:11.016683+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6655439 2022-02-02 15:32:49 2022-02-02 16:36:07 2022-02-02 17:11:51 0:35:44 0:24:19 0:11:25 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-02-02T17:10:06.631462+0000 mon.a (mon.0) 337 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6655440 2022-02-02 15:32:50 2022-02-02 16:36:27 2022-02-02 17:27:15 0:50:48 0:34:54 0:15:54 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2022-02-02T17:13:47.976293+0000 mon.a (mon.0) 135 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655441 2022-02-02 15:32:52 2022-02-02 16:38:28 2022-02-02 17:03:51 0:25:23 0:14:08 0:11:15 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_read_only} 1
pass 6655442 2022-02-02 15:32:53 2022-02-02 16:38:28 2022-02-02 17:14:23 0:35:55 0:23:57 0:11:58 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6655443 2022-02-02 15:32:54 2022-02-02 16:38:58 2022-02-02 17:35:16 0:56:18 0:45:13 0:11:05 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6655444 2022-02-02 15:32:55 2022-02-02 16:39:39 2022-02-02 17:13:13 0:33:34 0:18:55 0:14:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6655445 2022-02-02 15:32:56 2022-02-02 16:41:09 2022-02-02 17:11:11 0:30:02 0:15:55 0:14:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6655446 2022-02-02 15:32:57 2022-02-02 16:43:30 2022-02-02 17:21:57 0:38:27 0:27:30 0:10:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-02-02T17:09:51.215041+0000 mon.a (mon.0) 173 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655447 2022-02-02 15:32:58 2022-02-02 16:44:40 2022-02-02 17:16:02 0:31:22 0:18:55 0:12:27 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6655448 2022-02-02 15:32:59 2022-02-02 16:45:21 2022-02-02 17:19:38 0:34:17 0:22:07 0:12:10 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-02-02T17:06:40.378407+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6655449 2022-02-02 15:33:00 2022-02-02 16:45:51 2022-02-02 17:26:03 0:40:12 0:27:26 0:12:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-02-02T17:23:07.777635+0000 mon.a (mon.0) 324 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6655450 2022-02-02 15:33:01 2022-02-02 16:46:32 2022-02-02 17:15:47 0:29:15 0:19:33 0:09:42 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_whole_object_zeroout} 1
pass 6655451 2022-02-02 15:33:02 2022-02-02 16:46:32 2022-02-02 17:23:42 0:37:10 0:23:24 0:13:46 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6655452 2022-02-02 15:33:03 2022-02-02 16:47:53 2022-02-02 17:14:14 0:26:21 0:15:03 0:11:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_kernel} 3
pass 6655453 2022-02-02 15:33:04 2022-02-02 16:48:13 2022-02-02 17:20:39 0:32:26 0:20:45 0:11:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6655454 2022-02-02 15:33:05 2022-02-02 16:49:03 2022-02-02 17:25:52 0:36:49 0:23:20 0:13:29 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2022-02-02T17:23:35.548471+0000 mon.a (mon.0) 231 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655455 2022-02-02 15:33:06 2022-02-02 16:49:14 2022-02-02 17:33:17 0:44:03 0:31:54 0:12:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6655456 2022-02-02 15:33:07 2022-02-02 16:49:44 2022-02-02 17:18:09 0:28:25 0:15:37 0:12:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6655457 2022-02-02 15:33:08 2022-02-02 16:49:55 2022-02-02 17:33:50 0:43:55 0:31:47 0:12:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_simple_big} 3
pass 6655458 2022-02-02 15:33:09 2022-02-02 16:50:15 2022-02-02 17:13:15 0:23:00 0:12:08 0:10:52 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
pass 6655459 2022-02-02 15:33:10 2022-02-02 16:50:15 2022-02-02 17:31:50 0:41:35 0:29:45 0:11:50 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6655460 2022-02-02 15:33:11 2022-02-02 16:52:06 2022-02-02 17:41:47 0:49:41 0:37:04 0:12:37 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_fio} 3
pass 6655461 2022-02-02 15:33:12 2022-02-02 16:54:06 2022-02-02 17:35:13 0:41:07 0:27:03 0:14:04 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_data_pool} 3
pass 6655462 2022-02-02 15:33:14 2022-02-02 16:54:47 2022-02-02 17:21:38 0:26:51 0:13:37 0:13:14 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6655463 2022-02-02 15:33:15 2022-02-02 16:55:57 2022-02-02 17:32:47 0:36:50 0:25:33 0:11:17 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
pass 6655464 2022-02-02 15:33:16 2022-02-02 16:57:18 2022-02-02 17:24:11 0:26:53 0:14:25 0:12:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6655465 2022-02-02 15:33:17 2022-02-02 16:58:28 2022-02-02 17:35:53 0:37:25 0:26:40 0:10:45 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2022-02-02T17:33:23.190498+0000 mon.a (mon.0) 208 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655466 2022-02-02 15:33:18 2022-02-02 16:58:29 2022-02-02 17:30:44 0:32:15 0:21:23 0:10:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_fallocate} 3
fail 6655467 2022-02-02 15:33:19 2022-02-02 16:58:59 2022-02-02 17:38:49 0:39:50 0:26:15 0:13:35 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-02-02T17:36:44.830988+0000 mon.a (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655468 2022-02-02 15:33:20 2022-02-02 17:01:20 2022-02-02 17:53:18 0:51:58 0:40:39 0:11:19 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6655469 2022-02-02 15:33:21 2022-02-02 17:01:40 2022-02-02 17:37:37 0:35:57 0:20:13 0:15:44 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6655470 2022-02-02 15:33:22 2022-02-02 17:04:51 2022-02-02 17:26:28 0:21:37 0:11:44 0:09:53 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_discard} 1
pass 6655471 2022-02-02 15:33:23 2022-02-02 17:04:51 2022-02-02 17:44:19 0:39:28 0:24:29 0:14:59 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6655472 2022-02-02 15:33:24 2022-02-02 17:08:52 2022-02-02 18:52:48 1:43:56 1:32:05 0:11:51 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6655473 2022-02-02 15:33:25 2022-02-02 17:09:13 2022-02-02 17:39:26 0:30:13 0:19:23 0:10:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_namespaces} 3
pass 6655474 2022-02-02 15:33:26 2022-02-02 17:09:33 2022-02-02 18:49:05 1:39:32 1:26:26 0:13:06 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6655475 2022-02-02 15:33:27 2022-02-02 17:11:13 2022-02-02 18:05:17 0:54:04 0:43:12 0:10:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6655476 2022-02-02 15:33:28 2022-02-02 17:11:54 2022-02-02 17:46:32 0:34:38 0:23:10 0:11:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6655477 2022-02-02 15:33:29 2022-02-02 17:12:34 2022-02-02 17:45:16 0:32:42 0:19:37 0:13:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6655478 2022-02-02 15:33:30 2022-02-02 17:13:15 2022-02-02 17:44:26 0:31:11 0:18:54 0:12:17 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6655479 2022-02-02 15:33:32 2022-02-02 17:14:15 2022-02-02 17:35:19 0:21:04 0:11:44 0:09:20 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_huge_image} 1
pass 6655480 2022-02-02 15:33:32 2022-02-02 17:14:16 2022-02-02 17:53:55 0:39:39 0:29:55 0:09:44 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6655481 2022-02-02 15:33:34 2022-02-02 17:14:26 2022-02-02 17:45:23 0:30:57 0:18:27 0:12:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6655482 2022-02-02 15:33:35 2022-02-02 17:15:56 2022-02-02 17:44:49 0:28:53 0:17:13 0:11:40 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6655483 2022-02-02 15:33:36 2022-02-02 17:16:07 2022-02-02 17:54:40 0:38:33 0:25:17 0:13:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6655484 2022-02-02 15:33:37 2022-02-02 17:18:17 2022-02-02 17:57:27 0:39:10 0:26:56 0:12:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2022-02-02T17:55:31.502305+0000 mon.a (mon.0) 246 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655485 2022-02-02 15:33:38 2022-02-02 17:19:48 2022-02-02 17:58:24 0:38:36 0:26:37 0:11:59 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6655486 2022-02-02 15:33:39 2022-02-02 17:20:48 2022-02-02 17:51:08 0:30:20 0:18:11 0:12:09 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_image_read} 3
pass 6655487 2022-02-02 15:33:40 2022-02-02 17:20:49 2022-02-02 17:49:56 0:29:07 0:18:59 0:10:08 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
fail 6655488 2022-02-02 15:33:41 2022-02-02 17:20:49 2022-02-02 17:56:37 0:35:48 0:23:52 0:11:56 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2022-02-02T17:44:18.771481+0000 mon.a (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6655489 2022-02-02 15:33:42 2022-02-02 17:21:09 2022-02-02 17:59:16 0:38:07 0:26:21 0:11:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2022-02-02T17:57:41.912482+0000 mon.a (mon.0) 250 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655490 2022-02-02 15:33:43 2022-02-02 17:22:00 2022-02-02 17:57:45 0:35:45 0:21:27 0:14:18 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6655491 2022-02-02 15:33:44 2022-02-02 17:23:51 2022-02-02 17:50:11 0:26:20 0:14:21 0:11:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6655492 2022-02-02 15:33:45 2022-02-02 17:24:21 2022-02-02 17:59:06 0:34:45 0:22:06 0:12:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_map_unmap} 3
pass 6655493 2022-02-02 15:33:46 2022-02-02 17:25:31 2022-02-02 17:56:03 0:30:32 0:19:09 0:11:23 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6655494 2022-02-02 15:33:47 2022-02-02 17:26:02 2022-02-02 18:09:36 0:43:34 0:32:21 0:11:13 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6655495 2022-02-02 15:33:48 2022-02-02 17:26:12 2022-02-02 17:47:55 0:21:43 0:10:23 0:11:20 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_msgr_segments} 1
pass 6655496 2022-02-02 15:33:49 2022-02-02 17:26:13 2022-02-02 18:08:02 0:41:49 0:27:45 0:14:04 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6655497 2022-02-02 15:33:50 2022-02-02 17:27:23 2022-02-02 18:16:48 0:49:25 0:35:18 0:14:07 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_fio} 3
pass 6655498 2022-02-02 15:33:51 2022-02-02 17:30:54 2022-02-02 18:09:34 0:38:40 0:26:53 0:11:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_data_pool} 3
fail 6655499 2022-02-02 15:33:52 2022-02-02 17:31:54 2022-02-02 22:38:55 5:07:01 4:56:07 0:10:54 smithi master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc-rxbounce} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on smithi092 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude1emxtwjh -r -- '-g auto -g blockdev -x clone'"

pass 6655500 2022-02-02 15:33:54 2022-02-02 17:32:55 2022-02-02 19:23:04 1:50:09 1:38:04 0:12:05 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6655501 2022-02-02 15:33:55 2022-02-02 17:33:25 2022-02-02 18:03:28 0:30:03 0:16:44 0:13:19 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6655502 2022-02-02 15:33:56 2022-02-02 17:33:56 2022-02-02 18:13:38 0:39:42 0:25:10 0:14:32 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6655503 2022-02-02 15:33:57 2022-02-02 17:35:16 2022-02-02 18:10:54 0:35:38 0:23:53 0:11:45 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-02-02T18:09:22.119345+0000 mon.a (mon.0) 259 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655504 2022-02-02 15:33:58 2022-02-02 17:35:17 2022-02-02 18:01:15 0:25:58 0:13:42 0:12:16 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6655505 2022-02-02 15:33:59 2022-02-02 17:35:57 2022-02-02 18:33:19 0:57:22 0:43:56 0:13:26 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6655506 2022-02-02 15:34:00 2022-02-02 17:37:37 2022-02-02 18:10:24 0:32:47 0:19:26 0:13:21 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6655507 2022-02-02 15:34:01 2022-02-02 17:37:38 2022-02-02 18:00:38 0:23:00 0:12:19 0:10:41 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_parent_overlap} 1
pass 6655508 2022-02-02 15:34:02 2022-02-02 17:37:38 2022-02-02 18:12:23 0:34:45 0:22:22 0:12:23 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6655509 2022-02-02 15:34:03 2022-02-02 17:38:59 2022-02-02 18:17:22 0:38:23 0:26:22 0:12:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

"2022-02-02T18:15:21.104614+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655510 2022-02-02 15:34:04 2022-02-02 17:39:29 2022-02-02 19:13:33 1:34:04 1:21:01 0:13:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6655511 2022-02-02 15:34:05 2022-02-02 17:41:50 2022-02-02 18:41:56 1:00:06 0:45:44 0:14:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6655512 2022-02-02 15:34:06 2022-02-02 17:58:31 2022-02-02 18:26:47 0:28:16 0:16:51 0:11:25 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6655513 2022-02-02 15:34:07 2022-02-02 17:59:11 2022-02-02 18:26:43 0:27:32 0:17:00 0:10:32 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6655514 2022-02-02 15:34:08 2022-02-02 17:59:22 2022-02-02 18:26:06 0:26:44 0:13:36 0:13:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6655515 2022-02-02 15:34:09 2022-02-02 18:00:42 2022-02-02 18:24:51 0:24:09 0:11:55 0:12:14 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
fail 6655516 2022-02-02 15:34:10 2022-02-02 18:01:23 2022-02-02 18:37:49 0:36:26 0:23:07 0:13:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2022-02-02T18:35:43.322085+0000 mon.a (mon.0) 237 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655517 2022-02-02 15:34:11 2022-02-02 18:01:43 2022-02-02 18:29:11 0:27:28 0:14:51 0:12:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6655518 2022-02-02 15:34:13 2022-02-02 18:03:34 2022-02-02 18:28:14 0:24:40 0:13:04 0:11:36 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6655519 2022-02-02 15:34:14 2022-02-02 18:04:14 2022-02-02 18:36:48 0:32:34 0:20:43 0:11:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_concurrent} 3
pass 6655520 2022-02-02 15:34:15 2022-02-02 18:05:24 2022-02-02 18:34:50 0:29:26 0:13:47 0:15:39 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6655521 2022-02-02 15:34:16 2022-02-02 18:08:05 2022-02-02 18:43:14 0:35:09 0:22:09 0:13:00 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2022-02-02T18:41:12.535813+0000 mon.a (mon.0) 244 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6655522 2022-02-02 15:34:17 2022-02-02 18:09:36 2022-02-02 18:43:18 0:33:42 0:21:24 0:12:18 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-02-02T18:31:05.105684+0000 mon.a (mon.0) 140 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6655523 2022-02-02 15:34:18 2022-02-02 18:09:46 2022-02-02 18:34:01 0:24:15 0:11:02 0:13:13 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_whole_object_zeroout} 1
pass 6655524 2022-02-02 15:34:19 2022-02-02 18:10:26 2022-02-02 18:38:22 0:27:56 0:17:05 0:10:51 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6655525 2022-02-02 15:34:20 2022-02-02 18:10:57 2022-02-02 19:02:01 0:51:04 0:39:19 0:11:45 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6655526 2022-02-02 15:34:21 2022-02-02 18:12:27 2022-02-02 18:44:27 0:32:00 0:21:52 0:10:08 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_kernel} 3
pass 6655527 2022-02-02 15:34:22 2022-02-02 18:12:48 2022-02-02 18:42:11 0:29:23 0:16:43 0:12:40 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6655528 2022-02-02 15:34:23 2022-02-02 18:13:48 2022-02-02 18:38:36 0:24:48 0:10:57 0:13:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6655529 2022-02-02 15:34:24 2022-02-02 18:16:49 2022-02-02 18:45:13 0:28:24 0:15:56 0:12:28 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6655530 2022-02-02 15:34:25 2022-02-02 18:17:29 2022-02-02 18:42:59 0:25:30 0:13:14 0:12:16 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6655531 2022-02-02 15:34:26 2022-02-02 18:19:20 2022-02-02 18:58:31 0:39:11 0:24:28 0:14:43 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_simple_big} 3
pass 6655532 2022-02-02 15:34:27 2022-02-02 18:21:41 2022-02-02 18:44:26 0:22:45 0:10:31 0:12:14 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_deep_flatten} 1
pass 6655533 2022-02-02 15:34:28 2022-02-02 18:21:41 2022-02-02 18:59:28 0:37:47 0:24:51 0:12:56 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc-rxbounce} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6655534 2022-02-02 15:34:29 2022-02-02 18:23:42 2022-02-02 19:04:05 0:40:23 0:29:09 0:11:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6655535 2022-02-02 15:34:30 2022-02-02 18:24:22 2022-02-02 19:06:46 0:42:24 0:30:37 0:11:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6655536 2022-02-02 15:34:31 2022-02-02 18:24:32 2022-02-02 18:46:43 0:22:11 0:09:58 0:12:13 smithi master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6655537 2022-02-02 15:34:32 2022-02-02 18:24:43 2022-02-02 18:59:56 0:35:13 0:22:12 0:13:01 smithi master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6655538 2022-02-02 15:34:33 2022-02-02 18:26:03 2022-02-02 18:49:40 0:23:37 0:12:48 0:10:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6655539 2022-02-02 15:34:34 2022-02-02 18:26:14 2022-02-02 18:58:14 0:32:00 0:20:55 0:11:05 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6655540 2022-02-02 15:34:35 2022-02-02 18:26:44 2022-02-02 18:50:32 0:23:48 0:12:37 0:11:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_fallocate} 3
fail 6655541 2022-02-02 15:34:36 2022-02-02 18:26:54 2022-02-02 18:59:00 0:32:06 0:20:14 0:11:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2022-02-02T18:56:55.377897+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655542 2022-02-02 15:34:37 2022-02-02 18:27:15 2022-02-02 19:26:25 0:59:10 0:45:43 0:13:27 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6655543 2022-02-02 15:34:38 2022-02-02 18:28:15 2022-02-02 18:54:03 0:25:48 0:14:12 0:11:36 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6655544 2022-02-02 15:34:39 2022-02-02 18:28:26 2022-02-02 18:50:39 0:22:13 0:10:34 0:11:39 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6655545 2022-02-02 15:34:40 2022-02-02 18:28:26 2022-02-02 19:00:22 0:31:56 0:20:17 0:11:39 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6655546 2022-02-02 15:34:41 2022-02-02 18:28:36 2022-02-02 19:01:01 0:32:25 0:20:15 0:12:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2022-02-02T18:59:22.494887+0000 mon.a (mon.0) 293 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6655547 2022-02-02 15:34:42 2022-02-02 18:28:57 2022-02-02 19:00:54 0:31:57 0:20:13 0:11:44 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2022-02-02T18:59:28.934654+0000 mon.a (mon.0) 285 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655548 2022-02-02 15:34:43 2022-02-02 18:29:17 2022-02-02 19:24:30 0:55:13 0:38:23 0:16:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6655549 2022-02-02 15:34:44 2022-02-02 18:32:48 2022-02-02 19:07:29 0:34:41 0:20:04 0:14:37 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2022-02-02T19:06:01.364965+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655550 2022-02-02 15:34:45 2022-02-02 18:32:58 2022-02-02 18:58:32 0:25:34 0:12:46 0:12:48 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6655551 2022-02-02 15:34:46 2022-02-02 18:33:08 2022-02-02 18:59:09 0:26:01 0:12:09 0:13:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/few tasks/krbd_udev_netns} 3
pass 6655552 2022-02-02 15:34:47 2022-02-02 18:33:29 2022-02-02 18:56:59 0:23:30 0:10:12 0:13:18 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc-rxbounce} tasks/krbd_huge_image} 1
pass 6655553 2022-02-02 15:34:48 2022-02-02 18:33:29 2022-02-02 19:11:50 0:38:21 0:21:42 0:16:39 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy-rxbounce} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6655554 2022-02-02 15:34:49 2022-02-02 20:09:29 4830 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy-rxbounce} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6655555 2022-02-02 15:34:50 2022-02-02 18:35:30 2022-02-02 19:10:17 0:34:47 0:20:01 0:14:46 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2022-02-02T19:08:11.297592+0000 mon.a (mon.0) 205 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6655556 2022-02-02 15:34:51 2022-02-02 18:36:51 2022-02-02 19:03:51 0:27:00 0:12:04 0:14:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6655557 2022-02-02 15:34:52 2022-02-02 18:37:51 2022-02-02 19:08:10 0:30:19 0:16:45 0:13:34 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_concurrent} 3
pass 6655558 2022-02-02 15:34:53 2022-02-02 18:38:32 2022-02-02 19:02:35 0:24:03 0:11:37 0:12:26 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6655559 2022-02-02 15:34:54 2022-02-02 18:38:42 2022-02-02 19:13:38 0:34:56 0:19:22 0:15:34 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6655560 2022-02-02 15:34:55 2022-02-02 18:42:03 2022-02-02 19:06:32 0:24:29 0:11:36 0:12:53 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6655561 2022-02-02 15:34:57 2022-02-02 18:42:13 2022-02-02 19:03:36 0:21:23 0:09:52 0:11:31 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy} tasks/krbd_modprobe} 1
pass 6655562 2022-02-02 15:34:58 2022-02-02 18:42:14 2022-02-02 19:16:45 0:34:31 0:20:27 0:14:04 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6655563 2022-02-02 15:34:58 2022-02-02 18:43:05 2022-02-02 19:06:47 0:23:42 0:11:12 0:12:30 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_kernel} 3
fail 6655564 2022-02-02 15:34:59 2022-02-02 18:43:25 2022-02-02 19:16:53 0:33:28 0:20:28 0:13:00 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2022-02-02T19:14:09.192052+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655565 2022-02-02 15:35:01 2022-02-02 18:43:26 2022-02-02 19:08:15 0:24:49 0:10:48 0:14:01 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6655566 2022-02-02 15:35:02 2022-02-02 18:44:36 2022-02-02 19:17:10 0:32:34 0:19:43 0:12:51 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2022-02-02T19:15:07.914214+0000 mon.a (mon.0) 311 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655567 2022-02-02 15:35:03 2022-02-02 18:45:16 2022-02-02 19:13:32 0:28:16 0:12:18 0:15:58 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6655568 2022-02-02 15:35:04 2022-02-02 18:46:47 2022-02-02 19:24:50 0:38:03 0:23:41 0:14:22 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_simple_big} 3
pass 6655569 2022-02-02 15:35:05 2022-02-02 18:49:08 2022-02-02 19:11:19 0:22:11 0:10:30 0:11:41 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6655570 2022-02-02 15:35:06 2022-02-02 18:49:08 2022-02-02 19:29:13 0:40:05 0:27:18 0:12:47 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6655571 2022-02-02 15:35:07 2022-02-02 18:49:48 2022-02-02 19:30:10 0:40:22 0:29:19 0:11:03 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_fio} 3
fail 6655572 2022-02-02 15:35:08 2022-02-02 18:50:39 2022-02-02 19:24:21 0:33:42 0:19:42 0:14:00 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2022-02-02T19:23:06.660565+0000 mon.a (mon.0) 261 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6655573 2022-02-02 15:35:09 2022-02-02 18:52:49 2022-02-02 19:17:53 0:25:04 0:12:06 0:12:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6655574 2022-02-02 15:35:10 2022-02-02 18:54:10 2022-02-02 19:26:01 0:31:51 0:19:13 0:12:38 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6655575 2022-02-02 15:35:11 2022-02-02 18:55:50 2022-02-02 19:29:51 0:34:01 0:20:12 0:13:49 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2022-02-02T19:18:03.328602+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655576 2022-02-02 15:35:12 2022-02-02 18:58:21 2022-02-02 19:22:09 0:23:48 0:12:06 0:11:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6655577 2022-02-02 15:35:13 2022-02-02 18:58:42 2022-02-02 19:33:31 0:34:49 0:20:35 0:14:14 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2022-02-02T19:20:27.550678+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655578 2022-02-02 15:35:14 2022-02-02 18:58:42 2022-02-02 19:26:35 0:27:53 0:14:43 0:13:10 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6655579 2022-02-02 15:35:15 2022-02-02 18:59:02 2022-02-02 19:20:37 0:21:35 0:09:29 0:12:06 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc$/{crc} tasks/krbd_parent_overlap} 1
pass 6655580 2022-02-02 15:35:16 2022-02-02 18:59:03 2022-02-02 19:28:54 0:29:51 0:16:35 0:13:16 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6655581 2022-02-02 15:35:17 2022-02-02 18:59:13 2022-02-02 19:49:17 0:50:04 0:37:46 0:12:18 smithi master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6655582 2022-02-02 15:35:18 2022-02-02 18:59:33 2022-02-02 19:24:01 0:24:28 0:11:55 0:12:33 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/krbd_namespaces} 3
pass 6655583 2022-02-02 15:35:19 2022-02-02 19:00:04 2022-02-02 20:24:24 1:24:20 1:12:41 0:11:39 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6655584 2022-02-02 15:35:20 2022-02-02 19:00:24 2022-02-02 19:55:04 0:54:40 0:40:50 0:13:50 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6655585 2022-02-02 15:35:21 2022-02-02 19:01:05 2022-02-02 19:28:33 0:27:28 0:16:17 0:11:11 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6655586 2022-02-02 15:35:22 2022-02-02 19:01:05 2022-02-02 19:33:36 0:32:31 0:20:10 0:12:21 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2022-02-02T19:21:15.656407+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6655587 2022-02-02 15:35:23 2022-02-02 19:02:05 2022-02-02 19:26:22 0:24:17 0:12:02 0:12:15 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/krbd_udev_netns} 3
pass 6655588 2022-02-02 15:35:24 2022-02-02 19:02:36 2022-02-02 19:23:13 0:20:37 0:09:53 0:10:44 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy$/{legacy-rxbounce} tasks/krbd_read_only} 1
pass 6655589 2022-02-02 15:35:25 2022-02-02 19:03:46 2022-02-02 19:38:05 0:34:19 0:23:15 0:11:04 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6655590 2022-02-02 15:35:26 2022-02-02 19:03:57 2022-02-02 19:25:44 0:21:47 0:11:49 0:09:58 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6655591 2022-02-02 15:35:27 2022-02-02 19:04:07 2022-02-02 19:30:59 0:26:52 0:11:28 0:15:24 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6655592 2022-02-02 15:35:28 2022-02-02 19:06:38 2022-02-02 19:36:20 0:29:42 0:19:46 0:09:56 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2022-02-02T19:35:04.206644+0000 mon.a (mon.0) 256 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655593 2022-02-02 15:35:29 2022-02-02 19:06:48 2022-02-02 19:31:10 0:24:22 0:11:25 0:12:57 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6655594 2022-02-02 15:35:30 2022-02-02 19:06:48 2022-02-02 19:42:02 0:35:14 0:21:23 0:13:51 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6655595 2022-02-02 15:35:31 2022-02-02 19:07:39 2022-02-02 19:39:56 0:32:17 0:20:03 0:12:14 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy} msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2022-02-02T19:38:07.406730+0000 mon.a (mon.0) 275 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6655596 2022-02-02 15:35:33 2022-02-02 19:08:19 2022-02-02 19:29:53 0:21:34 0:10:28 0:11:06 smithi master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6655597 2022-02-02 15:35:34 2022-02-02 19:08:20 2022-02-02 19:38:05 0:29:45 0:17:26 0:12:19 smithi master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6655598 2022-02-02 15:35:35 2022-02-02 19:08:20 2022-02-02 19:31:36 0:23:16 0:10:34 0:12:42 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6655599 2022-02-02 15:35:36 2022-02-02 19:10:20 2022-02-02 19:40:22 0:30:02 0:17:06 0:12:56 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6655600 2022-02-02 15:35:37 2022-02-02 19:11:51 2022-02-02 19:35:15 0:23:24 0:11:00 0:12:24 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6655601 2022-02-02 15:35:38 2022-02-02 19:13:42 2022-02-02 19:39:32 0:25:50 0:15:47 0:10:03 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/few tasks/rbd_map_unmap} 3
pass 6655602 2022-02-02 15:35:39 2022-02-02 19:13:42 2022-02-02 19:38:32 0:24:50 0:12:51 0:11:59 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6655603 2022-02-02 15:35:40 2022-02-02 19:13:42 2022-02-02 19:58:37 0:44:55 0:30:03 0:14:52 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3