Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6512512 2021-11-19 03:20:57 2021-11-21 08:10:07 2021-11-21 08:31:00 0:20:53 0:07:25 0:13:28 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} 1
pass 6512513 2021-11-19 03:20:58 2021-11-21 08:13:08 2021-11-21 08:46:42 0:33:34 0:20:15 0:13:19 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6512514 2021-11-19 03:20:59 2021-11-21 08:14:48 2021-11-21 08:51:42 0:36:54 0:24:32 0:12:22 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} 3
pass 6512515 2021-11-19 03:20:59 2021-11-21 08:15:49 2021-11-21 08:43:23 0:27:34 0:15:12 0:12:22 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} 3
fail 6512516 2021-11-19 03:21:00 2021-11-21 08:16:23 2021-11-21 12:14:38 3:58:15 3:39:26 0:18:49 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on gibba016 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeww6it24p -r -- '-g auto -g blockdev -x clone'"

pass 6512517 2021-11-19 03:21:01 2021-11-21 08:16:30 2021-11-21 09:34:34 1:18:04 0:39:45 0:38:19 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6512518 2021-11-19 03:21:02 2021-11-21 08:17:29 2021-11-21 08:25:42 0:08:13 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6512519 2021-11-19 03:21:03 2021-11-21 08:17:50 2021-11-21 08:36:03 0:18:13 0:07:17 0:10:56 gibba master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6512520 2021-11-19 03:21:04 2021-11-21 08:18:10 2021-11-21 08:48:25 0:30:15 0:09:19 0:20:56 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6512521 2021-11-19 03:21:04 2021-11-21 08:18:25 2021-11-21 08:46:26 0:28:01 0:17:16 0:10:45 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-11-21T08:45:08.957389+0000 mon.b (mon.0) 160 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512522 2021-11-19 03:21:05 2021-11-21 08:25:23 2021-11-21 08:45:23 0:20:00 0:09:07 0:10:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
fail 6512523 2021-11-19 03:21:06 2021-11-21 08:28:32 2021-11-21 08:58:56 0:30:24 0:17:03 0:13:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2021-11-21T08:57:12.510671+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512524 2021-11-19 03:21:07 2021-11-21 08:31:02 2021-11-21 09:30:32 0:59:30 0:35:44 0:23:46 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6512525 2021-11-19 03:21:08 2021-11-21 08:44:34 2021-11-21 09:07:58 0:23:24 0:11:43 0:11:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6512526 2021-11-19 03:21:08 2021-11-21 08:45:24 2021-11-21 09:03:24 0:18:00 0:07:23 0:10:37 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} 1
pass 6512527 2021-11-19 03:21:09 2021-11-21 08:46:15 2021-11-21 09:12:44 0:26:29 0:15:50 0:10:39 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6512528 2021-11-19 03:21:10 2021-11-21 08:46:28 2021-11-21 09:56:31 1:10:03 0:17:03 0:53:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2021-11-21T09:54:12.099400+0000 mon.b (mon.0) 244 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6512529 2021-11-19 03:21:11 2021-11-21 08:46:45 2021-11-21 10:06:24 1:19:39 1:07:32 0:12:07 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6512530 2021-11-19 03:21:12 2021-11-21 08:48:26 2021-11-21 09:21:45 0:33:19 0:19:36 0:13:43 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6512531 2021-11-19 03:21:13 2021-11-21 08:51:46 2021-11-21 09:15:41 0:23:55 0:12:34 0:11:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6512532 2021-11-19 03:21:13 2021-11-21 08:51:47 2021-11-21 09:20:39 0:28:52 0:11:12 0:17:40 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6512533 2021-11-19 03:21:14 2021-11-21 08:58:58 2021-11-21 09:27:57 0:28:59 0:08:53 0:20:06 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6512534 2021-11-19 03:21:15 2021-11-21 09:03:25 2021-11-21 09:21:25 0:18:00 0:06:53 0:11:07 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
pass 6512535 2021-11-19 03:21:16 2021-11-21 09:08:00 2021-11-21 09:46:44 0:38:44 0:23:41 0:15:03 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6512536 2021-11-19 03:21:17 2021-11-21 09:12:51 2021-11-21 09:43:39 0:30:48 0:17:02 0:13:46 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2021-11-21T09:41:50.556690+0000 mon.a (mon.0) 299 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6512537 2021-11-19 03:21:18 2021-11-21 09:15:41 2021-11-21 09:48:40 0:32:59 0:17:05 0:15:54 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-11-21T09:46:54.311039+0000 mon.a (mon.0) 241 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512538 2021-11-19 03:21:18 2021-11-21 09:20:42 2021-11-21 09:54:31 0:33:49 0:13:06 0:20:43 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} 3
pass 6512539 2021-11-19 03:21:19 2021-11-21 09:21:27 2021-11-21 09:41:27 0:20:00 0:07:51 0:12:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6512540 2021-11-19 03:21:20 2021-11-21 09:30:34 2021-11-21 10:04:52 0:34:18 0:20:42 0:13:36 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6512541 2021-11-19 03:21:21 2021-11-21 09:32:55 2021-11-21 09:59:33 0:26:38 0:08:37 0:18:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} 3
pass 6512542 2021-11-19 03:21:22 2021-11-21 09:34:35 2021-11-21 09:52:35 0:18:00 0:06:47 0:11:13 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_modprobe} 1
pass 6512543 2021-11-19 03:21:22 2021-11-21 09:39:36 2021-11-21 10:11:28 0:31:52 0:16:42 0:15:10 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6512544 2021-11-19 03:21:23 2021-11-21 09:41:28 2021-11-21 11:21:34 1:40:06 0:59:21 0:40:45 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6512545 2021-11-19 03:21:24 2021-11-21 09:43:47 2021-11-21 10:06:41 0:22:54 0:08:14 0:14:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} 3
pass 6512546 2021-11-19 03:21:25 2021-11-21 09:46:47 2021-11-21 10:16:33 0:29:46 0:18:05 0:11:41 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6512547 2021-11-19 03:21:26 2021-11-21 09:48:48 2021-11-21 10:08:52 0:20:04 0:08:02 0:12:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6512548 2021-11-19 03:21:26 2021-11-21 09:50:59 2021-11-21 10:18:43 0:27:44 0:13:01 0:14:43 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} 3
pass 6512549 2021-11-19 03:21:27 2021-11-21 09:52:36 2021-11-21 10:16:37 0:24:01 0:10:30 0:13:31 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6512550 2021-11-19 03:21:28 2021-11-21 09:55:10 2021-11-21 10:35:22 0:40:12 0:25:24 0:14:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} 3
pass 6512551 2021-11-19 03:21:29 2021-11-21 09:56:32 2021-11-21 10:22:33 0:26:01 0:06:47 0:19:14 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} 1
pass 6512552 2021-11-19 03:21:30 2021-11-21 09:59:40 2021-11-21 10:40:30 0:40:50 0:23:13 0:17:37 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6512553 2021-11-19 03:21:30 2021-11-21 10:06:31 2021-11-21 10:42:40 0:36:09 0:24:50 0:11:19 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
fail 6512554 2021-11-19 03:21:31 2021-11-21 10:06:42 2021-11-21 10:36:46 0:30:04 0:17:10 0:12:54 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2021-11-21T10:25:07.318116+0000 mon.a (mon.0) 119 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512555 2021-11-19 03:21:32 2021-11-21 10:08:52 2021-11-21 10:34:51 0:25:59 0:08:48 0:17:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6512556 2021-11-19 03:21:33 2021-11-21 10:14:53 2021-11-21 10:40:32 0:25:39 0:13:52 0:11:47 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6512557 2021-11-19 03:21:34 2021-11-21 10:16:34 2021-11-21 10:38:37 0:22:03 0:09:15 0:12:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} 3
pass 6512558 2021-11-19 03:21:34 2021-11-21 10:16:38 2021-11-21 11:00:39 0:44:01 0:09:04 0:34:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6512559 2021-11-19 03:21:35 2021-11-21 10:18:45 2021-11-21 11:09:13 0:50:28 0:37:15 0:13:13 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
pass 6512560 2021-11-19 03:21:36 2021-11-21 10:21:15 2021-11-21 10:57:21 0:36:06 0:11:49 0:24:17 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6512561 2021-11-19 03:21:37 2021-11-21 10:22:34 2021-11-21 10:52:35 0:30:01 0:07:09 0:22:52 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
fail 6512562 2021-11-19 03:21:38 2021-11-21 10:35:27 2021-11-21 11:04:33 0:29:06 0:16:01 0:13:05 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2021-11-21T10:52:24.348814+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512563 2021-11-19 03:21:39 2021-11-21 10:36:48 2021-11-21 11:00:36 0:23:48 0:09:09 0:14:39 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} 3
pass 6512564 2021-11-19 03:21:39 2021-11-21 10:38:38 2021-11-21 11:52:37 1:13:59 1:00:46 0:13:13 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6512565 2021-11-19 03:21:40 2021-11-21 10:40:39 2021-11-21 11:10:35 0:29:56 0:16:42 0:13:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2021-11-21T10:58:38.568527+0000 mon.a (mon.0) 129 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512566 2021-11-19 03:21:41 2021-11-21 10:42:49 2021-11-21 11:15:38 0:32:49 0:12:09 0:20:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6512567 2021-11-19 03:21:42 2021-11-21 10:51:41 2021-11-21 11:21:15 0:29:34 0:12:02 0:17:32 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
fail 6512568 2021-11-19 03:21:43 2021-11-21 10:52:36 2021-11-21 11:26:37 0:34:01 0:16:52 0:17:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

"2021-11-21T11:15:32.259145+0000 mon.a (mon.0) 154 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512569 2021-11-19 03:21:44 2021-11-21 10:57:22 2021-11-21 11:22:23 0:25:01 0:07:26 0:17:35 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} 1
pass 6512570 2021-11-19 03:21:44 2021-11-21 11:00:40 2021-11-21 11:28:41 0:28:01 0:18:10 0:09:51 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6512571 2021-11-19 03:21:45 2021-11-21 11:04:43 2021-11-21 11:59:16 0:54:33 0:38:44 0:15:49 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6512572 2021-11-19 03:21:46 2021-11-21 11:09:14 2021-11-21 11:30:29 0:21:15 0:09:06 0:12:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6512573 2021-11-19 03:21:47 2021-11-21 11:10:44 2021-11-21 11:35:43 0:24:59 0:08:23 0:16:36 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6512574 2021-11-19 03:21:48 2021-11-21 11:15:45 2021-11-21 11:45:09 0:29:24 0:13:05 0:16:19 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} 3
pass 6512575 2021-11-19 03:21:49 2021-11-21 11:20:36 2021-11-21 11:41:15 0:20:39 0:08:28 0:12:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6512576 2021-11-19 03:21:49 2021-11-21 11:21:17 2021-11-21 11:54:20 0:33:03 0:17:09 0:15:54 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2021-11-21T11:52:30.170488+0000 mon.a (mon.0) 211 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512577 2021-11-19 03:21:50 2021-11-21 11:21:35 2021-11-21 11:41:35 0:20:00 0:09:17 0:10:43 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6512578 2021-11-19 03:21:51 2021-11-21 11:26:38 2021-11-21 11:46:38 0:20:00 0:07:07 0:12:53 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} 1
pass 6512579 2021-11-19 03:21:52 2021-11-21 11:26:38 2021-11-21 12:03:47 0:37:09 0:18:07 0:19:02 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6512580 2021-11-19 03:21:53 2021-11-21 11:28:42 2021-11-21 11:48:42 0:20:00 0:07:49 0:12:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} 3
pass 6512581 2021-11-19 03:21:53 2021-11-21 11:35:50 2021-11-21 12:09:18 0:33:28 0:16:18 0:17:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6512582 2021-11-19 03:21:54 2021-11-21 11:41:21 2021-11-21 12:01:18 0:19:57 0:08:19 0:11:38 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6512583 2021-11-19 03:21:55 2021-11-21 11:41:21 2021-11-21 12:09:09 0:27:48 0:12:59 0:14:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
pass 6512584 2021-11-19 03:21:56 2021-11-21 11:41:36 2021-11-21 12:11:37 0:30:01 0:12:13 0:17:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6512585 2021-11-19 03:21:57 2021-11-21 11:45:12 2021-11-21 12:23:41 0:38:29 0:27:54 0:10:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} 3
pass 6512586 2021-11-19 03:21:58 2021-11-21 11:45:42 2021-11-21 12:12:03 0:26:21 0:07:12 0:19:09 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
fail 6512587 2021-11-19 03:21:58 2021-11-21 11:46:39 2021-11-21 12:20:40 0:34:01 0:16:04 0:17:57 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6512588 2021-11-19 03:21:59 2021-11-21 11:48:43 2021-11-21 12:38:46 0:50:03 0:24:12 0:25:51 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} 3
pass 6512589 2021-11-19 03:22:00 2021-11-21 11:54:23 2021-11-21 12:25:23 0:31:00 0:15:33 0:15:27 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} 3
pass 6512590 2021-11-19 03:22:01 2021-11-21 11:59:24 2021-11-21 12:16:57 0:17:33 0:07:24 0:10:09 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
pass 6512591 2021-11-19 03:22:02 2021-11-21 11:59:25 2021-11-21 12:31:10 0:31:45 0:18:52 0:12:53 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
pass 6512592 2021-11-19 03:22:03 2021-11-21 12:01:25 2021-11-21 12:29:14 0:27:49 0:09:04 0:18:45 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6512593 2021-11-19 03:22:03 2021-11-21 12:09:17 2021-11-21 12:37:26 0:28:09 0:17:13 0:10:56 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-11-21T12:25:37.044492+0000 mon.a (mon.0) 128 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512594 2021-11-19 03:22:04 2021-11-21 12:09:27 2021-11-21 12:30:33 0:21:06 0:08:54 0:12:12 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} 3
fail 6512595 2021-11-19 03:22:05 2021-11-21 12:10:48 2021-11-21 12:41:20 0:30:32 0:17:06 0:13:26 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2021-11-21T12:29:14.734767+0000 mon.a (mon.0) 111 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512596 2021-11-19 03:22:06 2021-11-21 12:11:38 2021-11-21 13:11:41 1:00:03 0:35:35 0:24:28 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6512597 2021-11-19 03:22:07 2021-11-21 12:13:38 2021-11-21 12:38:52 0:25:14 0:11:10 0:14:04 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6512598 2021-11-19 03:22:07 2021-11-21 12:14:39 2021-11-21 12:32:39 0:18:00 0:07:46 0:10:14 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} 1
pass 6512599 2021-11-19 03:22:08 2021-11-21 12:16:59 2021-11-21 12:46:35 0:29:36 0:16:37 0:12:59 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6512600 2021-11-19 03:22:09 2021-11-21 12:18:50 2021-11-21 13:41:22 1:22:32 1:00:40 0:21:52 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6512601 2021-11-19 03:22:10 2021-11-21 12:20:41 2021-11-21 12:44:42 0:24:01 0:09:32 0:14:29 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} 3
fail 6512602 2021-11-19 03:22:11 2021-11-21 12:29:21 2021-11-21 12:58:23 0:29:02 0:17:10 0:11:52 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-11-21T12:56:26.843778+0000 mon.a (mon.0) 231 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512603 2021-11-19 03:22:11 2021-11-21 12:30:42 2021-11-21 13:01:24 0:30:42 0:19:29 0:11:13 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6512604 2021-11-19 03:22:12 2021-11-21 12:31:12 2021-11-21 13:05:51 0:34:39 0:17:03 0:17:36 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2021-11-21T13:04:59.844311+0000 mon.a (mon.0) 318 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512605 2021-11-19 03:22:13 2021-11-21 12:32:40 2021-11-21 13:00:41 0:28:01 0:11:08 0:16:53 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6512606 2021-11-19 03:22:14 2021-11-21 12:37:53 2021-11-21 12:58:51 0:20:58 0:09:11 0:11:47 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} 3
pass 6512607 2021-11-19 03:22:15 2021-11-21 12:38:47 2021-11-21 12:56:47 0:18:00 0:07:12 0:10:48 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} 1
fail 6512608 2021-11-19 03:22:16 2021-11-21 12:38:54 2021-11-21 13:27:11 0:48:17 0:35:24 0:12:53 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
Failure Reason:

"2021-11-21T12:57:04.184467+0000 mon.a (mon.0) 114 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512609 2021-11-19 03:22:16 2021-11-21 12:41:24 2021-11-21 13:03:02 0:21:38 0:09:31 0:12:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6512610 2021-11-19 03:22:17 2021-11-21 12:43:05 2021-11-21 13:18:12 0:35:07 0:08:49 0:26:18 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6512611 2021-11-19 03:22:18 2021-11-21 12:44:43 2021-11-21 13:10:44 0:26:01 0:12:35 0:13:26 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6512612 2021-11-19 03:22:19 2021-11-21 12:56:48 2021-11-21 13:38:50 0:42:02 0:17:09 0:24:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2021-11-21T13:36:01.427096+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512613 2021-11-19 03:22:20 2021-11-21 12:58:27 2021-11-21 13:30:57 0:32:30 0:20:45 0:11:45 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6512614 2021-11-19 03:22:20 2021-11-21 12:58:58 2021-11-21 13:19:05 0:20:07 0:08:56 0:11:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} 3
pass 6512615 2021-11-19 03:22:21 2021-11-21 12:59:08 2021-11-21 13:16:53 0:17:45 0:06:44 0:11:01 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6512616 2021-11-19 03:22:22 2021-11-21 12:59:08 2021-11-21 13:31:27 0:32:19 0:19:23 0:12:56 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6512617 2021-11-19 03:22:23 2021-11-21 13:00:42 2021-11-21 13:30:43 0:30:01 0:16:51 0:13:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2021-11-21T13:19:10.221420+0000 mon.b (mon.0) 139 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512618 2021-11-19 03:22:24 2021-11-21 13:01:29 2021-11-21 13:35:45 0:34:16 0:19:13 0:15:03 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6512619 2021-11-19 03:22:24 2021-11-21 13:06:00 2021-11-21 13:37:08 0:31:08 0:08:01 0:23:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6512620 2021-11-19 03:22:25 2021-11-21 13:10:45 2021-11-21 13:34:45 0:24:00 0:13:20 0:10:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} 3
pass 6512621 2021-11-19 03:22:26 2021-11-21 13:11:42 2021-11-21 13:39:43 0:28:01 0:10:35 0:17:26 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6512622 2021-11-19 03:22:27 2021-11-21 13:19:12 2021-11-21 14:06:47 0:47:35 0:28:21 0:19:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6512623 2021-11-19 03:22:28 2021-11-21 13:27:13 2021-11-21 13:44:54 0:17:41 0:07:07 0:10:34 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} 1
pass 6512624 2021-11-19 03:22:28 2021-11-21 13:27:13 2021-11-21 14:08:00 0:40:47 0:27:52 0:12:55 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
fail 6512625 2021-11-19 03:22:29 2021-11-21 13:30:14 2021-11-21 13:58:57 0:28:43 0:17:27 0:11:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2021-11-21T13:57:24.718606+0000 mon.a (mon.0) 204 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512626 2021-11-19 03:22:30 2021-11-21 13:30:44 2021-11-21 14:00:45 0:30:01 0:18:12 0:11:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} 3
fail 6512627 2021-11-19 03:22:31 2021-11-21 13:31:04 2021-11-21 17:53:09 4:22:05 4:04:33 0:17:32 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on gibba039 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude5o1uf989 -r -- '-g auto -g blockdev -x clone'"

pass 6512628 2021-11-19 03:22:32 2021-11-21 13:34:47 2021-11-21 14:44:50 1:10:03 0:59:59 0:10:04 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6512629 2021-11-19 03:22:32 2021-11-21 13:36:55 2021-11-21 13:59:13 0:22:18 0:09:25 0:12:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6512630 2021-11-19 03:22:33 2021-11-21 13:37:16 2021-11-21 14:04:51 0:27:35 0:14:25 0:13:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6512631 2021-11-19 03:22:34 2021-11-21 13:38:51 2021-11-21 14:20:52 0:42:01 0:09:34 0:32:27 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
pass 6512632 2021-11-19 03:22:35 2021-11-21 13:38:56 2021-11-21 14:04:55 0:25:59 0:09:01 0:16:58 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6512633 2021-11-19 03:22:36 2021-11-21 13:39:44 2021-11-21 14:09:45 0:30:01 0:16:57 0:13:04 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-11-21T13:58:13.403896+0000 mon.b (mon.0) 124 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6512634 2021-11-19 03:22:37 2021-11-21 13:44:57 2021-11-21 14:20:41 0:35:44 0:11:32 0:24:12 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6512635 2021-11-19 03:22:37 2021-11-21 13:58:59 2021-11-21 14:17:42 0:18:43 0:07:24 0:11:19 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} 1
pass 6512636 2021-11-19 03:22:38 2021-11-21 13:59:50 2021-11-21 14:30:58 0:31:08 0:14:33 0:16:35 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
dead 6512637 2021-11-19 03:22:39 2021-11-21 14:00:46 2021-11-21 14:18:47 0:18:01 0:03:16 0:14:45 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

Failure object was: {'gibba031.front.sepia.ceph.com': {'cache_updated': False, 'stdout': '', 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': True, 'force_apt_get': False, 'policy_rc_d': None, 'package': ['apt', 'apache2'], 'autoclean': False, 'install_recommends': None, 'name': ['apt', 'apache2'], 'purge': False, 'allow_unauthenticated': False, 'state': 'latest', 'upgrade': None, 'update_cache': None, 'default_release': None, 'only_upgrade': False, 'deb': None, 'cache_valid_time': 0}}, 'cache_update_time': 1637504177, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'apache2\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stdout_lines': [], 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 6439 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], '_ansible_no_log': False, 'changed': False}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping('tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)yaml.representer.RepresenterError: ('cannot represent an object', 'cache_update_time')

pass 6512638 2021-11-19 03:22:40 2021-11-21 14:05:01 2021-11-21 15:16:48 1:11:47 0:59:22 0:12:25 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6512639 2021-11-19 03:22:41 2021-11-21 14:06:51 2021-11-21 14:40:00 0:33:09 0:21:47 0:11:22 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6512640 2021-11-19 03:22:41 2021-11-21 14:08:02 2021-11-21 14:32:07 0:24:05 0:12:10 0:11:55 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6512641 2021-11-19 03:22:42 2021-11-21 14:08:22 2021-11-21 14:41:37 0:33:15 0:13:37 0:19:38 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6512642 2021-11-19 03:22:43 2021-11-21 14:09:46 2021-11-21 14:51:48 0:42:02 0:09:30 0:32:32 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6512643 2021-11-19 03:22:44 2021-11-21 14:17:44 2021-11-21 14:36:07 0:18:23 0:07:12 0:11:11 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6512644 2021-11-19 03:22:45 2021-11-21 14:18:34 2021-11-21 14:58:58 0:40:24 0:28:18 0:12:06 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6512645 2021-11-19 03:22:45 2021-11-21 14:18:48 2021-11-21 14:52:49 0:34:01 0:09:01 0:25:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6512646 2021-11-19 03:22:46 2021-11-21 14:19:05 2021-11-21 14:40:36 0:21:31 0:08:11 0:13:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6512647 2021-11-19 03:22:47 2021-11-21 14:20:45 2021-11-21 16:47:52 2:27:07 2:00:59 0:26:08 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2021-11-21T15:53:59.600910+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.9533.0:11518394 1.6 1:6a7654a4:::rbd_header.2f499650ec18:head [watch reconnect cookie 18446462598732841104 gen 130136] snapc 0=[] ondisk+write+known_if_redirected e19) initiated 2021-11-21T15:53:29.538823+0000 currently started" in cluster log

pass 6512648 2021-11-19 03:22:48 2021-11-21 14:20:54 2021-11-21 14:58:55 0:38:01 0:08:12 0:29:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6512649 2021-11-19 03:22:49 2021-11-21 14:36:07 2021-11-21 15:14:37 0:38:30 0:22:49 0:15:41 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6512650 2021-11-19 03:22:50 2021-11-21 14:40:38 2021-11-21 15:09:36 0:28:58 0:17:06 0:11:52 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2021-11-21T15:08:00.301497+0000 mon.a (mon.0) 337 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512651 2021-11-19 03:22:50 2021-11-21 14:41:39 2021-11-21 15:02:42 0:21:03 0:07:00 0:14:03 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} 1
fail 6512652 2021-11-19 03:22:51 2021-11-21 14:44:49 2021-11-21 15:16:37 0:31:48 0:13:32 0:18:16 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2021-11-21T15:07:23.199820+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512653 2021-11-19 03:22:52 2021-11-21 14:44:51 2021-11-21 16:06:55 1:22:04 0:38:23 0:43:41 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
fail 6512654 2021-11-19 03:22:53 2021-11-21 14:50:40 2021-11-21 15:19:15 0:28:35 0:17:14 0:11:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} 3
Failure Reason:

"2021-11-21T15:17:42.976556+0000 mon.a (mon.0) 209 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512655 2021-11-19 03:22:54 2021-11-21 14:51:21 2021-11-21 15:30:41 0:39:20 0:16:28 0:22:52 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6512656 2021-11-19 03:22:55 2021-11-21 14:51:49 2021-11-21 15:17:50 0:26:01 0:07:54 0:18:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6512657 2021-11-19 03:22:55 2021-11-21 14:52:50 2021-11-21 15:22:51 0:30:01 0:12:43 0:17:18 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} 3
fail 6512658 2021-11-19 03:22:56 2021-11-21 14:58:56 2021-11-21 15:36:57 0:38:01 0:16:54 0:21:07 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2021-11-21T15:25:33.462730+0000 mon.b (mon.0) 109 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6512659 2021-11-19 03:22:57 2021-11-21 15:02:43 2021-11-21 15:52:39 0:49:56 0:27:03 0:22:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} 3
pass 6512660 2021-11-19 03:22:58 2021-11-21 15:14:44 2021-11-21 15:34:30 0:19:46 0:07:23 0:12:23 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} 1
pass 6512661 2021-11-19 03:22:59 2021-11-21 15:16:45 2021-11-21 15:56:53 0:40:08 0:29:11 0:10:57 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6512662 2021-11-19 03:22:59 2021-11-21 15:16:55 2021-11-21 15:55:03 0:38:08 0:24:57 0:13:11 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6512663 2021-11-19 03:23:00 2021-11-21 15:17:51 2021-11-21 15:47:52 0:30:01 0:15:12 0:14:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6512664 2021-11-19 03:23:01 2021-11-21 15:19:16 2021-11-21 15:37:09 0:17:53 0:07:49 0:10:04 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6512665 2021-11-19 03:23:02 2021-11-21 15:19:16 2021-11-21 15:49:40 0:30:24 0:19:23 0:11:01 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6512666 2021-11-19 03:23:03 2021-11-21 15:20:07 2021-11-21 15:56:34 0:36:27 0:09:39 0:26:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6512667 2021-11-19 03:23:04 2021-11-21 15:22:52 2021-11-21 15:58:53 0:36:01 0:17:22 0:18:39 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-11-21T15:57:23.945322+0000 mon.b (mon.0) 171 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 6512668 2021-11-19 03:23:04 2021-11-21 15:36:39 2021-11-21 17:37:48 2:01:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

fail 6512669 2021-11-19 03:23:05 2021-11-21 15:36:58 2021-11-21 16:04:59 0:28:01 0:17:08 0:10:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

"2021-11-21T16:04:04.070556+0000 mon.b (mon.0) 176 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6512670 2021-11-19 03:23:06 2021-11-21 15:38:20 2021-11-21 16:17:40 0:39:20 0:17:06 0:22:14 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-11-21T16:15:12.007229+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512671 2021-11-19 03:23:07 2021-11-21 15:47:11 2021-11-21 16:13:40 0:26:29 0:11:36 0:14:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6512672 2021-11-19 03:23:08 2021-11-21 15:47:53 2021-11-21 16:13:54 0:26:01 0:07:43 0:18:18 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
pass 6512673 2021-11-19 03:23:08 2021-11-21 15:49:42 2021-11-21 16:18:41 0:28:59 0:14:34 0:14:25 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6512674 2021-11-19 03:23:09 2021-11-21 15:52:42 2021-11-21 16:16:37 0:23:55 0:09:38 0:14:17 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} 3
pass 6512675 2021-11-19 03:23:10 2021-11-21 15:56:43 2021-11-21 17:14:09 1:17:26 1:07:05 0:10:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
pass 6512676 2021-11-19 03:23:11 2021-11-21 15:56:54 2021-11-21 16:28:01 0:31:07 0:19:28 0:11:39 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6512677 2021-11-19 03:23:12 2021-11-21 15:58:14 2021-11-21 16:32:32 0:34:18 0:17:05 0:17:13 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2021-11-21T16:31:07.532398+0000 mon.a (mon.0) 278 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512678 2021-11-19 03:23:12 2021-11-21 15:58:55 2021-11-21 16:26:55 0:28:00 0:10:27 0:17:33 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6512679 2021-11-19 03:23:13 2021-11-21 16:04:45 2021-11-21 16:33:44 0:28:59 0:08:50 0:20:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} 3
pass 6512680 2021-11-19 03:23:14 2021-11-21 16:05:01 2021-11-21 16:21:01 0:16:00 0:06:57 0:09:03 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} 1
pass 6512681 2021-11-19 03:23:15 2021-11-21 16:06:56 2021-11-21 16:52:59 0:46:03 0:24:19 0:21:44 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6512682 2021-11-19 03:23:16 2021-11-21 16:13:47 2021-11-21 17:32:39 1:18:52 1:04:11 0:14:41 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6512683 2021-11-19 03:23:16 2021-11-21 16:13:55 2021-11-21 16:37:56 0:24:01 0:09:11 0:14:50 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6512684 2021-11-19 03:23:17 2021-11-21 16:16:38 2021-11-21 16:48:47 0:32:09 0:16:59 0:15:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-11-21T16:47:33.354119+0000 mon.a (mon.0) 230 : cluster [WRN] Health check failed: 4 osds down (OSD_DOWN)" in cluster log

pass 6512685 2021-11-19 03:23:18 2021-11-21 16:20:49 2021-11-21 16:49:27 0:28:38 0:12:33 0:16:05 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} 3
pass 6512686 2021-11-19 03:23:19 2021-11-21 16:21:02 2021-11-21 16:47:02 0:26:00 0:08:21 0:17:39 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6512687 2021-11-19 03:23:20 2021-11-21 16:25:30 2021-11-21 17:05:49 0:40:19 0:22:02 0:18:17 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6512688 2021-11-19 03:23:21 2021-11-21 16:26:57 2021-11-21 16:50:57 0:24:00 0:08:36 0:15:24 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6512689 2021-11-19 03:23:22 2021-11-21 16:33:51 2021-11-21 16:54:38 0:20:47 0:07:12 0:13:35 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_modprobe} 1
pass 6512690 2021-11-19 03:23:22 2021-11-21 16:36:42 2021-11-21 17:13:39 0:36:57 0:16:12 0:20:45 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6512691 2021-11-19 03:23:23 2021-11-21 16:37:57 2021-11-21 17:03:58 0:26:01 0:08:04 0:17:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} 3
pass 6512692 2021-11-19 03:23:24 2021-11-21 16:47:04 2021-11-21 17:17:04 0:30:00 0:17:27 0:12:33 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
pass 6512693 2021-11-19 03:23:25 2021-11-21 16:47:54 2021-11-21 17:07:32 0:19:38 0:08:06 0:11:32 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6512694 2021-11-19 03:23:26 2021-11-21 16:49:35 2021-11-21 17:14:27 0:24:52 0:13:04 0:11:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6512695 2021-11-19 03:23:27 2021-11-21 16:50:45 2021-11-21 17:14:40 0:23:55 0:10:31 0:13:24 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6512696 2021-11-19 03:23:27 2021-11-21 16:50:58 2021-11-21 17:41:01 0:50:03 0:25:01 0:25:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} 3
pass 6512697 2021-11-19 03:23:28 2021-11-21 16:52:46 2021-11-21 17:09:53 0:17:07 0:07:17 0:09:50 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
fail 6512698 2021-11-19 03:23:29 2021-11-21 16:52:46 2021-11-21 17:35:36 0:42:50 0:16:12 0:26:38 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

fail 6512699 2021-11-19 03:23:30 2021-11-21 16:53:00 2021-11-21 17:35:01 0:42:01 0:17:00 0:25:01 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2021-11-21T17:32:41.638526+0000 mon.b (mon.0) 274 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512700 2021-11-19 03:23:31 2021-11-21 17:03:59 2021-11-21 17:44:00 0:40:01 0:18:46 0:21:15 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} 3
pass 6512701 2021-11-19 03:23:31 2021-11-21 17:07:38 2021-11-21 17:29:56 0:22:18 0:09:24 0:12:54 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6512702 2021-11-19 03:23:32 2021-11-21 17:09:59 2021-11-21 17:37:38 0:27:39 0:13:55 0:13:44 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6512703 2021-11-19 03:23:33 2021-11-21 17:13:40 2021-11-21 17:36:28 0:22:48 0:09:51 0:12:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} 3
pass 6512704 2021-11-19 03:23:34 2021-11-21 17:14:30 2021-11-21 17:34:43 0:20:13 0:09:04 0:11:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6512705 2021-11-19 03:23:34 2021-11-21 17:14:50 2021-11-21 18:03:50 0:49:00 0:36:53 0:12:07 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6512706 2021-11-19 03:23:35 2021-11-21 17:15:51 2021-11-21 18:00:28 0:44:37 0:17:01 0:27:36 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2021-11-21T17:58:53.614658+0000 mon.a (mon.0) 270 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512707 2021-11-19 03:23:36 2021-11-21 17:17:06 2021-11-21 17:47:06 0:30:00 0:06:43 0:23:17 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} 1
pass 6512708 2021-11-19 03:23:37 2021-11-21 17:32:43 2021-11-21 17:59:27 0:26:44 0:14:42 0:12:02 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6512709 2021-11-19 03:23:38 2021-11-21 17:33:34 2021-11-21 18:22:43 0:49:09 0:37:36 0:11:33 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6512710 2021-11-19 03:23:39 2021-11-21 17:34:44 2021-11-21 17:56:27 0:21:43 0:08:55 0:12:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} 3
pass 6512711 2021-11-19 03:23:39 2021-11-21 17:35:03 2021-11-21 18:45:06 1:10:03 0:58:57 0:11:06 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6512712 2021-11-19 03:23:40 2021-11-21 17:36:35 2021-11-21 18:13:44 0:37:09 0:24:59 0:12:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} 3
dead 6512713 2021-11-19 03:23:41 2021-11-21 17:37:45 2021-11-21 19:36:22 1:58:37 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

pass 6512714 2021-11-19 03:23:42 2021-11-21 17:37:56 2021-11-21 18:01:44 0:23:48 0:11:31 0:12:17 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6512715 2021-11-19 03:23:43 2021-11-21 17:39:46 2021-11-21 18:02:44 0:22:58 0:09:00 0:13:58 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} 3
pass 6512716 2021-11-19 03:23:43 2021-11-21 17:41:02 2021-11-21 17:59:02 0:18:00 0:07:11 0:10:49 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} 1
pass 6512717 2021-11-19 03:23:44 2021-11-21 17:42:47 2021-11-21 18:29:13 0:46:26 0:24:51 0:21:35 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6512718 2021-11-19 03:23:45 2021-11-21 17:44:01 2021-11-21 18:14:02 0:30:01 0:09:40 0:20:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6512719 2021-11-19 03:23:46 2021-11-21 17:47:08 2021-11-21 18:15:08 0:28:00 0:08:15 0:19:45 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6512720 2021-11-19 03:23:47 2021-11-21 17:53:18 2021-11-21 18:27:27 0:34:09 0:16:50 0:17:19 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2021-11-21T18:16:15.725197+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log

pass 6512721 2021-11-19 03:23:47 2021-11-21 17:59:03 2021-11-21 18:31:04 0:32:01 0:08:12 0:23:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6512722 2021-11-19 03:23:48 2021-11-21 17:59:30 2021-11-21 18:36:16 0:36:46 0:24:31 0:12:15 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6512723 2021-11-19 03:23:49 2021-11-21 18:00:30 2021-11-21 18:29:46 0:29:16 0:17:16 0:12:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2021-11-21T18:28:02.395547+0000 mon.a (mon.0) 215 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512724 2021-11-19 03:23:50 2021-11-21 18:01:51 2021-11-21 18:19:44 0:17:53 0:07:10 0:10:43 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6512725 2021-11-19 03:23:51 2021-11-21 18:01:51 2021-11-21 18:28:44 0:26:53 0:16:09 0:10:44 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6512726 2021-11-19 03:23:51 2021-11-21 18:02:51 2021-11-21 18:21:49 0:18:58 0:08:13 0:10:45 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6512727 2021-11-19 03:23:52 2021-11-21 18:03:52 2021-11-21 18:40:08 0:36:16 0:15:26 0:20:50 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6512728 2021-11-19 03:23:53 2021-11-21 18:13:53 2021-11-21 18:42:42 0:28:49 0:08:15 0:20:34 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6512729 2021-11-19 03:23:54 2021-11-21 18:14:03 2021-11-21 18:38:04 0:24:01 0:12:59 0:11:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} 3
fail 6512730 2021-11-19 03:23:55 2021-11-21 18:15:09 2021-11-21 18:49:10 0:34:01 0:17:02 0:16:59 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2021-11-21T18:48:15.680346+0000 mon.b (mon.0) 296 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6512731 2021-11-19 03:23:55 2021-11-21 18:22:45 2021-11-21 19:07:22 0:44:37 0:28:34 0:16:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3