Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 6527405 2021-11-26 03:20:49 2021-11-26 11:25:26 2021-11-26 11:43:26 0:18:00 0:07:52 0:10:08 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} 1
dead 6527406 2021-11-26 03:20:50 2021-11-26 11:25:25 2021-11-26 11:49:41 0:24:16 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

SSH connection to gibba001 was lost: 'uname -r'

pass 6527407 2021-11-26 03:20:51 2021-11-26 11:25:29 2021-11-26 12:03:30 0:38:01 0:25:26 0:12:35 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} 3
pass 6527408 2021-11-26 03:20:51 2021-11-26 11:33:46 2021-11-26 12:02:05 0:28:19 0:15:35 0:12:44 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} 3
fail 6527409 2021-11-26 03:20:52 2021-11-26 11:36:07 2021-11-26 15:27:13 3:51:06 3:34:14 0:16:52 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{legacy} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Command failed on gibba032 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude72z0v39m -r -- '-g auto -g blockdev -x clone'"

pass 6527410 2021-11-26 03:20:53 2021-11-26 11:36:42 2021-11-26 12:30:45 0:54:03 0:39:25 0:14:38 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
fail 6527411 2021-11-26 03:20:54 2021-11-26 11:40:58 2021-11-26 11:49:53 0:08:55 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major

pass 6527412 2021-11-26 03:20:55 2021-11-26 11:41:34 2021-11-26 11:59:35 0:18:01 0:07:22 0:10:39 gibba master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
pass 6527413 2021-11-26 03:20:55 2021-11-26 11:41:59 2021-11-26 12:05:18 0:23:19 0:09:38 0:13:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6527414 2021-11-26 03:20:56 2021-11-26 11:43:19 2021-11-26 12:19:48 0:36:29 0:18:06 0:18:23 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-11-26T12:17:28.578002+0000 mon.a (mon.0) 216 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527415 2021-11-26 03:20:57 2021-11-26 11:43:27 2021-11-26 12:11:28 0:28:01 0:10:02 0:17:59 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
dead 6527416 2021-11-26 03:20:58 2021-11-26 11:50:01 2021-11-26 13:57:38 2:07:37 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_huge_osdmap} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

fail 6527417 2021-11-26 03:20:59 2021-11-26 11:59:33 2021-11-26 12:30:27 0:30:54 0:17:44 0:13:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-11-26T12:27:58.092432+0000 mon.a (mon.0) 225 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527418 2021-11-26 03:20:59 2021-11-26 11:59:36 2021-11-26 12:25:37 0:26:01 0:11:34 0:14:27 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6527419 2021-11-26 03:21:00 2021-11-26 12:01:14 2021-11-26 12:22:29 0:21:15 0:07:52 0:13:23 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} 1
pass 6527420 2021-11-26 03:21:01 2021-11-26 12:02:34 2021-11-26 12:38:25 0:35:51 0:15:55 0:19:56 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6527421 2021-11-26 03:21:02 2021-11-26 12:03:32 2021-11-26 12:27:32 0:24:00 0:09:43 0:14:17 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
fail 6527422 2021-11-26 03:21:03 2021-11-26 12:10:26 2021-11-26 12:50:53 0:40:27 0:17:10 0:23:17 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-11-26T12:49:19.006464+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527423 2021-11-26 03:21:04 2021-11-26 12:11:30 2021-11-26 12:49:31 0:38:01 0:19:45 0:18:16 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} 3
fail 6527424 2021-11-26 03:21:05 2021-11-26 12:22:38 2021-11-26 12:52:29 0:29:51 0:17:12 0:12:39 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

"2021-11-26T12:40:36.748011+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527425 2021-11-26 03:21:05 2021-11-26 12:24:28 2021-11-26 12:54:17 0:29:49 0:11:54 0:17:55 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6527426 2021-11-26 03:21:06 2021-11-26 12:25:38 2021-11-26 12:45:38 0:20:00 0:08:57 0:11:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
pass 6527427 2021-11-26 03:21:07 2021-11-26 12:27:34 2021-11-26 12:49:34 0:22:00 0:07:40 0:14:20 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
fail 6527428 2021-11-26 03:21:08 2021-11-26 12:30:30 2021-11-26 13:06:26 0:35:56 0:16:57 0:18:59 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Command failed on gibba027 with status 120: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0'

pass 6527429 2021-11-26 03:21:09 2021-11-26 12:30:46 2021-11-26 12:58:47 0:28:01 0:09:50 0:18:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} 3
pass 6527430 2021-11-26 03:21:10 2021-11-26 12:38:31 2021-11-26 13:04:47 0:26:16 0:08:29 0:17:47 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6527431 2021-11-26 03:21:10 2021-11-26 12:44:52 2021-11-26 13:18:24 0:33:32 0:13:11 0:20:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} 3
pass 6527432 2021-11-26 03:21:11 2021-11-26 12:45:40 2021-11-26 13:07:40 0:22:00 0:08:25 0:13:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
pass 6527433 2021-11-26 03:21:12 2021-11-26 12:49:32 2021-11-26 13:25:34 0:36:02 0:21:46 0:14:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6527434 2021-11-26 03:21:13 2021-11-26 12:49:35 2021-11-26 13:09:36 0:20:01 0:08:46 0:11:15 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} 3
pass 6527435 2021-11-26 03:21:14 2021-11-26 12:54:24 2021-11-26 13:12:18 0:17:54 0:07:01 0:10:53 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_modprobe} 1
pass 6527436 2021-11-26 03:21:14 2021-11-26 12:54:24 2021-11-26 13:26:29 0:32:05 0:16:37 0:15:28 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6527437 2021-11-26 03:21:15 2021-11-26 12:58:35 2021-11-26 14:18:23 1:19:48 1:01:25 0:18:23 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6527438 2021-11-26 03:21:16 2021-11-26 12:58:49 2021-11-26 13:24:50 0:26:01 0:08:43 0:17:18 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} 3
pass 6527439 2021-11-26 03:21:17 2021-11-26 13:06:36 2021-11-26 13:37:27 0:30:51 0:17:58 0:12:53 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
fail 6527440 2021-11-26 03:21:18 2021-11-26 13:07:27 2021-11-26 13:48:23 0:40:56 0:17:23 0:23:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2021-11-26T13:45:43.182938+0000 mon.a (mon.0) 210 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527441 2021-11-26 03:21:18 2021-11-26 13:07:42 2021-11-26 13:33:42 0:26:00 0:13:25 0:12:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} 3
pass 6527442 2021-11-26 03:21:19 2021-11-26 13:09:37 2021-11-26 13:45:39 0:36:02 0:10:44 0:25:18 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6527443 2021-11-26 03:21:20 2021-11-26 13:18:29 2021-11-26 14:02:07 0:43:38 0:25:26 0:18:12 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} 3
pass 6527444 2021-11-26 03:21:21 2021-11-26 13:24:10 2021-11-26 13:42:05 0:17:55 0:06:59 0:10:56 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} 1
pass 6527445 2021-11-26 03:21:22 2021-11-26 13:24:10 2021-11-26 14:04:31 0:40:21 0:26:35 0:13:46 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6527446 2021-11-26 03:21:22 2021-11-26 13:24:51 2021-11-26 14:24:54 1:00:03 0:25:57 0:34:06 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
pass 6527447 2021-11-26 03:21:23 2021-11-26 13:25:35 2021-11-26 14:03:37 0:38:02 0:19:40 0:18:22 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
dead 6527448 2021-11-26 03:21:24 2021-11-26 13:26:31 2021-11-26 13:53:31 0:27:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} 3
Failure Reason:

SSH connection to gibba001 was lost: 'uname -r'

dead 6527449 2021-11-26 03:21:25 2021-11-26 13:33:44 2021-11-26 15:57:54 2:24:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build}
Failure Reason:

reached maximum tries (100) after waiting for 600 seconds

pass 6527450 2021-11-26 03:21:26 2021-11-26 13:37:32 2021-11-26 14:07:22 0:29:50 0:10:11 0:19:39 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} 3
pass 6527451 2021-11-26 03:21:26 2021-11-26 13:45:24 2021-11-26 14:13:27 0:28:03 0:09:10 0:18:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6527452 2021-11-26 03:21:27 2021-11-26 13:45:40 2021-11-26 14:17:41 0:32:01 0:17:45 0:14:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-11-26T14:16:42.787532+0000 mon.a (mon.0) 198 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6527453 2021-11-26 03:21:28 2021-11-26 13:53:45 2021-11-26 14:29:28 0:35:43 0:17:09 0:18:34 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2021-11-26T14:28:41.359146+0000 mon.a (mon.0) 202 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527454 2021-11-26 03:21:29 2021-11-26 14:02:17 2021-11-26 14:20:31 0:18:14 0:07:19 0:10:55 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
fail 6527455 2021-11-26 03:21:30 2021-11-26 14:02:37 2021-11-26 14:32:38 0:30:01 0:16:09 0:13:52 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

"2021-11-26T14:20:42.455328+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527456 2021-11-26 03:21:30 2021-11-26 14:03:38 2021-11-26 14:37:40 0:34:02 0:09:14 0:24:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} 3
pass 6527457 2021-11-26 03:21:31 2021-11-26 14:04:38 2021-11-26 15:19:28 1:14:50 1:00:06 0:14:44 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6527458 2021-11-26 03:21:32 2021-11-26 14:07:29 2021-11-26 14:45:29 0:38:00 0:21:42 0:16:18 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6527459 2021-11-26 03:21:33 2021-11-26 14:13:30 2021-11-26 14:42:30 0:29:00 0:12:11 0:16:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6527460 2021-11-26 03:21:34 2021-11-26 14:17:43 2021-11-26 14:49:44 0:32:01 0:17:12 0:14:49 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-11-26T14:36:56.415212+0000 mon.b (mon.0) 120 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527461 2021-11-26 03:21:34 2021-11-26 14:18:31 2021-11-26 14:46:57 0:28:26 0:09:38 0:18:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} 3
pass 6527462 2021-11-26 03:21:35 2021-11-26 14:24:42 2021-11-26 14:48:35 0:23:53 0:07:21 0:16:32 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} 1
fail 6527463 2021-11-26 03:21:36 2021-11-26 14:24:56 2021-11-26 15:08:58 0:44:02 0:19:10 0:24:52 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

"2021-11-26T14:54:09.116664+0000 mon.a (mon.0) 115 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527464 2021-11-26 03:21:37 2021-11-26 14:29:33 2021-11-26 15:22:33 0:53:00 0:39:22 0:13:38 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
pass 6527465 2021-11-26 03:21:38 2021-11-26 14:32:44 2021-11-26 14:54:01 0:21:17 0:09:49 0:11:28 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6527466 2021-11-26 03:21:38 2021-11-26 14:34:14 2021-11-26 15:05:36 0:31:22 0:08:50 0:22:32 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6527467 2021-11-26 03:21:39 2021-11-26 14:37:41 2021-11-26 15:13:43 0:36:02 0:17:04 0:18:58 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2021-11-26T15:01:52.154455+0000 mon.b (mon.0) 204 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527468 2021-11-26 03:21:40 2021-11-26 14:45:36 2021-11-26 15:07:05 0:21:29 0:08:40 0:12:49 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6527469 2021-11-26 03:21:41 2021-11-26 14:47:07 2021-11-26 15:21:55 0:34:48 0:23:05 0:11:43 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
pass 6527470 2021-11-26 03:21:41 2021-11-26 14:48:07 2021-11-26 15:16:07 0:28:00 0:08:52 0:19:08 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
pass 6527471 2021-11-26 03:21:42 2021-11-26 14:49:45 2021-11-26 15:07:45 0:18:00 0:07:36 0:10:24 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} 1
pass 6527472 2021-11-26 03:21:43 2021-11-26 14:54:09 2021-11-26 15:35:39 0:41:30 0:18:33 0:22:57 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6527473 2021-11-26 03:21:44 2021-11-26 15:05:40 2021-11-26 15:27:09 0:21:29 0:08:14 0:13:15 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} 3
pass 6527474 2021-11-26 03:21:44 2021-11-26 15:07:11 2021-11-26 15:35:18 0:28:07 0:15:51 0:12:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
fail 6527475 2021-11-26 03:21:45 2021-11-26 15:07:31 2021-11-26 15:47:32 0:40:01 0:17:03 0:22:58 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

"2021-11-26T15:35:35.736313+0000 mon.a (mon.0) 159 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527476 2021-11-26 03:21:46 2021-11-26 15:07:47 2021-11-26 15:37:48 0:30:01 0:12:56 0:17:05 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
fail 6527477 2021-11-26 03:21:47 2021-11-26 15:08:59 2021-11-26 15:45:01 0:36:02 0:17:16 0:18:46 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

"2021-11-26T15:43:12.934180+0000 mon.b (mon.0) 238 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6527478 2021-11-26 03:21:48 2021-11-26 15:13:44 2021-11-26 15:51:46 0:38:02 0:17:05 0:20:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2021-11-26T15:49:33.162818+0000 mon.b (mon.0) 180 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

pass 6527479 2021-11-26 03:21:48 2021-11-26 15:19:33 2021-11-26 15:37:31 0:17:58 0:07:31 0:10:27 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
fail 6527480 2021-11-26 03:21:49 2021-11-26 15:19:34 2021-11-26 15:50:34 0:31:00 0:16:41 0:14:19 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6527481 2021-11-26 03:21:50 2021-11-26 15:22:34 2021-11-26 16:03:03 0:40:29 0:24:43 0:15:46 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} 3
pass 6527482 2021-11-26 03:21:51 2021-11-26 15:27:15 2021-11-26 15:53:15 0:26:00 0:15:26 0:10:34 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} 3
pass 6527483 2021-11-26 03:21:52 2021-11-26 15:27:16 2021-11-26 15:47:10 0:19:54 0:08:13 0:11:41 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
fail 6527484 2021-11-26 03:21:52 2021-11-26 15:27:16 2021-11-26 16:05:25 0:38:09 0:17:19 0:20:50 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
Failure Reason:

"2021-11-26T15:52:26.500087+0000 mon.a (mon.0) 122 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527485 2021-11-26 03:21:53 2021-11-26 15:35:27 2021-11-26 15:55:42 0:20:15 0:09:19 0:10:56 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
fail 6527486 2021-11-26 03:21:54 2021-11-26 15:35:48 2021-11-26 16:05:18 0:29:30 0:17:18 0:12:12 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

"2021-11-26T15:53:37.738294+0000 mon.a (mon.0) 130 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527487 2021-11-26 03:21:55 2021-11-26 15:37:18 2021-11-26 16:04:08 0:26:50 0:08:53 0:17:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} 3
pass 6527488 2021-11-26 03:21:56 2021-11-26 15:37:49 2021-11-26 16:09:50 0:32:01 0:09:26 0:22:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6527489 2021-11-26 03:21:57 2021-11-26 15:44:10 2021-11-26 16:34:43 0:50:33 0:35:45 0:14:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6527490 2021-11-26 03:21:57 2021-11-26 15:45:02 2021-11-26 16:19:03 0:34:01 0:11:32 0:22:29 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6527491 2021-11-26 03:21:58 2021-11-26 15:47:40 2021-11-26 16:07:13 0:19:33 0:07:24 0:12:09 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} 1
pass 6527492 2021-11-26 03:21:59 2021-11-26 15:50:11 2021-11-26 16:18:28 0:28:17 0:17:26 0:10:51 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6527493 2021-11-26 03:22:00 2021-11-26 15:50:41 2021-11-26 17:17:06 1:26:25 1:02:56 0:23:29 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6527494 2021-11-26 03:22:01 2021-11-26 15:51:47 2021-11-26 16:21:48 0:30:01 0:17:00 0:13:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

"2021-11-26T16:19:20.622152+0000 mon.b (mon.0) 266 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6527495 2021-11-26 03:22:01 2021-11-26 15:57:55 2021-11-26 16:53:58 0:56:03 0:17:15 0:38:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-11-26T16:51:50.455570+0000 mon.a (mon.0) 309 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527496 2021-11-26 03:22:02 2021-11-26 16:03:03 2021-11-26 16:36:13 0:33:10 0:20:02 0:13:08 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} 3
pass 6527497 2021-11-26 03:22:03 2021-11-26 16:04:14 2021-11-26 16:29:11 0:24:57 0:12:15 0:12:42 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
fail 6527498 2021-11-26 03:22:04 2021-11-26 16:05:24 2021-11-26 16:35:22 0:29:58 0:17:16 0:12:42 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-11-26T16:33:19.663294+0000 mon.a (mon.0) 226 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527499 2021-11-26 03:22:04 2021-11-26 16:05:35 2021-11-26 16:27:10 0:21:35 0:09:05 0:12:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} 3
pass 6527500 2021-11-26 03:22:05 2021-11-26 16:07:15 2021-11-26 16:25:26 0:18:11 0:07:27 0:10:44 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} 1
pass 6527501 2021-11-26 03:22:06 2021-11-26 16:08:16 2021-11-26 16:52:32 0:44:16 0:23:29 0:20:47 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6527502 2021-11-26 03:22:07 2021-11-26 16:09:51 2021-11-26 16:37:52 0:28:01 0:09:55 0:18:06 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} 3
fail 6527503 2021-11-26 03:22:08 2021-11-26 16:18:37 2021-11-26 16:47:49 0:29:12 0:17:16 0:11:56 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

"2021-11-26T16:46:35.453414+0000 mon.a (mon.0) 304 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527504 2021-11-26 03:22:08 2021-11-26 16:19:05 2021-11-26 16:53:06 0:34:01 0:12:47 0:21:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 3
fail 6527505 2021-11-26 03:22:09 2021-11-26 16:20:08 2021-11-26 17:02:46 0:42:38 0:17:22 0:25:16 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2021-11-26T16:51:32.883093+0000 mon.a (mon.0) 123 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527506 2021-11-26 03:22:10 2021-11-26 16:21:49 2021-11-26 17:01:51 0:40:02 0:22:05 0:17:57 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6527507 2021-11-26 03:22:11 2021-11-26 16:34:50 2021-11-26 16:55:12 0:20:22 0:09:02 0:11:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} 3
pass 6527508 2021-11-26 03:22:12 2021-11-26 16:35:30 2021-11-26 16:54:15 0:18:45 0:07:10 0:11:35 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_modprobe} 1
pass 6527509 2021-11-26 03:22:12 2021-11-26 16:36:21 2021-11-26 17:07:31 0:31:10 0:19:34 0:11:36 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
fail 6527510 2021-11-26 03:22:13 2021-11-26 16:37:31 2021-11-26 17:15:36 0:38:05 0:17:31 0:20:34 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2021-11-26T17:04:02.772709+0000 mon.a (mon.0) 191 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6527511 2021-11-26 03:22:14 2021-11-26 16:37:54 2021-11-26 17:23:56 0:46:02 0:16:58 0:29:04 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

"2021-11-26T17:12:38.653328+0000 mon.b (mon.0) 211 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527512 2021-11-26 03:22:15 2021-11-26 16:47:54 2021-11-26 17:12:40 0:24:46 0:08:36 0:16:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} 3
fail 6527513 2021-11-26 03:22:15 2021-11-26 16:52:25 2021-11-26 17:22:35 0:30:10 0:17:09 0:13:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

"2021-11-26T17:20:13.410339+0000 mon.a (mon.0) 296 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6527514 2021-11-26 03:22:16 2021-11-26 16:52:36 2021-11-26 17:16:45 0:24:09 0:10:53 0:13:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6527515 2021-11-26 03:22:17 2021-11-26 16:52:46 2021-11-26 17:38:06 0:45:20 0:27:04 0:18:16 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
pass 6527516 2021-11-26 03:22:18 2021-11-26 16:53:07 2021-11-26 17:13:08 0:20:01 0:07:27 0:12:34 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} 1
pass 6527517 2021-11-26 03:22:19 2021-11-26 16:54:00 2021-11-26 17:46:03 0:52:03 0:26:50 0:25:13 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6527518 2021-11-26 03:22:19 2021-11-26 17:00:07 2021-11-26 17:38:33 0:38:26 0:24:27 0:13:59 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} 3
fail 6527519 2021-11-26 03:22:20 2021-11-26 17:01:52 2021-11-26 17:39:54 0:38:02 0:17:03 0:20:59 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2021-11-26T17:37:58.488058+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6527520 2021-11-26 03:22:21 2021-11-26 17:02:49 2021-11-26 21:29:04 4:26:15 4:01:07 0:25:08 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{secure} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Command failed on gibba035 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude9s4sf151 -r -- '-g auto -g blockdev -x clone'"

pass 6527521 2021-11-26 03:22:22 2021-11-26 17:13:09 2021-11-26 18:27:14 1:14:05 1:03:28 0:10:37 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
pass 6527522 2021-11-26 03:22:23 2021-11-26 17:16:51 2021-11-26 17:37:26 0:20:35 0:09:29 0:11:06 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6527523 2021-11-26 03:22:23 2021-11-26 17:17:12 2021-11-26 17:48:30 0:31:18 0:14:19 0:16:59 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
fail 6527524 2021-11-26 03:22:24 2021-11-26 17:22:43 2021-11-26 17:51:20 0:28:37 0:17:11 0:11:26 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

"2021-11-26T17:39:32.843219+0000 mon.a (mon.0) 118 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527525 2021-11-26 03:22:25 2021-11-26 17:23:33 2021-11-26 17:44:11 0:20:38 0:09:05 0:11:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_huge_osdmap} 3
fail 6527526 2021-11-26 03:22:26 2021-11-26 17:23:57 2021-11-26 18:05:59 0:42:02 0:17:21 0:24:41 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

"2021-11-26T18:04:04.931341+0000 mon.b (mon.0) 158 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527527 2021-11-26 03:22:27 2021-11-26 17:24:24 2021-11-26 18:00:16 0:35:52 0:11:45 0:24:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
pass 6527528 2021-11-26 03:22:28 2021-11-26 17:38:16 2021-11-26 17:55:19 0:17:03 0:06:53 0:10:10 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} 1
pass 6527529 2021-11-26 03:22:28 2021-11-26 17:38:16 2021-11-26 18:06:37 0:28:21 0:16:39 0:11:42 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
pass 6527530 2021-11-26 03:22:29 2021-11-26 17:38:37 2021-11-26 17:58:44 0:20:07 0:09:03 0:11:04 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
pass 6527531 2021-11-26 03:22:30 2021-11-26 17:38:57 2021-11-26 18:56:21 1:17:24 1:00:46 0:16:38 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
pass 6527532 2021-11-26 03:22:31 2021-11-26 17:39:55 2021-11-26 18:17:57 0:38:02 0:22:47 0:15:15 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} 3
pass 6527533 2021-11-26 03:22:32 2021-11-26 17:44:18 2021-11-26 18:13:12 0:28:54 0:12:04 0:16:50 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
fail 6527534 2021-11-26 03:22:32 2021-11-26 17:46:04 2021-11-26 18:16:05 0:30:01 0:17:01 0:13:00 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

"2021-11-26T18:15:07.083809+0000 mon.b (mon.0) 304 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

pass 6527535 2021-11-26 03:22:33 2021-11-26 17:51:30 2021-11-26 18:18:50 0:27:20 0:09:23 0:17:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
pass 6527536 2021-11-26 03:22:34 2021-11-26 17:58:51 2021-11-26 18:16:46 0:17:55 0:07:45 0:10:10 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
pass 6527537 2021-11-26 03:22:35 2021-11-26 17:58:51 2021-11-26 18:41:50 0:42:59 0:30:57 0:12:02 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
pass 6527538 2021-11-26 03:22:36 2021-11-26 18:00:22 2021-11-26 18:24:50 0:24:28 0:09:17 0:15:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6527539 2021-11-26 03:22:36 2021-11-26 18:05:03 2021-11-26 18:33:12 0:28:09 0:08:24 0:19:45 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
fail 6527540 2021-11-26 03:22:37 2021-11-26 18:06:00 2021-11-26 18:34:02 0:28:02 0:16:59 0:11:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

"2021-11-26T18:32:47.548406+0000 mon.b (mon.0) 317 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

pass 6527541 2021-11-26 03:22:38 2021-11-26 18:13:14 2021-11-26 18:35:37 0:22:23 0:08:16 0:14:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
pass 6527542 2021-11-26 03:22:39 2021-11-26 18:15:55 2021-11-26 18:53:24 0:37:29 0:23:45 0:13:44 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
fail 6527543 2021-11-26 03:22:40 2021-11-26 18:16:06 2021-11-26 19:04:09 0:48:03 0:17:12 0:30:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2021-11-26T19:01:38.130531+0000 mon.b (mon.0) 310 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527544 2021-11-26 03:22:40 2021-11-26 18:17:26 2021-11-26 18:35:20 0:17:54 0:07:32 0:10:22 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} 1
pass 6527545 2021-11-26 03:22:41 2021-11-26 18:17:26 2021-11-26 18:42:52 0:25:26 0:13:42 0:11:44 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6527546 2021-11-26 03:22:42 2021-11-26 18:17:58 2021-11-26 19:44:04 1:26:06 0:39:12 0:46:54 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
pass 6527547 2021-11-26 03:22:43 2021-11-26 18:18:56 2021-11-26 18:44:52 0:25:56 0:08:23 0:17:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} 3
pass 6527548 2021-11-26 03:22:43 2021-11-26 18:24:58 2021-11-26 18:54:41 0:29:43 0:16:48 0:12:55 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6527549 2021-11-26 03:22:44 2021-11-26 18:26:28 2021-11-26 18:53:26 0:26:58 0:08:35 0:18:23 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6527550 2021-11-26 03:22:45 2021-11-26 18:27:15 2021-11-26 18:59:16 0:32:01 0:13:28 0:18:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} 3
pass 6527551 2021-11-26 03:22:46 2021-11-26 18:33:39 2021-11-26 19:05:50 0:32:11 0:12:14 0:19:57 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
pass 6527552 2021-11-26 03:22:47 2021-11-26 18:34:03 2021-11-26 19:20:05 0:46:02 0:26:46 0:19:16 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} 3
pass 6527553 2021-11-26 03:22:47 2021-11-26 18:41:51 2021-11-26 19:01:46 0:19:55 0:07:34 0:12:21 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} 1
pass 6527554 2021-11-26 03:22:48 2021-11-26 18:41:51 2021-11-26 19:26:59 0:45:08 0:30:03 0:15:05 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
pass 6527555 2021-11-26 03:22:49 2021-11-26 18:45:02 2021-11-26 19:29:33 0:44:31 0:24:53 0:19:38 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
pass 6527556 2021-11-26 03:22:50 2021-11-26 18:53:34 2021-11-26 19:20:44 0:27:10 0:15:50 0:11:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
pass 6527557 2021-11-26 03:22:51 2021-11-26 18:54:45 2021-11-26 19:14:20 0:19:35 0:07:37 0:11:58 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
pass 6527558 2021-11-26 03:22:51 2021-11-26 18:56:25 2021-11-26 19:27:56 0:31:31 0:19:21 0:12:10 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
pass 6527559 2021-11-26 03:22:52 2021-11-26 18:57:56 2021-11-26 19:22:26 0:24:30 0:09:33 0:14:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} 3
pass 6527560 2021-11-26 03:22:53 2021-11-26 18:59:17 2021-11-26 19:47:20 0:48:03 0:15:29 0:32:34 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
pass 6527561 2021-11-26 03:22:54 2021-11-26 19:02:27 2021-11-26 19:34:23 0:31:56 0:09:10 0:22:46 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} 3
pass 6527562 2021-11-26 03:22:55 2021-11-26 19:04:10 2021-11-26 19:26:11 0:22:01 0:09:10 0:12:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_huge_osdmap} 3
pass 6527563 2021-11-26 03:22:55 2021-11-26 19:14:29 2021-11-26 20:07:51 0:53:22 0:36:56 0:16:26 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
pass 6527564 2021-11-26 03:22:56 2021-11-26 19:19:50 2021-11-26 19:44:30 0:24:40 0:11:12 0:13:28 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
pass 6527565 2021-11-26 03:22:57 2021-11-26 19:20:06 2021-11-26 19:38:07 0:18:01 0:07:30 0:10:31 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
fail 6527566 2021-11-26 03:22:58 2021-11-26 19:22:31 2021-11-26 19:52:36 0:30:05 0:16:16 0:13:49 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

reached maximum tries (90) after waiting for 540 seconds

pass 6527567 2021-11-26 03:22:59 2021-11-26 19:24:41 2021-11-26 19:47:00 0:22:19 0:09:55 0:12:24 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} 3
fail 6527568 2021-11-26 03:22:59 2021-11-26 19:26:12 2021-11-26 19:56:13 0:30:01 0:17:04 0:12:57 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

"2021-11-26T19:54:23.253348+0000 mon.b (mon.0) 181 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 6527569 2021-11-26 03:23:00 2021-11-26 19:27:02 2021-11-26 19:49:42 0:22:40 0:08:31 0:14:09 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
Failure Reason:

Command failed (workunit test rbd/krbd_udev_enumerate.sh) on gibba037 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3d3dcee87aa6ccf5c8e905a275b4673aab98570 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/krbd_udev_enumerate.sh'

pass 6527570 2021-11-26 03:23:01 2021-11-26 19:29:43 2021-11-26 19:58:33 0:28:50 0:12:14 0:16:36 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
pass 6527571 2021-11-26 03:23:02 2021-11-26 19:34:34 2021-11-26 20:04:51 0:30:17 0:11:18 0:18:59 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
pass 6527572 2021-11-26 03:23:03 2021-11-26 19:38:08 2021-11-26 20:06:09 0:28:01 0:09:07 0:18:54 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} 3
pass 6527573 2021-11-26 03:23:03 2021-11-26 19:42:55 2021-11-26 20:00:54 0:17:59 0:07:17 0:10:42 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} 1
pass 6527574 2021-11-26 03:23:04 2021-11-26 19:42:56 2021-11-26 20:23:07 0:40:11 0:24:42 0:15:29 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
pass 6527575 2021-11-26 03:23:05 2021-11-26 19:44:05 2021-11-26 20:56:10 1:12:05 1:00:54 0:11:11 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
fail 6527576 2021-11-26 03:23:06 2021-11-26 19:47:06 2021-11-26 20:17:34 0:30:28 0:17:09 0:13:19 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

"2021-11-26T20:16:15.712968+0000 mon.a (mon.0) 299 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527577 2021-11-26 03:23:07 2021-11-26 19:47:21 2021-11-26 20:25:23 0:38:02 0:08:51 0:29:11 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6527578 2021-11-26 03:23:07 2021-11-26 19:49:47 2021-11-26 20:16:25 0:26:38 0:13:03 0:13:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} 3
fail 6527579 2021-11-26 03:23:08 2021-11-26 19:52:38 2021-11-26 20:24:57 0:32:19 0:17:22 0:14:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

"2021-11-26T20:22:10.951541+0000 mon.a (mon.0) 220 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527580 2021-11-26 03:23:09 2021-11-26 19:55:09 2021-11-26 20:37:21 0:42:12 0:21:38 0:20:34 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
pass 6527581 2021-11-26 03:23:10 2021-11-26 19:56:14 2021-11-26 20:18:15 0:22:01 0:09:07 0:12:54 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
pass 6527582 2021-11-26 03:23:11 2021-11-26 20:05:21 2021-11-26 20:23:40 0:18:19 0:07:11 0:11:08 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_modprobe} 1
pass 6527583 2021-11-26 03:23:11 2021-11-26 20:05:21 2021-11-26 20:33:50 0:28:29 0:15:45 0:12:44 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
fail 6527584 2021-11-26 03:23:12 2021-11-26 20:06:10 2021-11-26 20:44:12 0:38:02 0:17:29 0:20:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

"2021-11-26T20:42:56.498057+0000 mon.b (mon.0) 155 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 6527585 2021-11-26 03:23:13 2021-11-26 20:07:52 2021-11-26 22:15:40 2:07:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

pass 6527586 2021-11-26 03:23:14 2021-11-26 20:17:44 2021-11-26 20:37:54 0:20:10 0:08:23 0:11:47 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} 3
pass 6527587 2021-11-26 03:23:15 2021-11-26 20:17:54 2021-11-26 20:47:35 0:29:41 0:13:07 0:16:34 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
pass 6527588 2021-11-26 03:23:16 2021-11-26 20:18:16 2021-11-26 20:48:18 0:30:02 0:11:08 0:18:54 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
pass 6527589 2021-11-26 03:23:16 2021-11-26 20:23:36 2021-11-26 21:00:54 0:37:18 0:24:56 0:12:22 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} 3
pass 6527590 2021-11-26 03:23:17 2021-11-26 20:25:06 2021-11-26 20:43:05 0:17:59 0:07:17 0:10:42 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
pass 6527591 2021-11-26 03:23:18 2021-11-26 20:25:06 2021-11-26 21:16:38 0:51:32 0:28:02 0:23:30 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{prefer-crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
fail 6527592 2021-11-26 03:23:19 2021-11-26 20:25:24 2021-11-26 21:01:26 0:36:02 0:17:14 0:18:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

"2021-11-26T20:49:56.323697+0000 mon.b (mon.0) 184 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

fail 6527593 2021-11-26 03:23:19 2021-11-26 21:05:58 1026 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

"2021-11-26T20:54:40.078165+0000 mon.a (mon.0) 191 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527594 2021-11-26 03:23:20 2021-11-26 20:37:59 2021-11-26 21:03:49 0:25:50 0:09:17 0:16:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
pass 6527595 2021-11-26 03:23:21 2021-11-26 20:43:50 2021-11-26 21:12:29 0:28:39 0:14:20 0:14:19 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
pass 6527596 2021-11-26 03:23:22 2021-11-26 20:44:13 2021-11-26 21:16:15 0:32:02 0:09:51 0:22:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} 3
pass 6527597 2021-11-26 03:23:23 2021-11-26 20:46:31 2021-11-26 21:07:36 0:21:05 0:09:02 0:12:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_huge_osdmap} 3
pass 6527598 2021-11-26 03:23:23 2021-11-26 20:47:41 2021-11-26 21:49:02 1:01:21 0:37:05 0:24:16 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
fail 6527599 2021-11-26 03:23:24 2021-11-26 20:48:19 2021-11-26 21:28:21 0:40:02 0:17:08 0:22:54 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

"2021-11-26T21:27:29.013549+0000 mon.b (mon.0) 176 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527600 2021-11-26 03:23:25 2021-11-26 20:56:11 2021-11-26 21:14:11 0:18:00 0:07:09 0:10:51 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} 1
pass 6527601 2021-11-26 03:23:26 2021-11-26 21:01:13 2021-11-26 21:27:53 0:26:40 0:13:54 0:12:46 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{legacy} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
pass 6527602 2021-11-26 03:23:27 2021-11-26 21:01:27 2021-11-26 21:59:30 0:58:03 0:40:22 0:17:41 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
pass 6527603 2021-11-26 03:23:28 2021-11-26 21:03:54 2021-11-26 21:25:51 0:21:57 0:08:56 0:13:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} 3
pass 6527604 2021-11-26 03:23:28 2021-11-26 21:06:05 2021-11-26 22:24:39 1:18:34 1:00:46 0:17:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
fail 6527605 2021-11-26 03:23:29 2021-11-26 21:12:37 2021-11-26 21:42:52 0:30:15 0:17:17 0:12:58 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

"2021-11-26T21:41:51.500538+0000 mon.a (mon.0) 267 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527606 2021-11-26 03:23:30 2021-11-26 21:14:12 2021-11-26 21:40:13 0:26:01 0:12:04 0:13:57 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
pass 6527607 2021-11-26 03:23:31 2021-11-26 21:15:08 2021-11-26 21:51:56 0:36:48 0:12:15 0:24:33 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 6527608 2021-11-26 03:23:32 2021-11-26 21:16:16 2021-11-26 21:46:17 0:30:01 0:09:02 0:20:59 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} 3
pass 6527609 2021-11-26 03:23:33 2021-11-26 21:28:00 2021-11-26 21:45:55 0:17:55 0:07:32 0:10:23 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} 1
fail 6527610 2021-11-26 03:23:33 2021-11-26 21:28:01 2021-11-26 22:06:05 0:38:04 0:26:39 0:11:25 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map ms_mode$/{secure} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
Failure Reason:

"2021-11-26T21:44:03.565492+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log

pass 6527611 2021-11-26 03:23:34 2021-11-26 21:28:22 2021-11-26 21:58:23 0:30:01 0:09:17 0:20:44 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} 3
pass 6527612 2021-11-26 03:23:35 2021-11-26 21:28:21 2021-11-26 21:48:54 0:20:33 0:08:25 0:12:08 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
pass 6527613 2021-11-26 03:23:36 2021-11-26 21:29:12 2021-11-26 22:06:41 0:37:29 0:13:01 0:24:28 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
pass 6527614 2021-11-26 03:23:37 2021-11-26 21:40:14 2021-11-26 22:08:15 0:28:01 0:08:34 0:19:27 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} 3
fail 6527615 2021-11-26 03:23:37 2021-11-26 21:42:54 2021-11-26 22:12:43 0:29:49 0:17:18 0:12:31 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

"2021-11-26T22:11:04.449803+0000 mon.a (mon.0) 335 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 6527616 2021-11-26 03:23:38 2021-11-26 21:44:45 2021-11-26 22:16:55 0:32:10 0:17:19 0:14:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

"2021-11-26T22:15:37.992445+0000 mon.a (mon.0) 332 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 6527617 2021-11-26 03:23:39 2021-11-26 21:46:18 2021-11-26 22:02:18 0:16:00 0:07:04 0:08:56 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout} 1
pass 6527618 2021-11-26 03:23:40 2021-11-26 21:48:56 2021-11-26 22:19:56 0:31:00 0:16:02 0:14:58 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map ms_mode$/{secure} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
pass 6527619 2021-11-26 03:23:41 2021-11-26 21:51:57 2021-11-26 22:17:56 0:25:59 0:08:08 0:17:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
pass 6527620 2021-11-26 03:23:41 2021-11-26 21:57:58 2021-11-26 22:25:57 0:27:59 0:15:54 0:12:05 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
pass 6527621 2021-11-26 03:23:42 2021-11-26 21:57:58 2021-11-26 22:25:56 0:27:58 0:08:05 0:19:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} 3
pass 6527622 2021-11-26 03:23:43 2021-11-26 21:58:24 2021-11-26 22:32:26 0:34:02 0:13:11 0:20:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} 3
pass 6527623 2021-11-26 03:23:44 2021-11-26 21:59:32 2021-11-26 22:29:33 0:30:01 0:11:47 0:18:14 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
fail 6527624 2021-11-26 03:23:45 2021-11-26 22:02:20 2021-11-26 22:54:22 0:52:02 0:17:20 0:34:42 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

"2021-11-26T22:42:24.812435+0000 mon.b (mon.0) 142 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log