Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 6155566 2021-06-06 05:15:27 2021-06-23 10:17:00 2021-06-23 10:30:38 0:13:38 0:04:20 0:09:18 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_deep_flatten} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155567 2021-06-06 05:15:28 2021-06-23 10:17:00 2021-06-23 10:18:51 0:01:51 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{prefer-crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155568 2021-06-06 05:15:29 2021-06-23 10:17:51 2021-06-23 10:33:51 0:16:00 0:04:33 0:11:27 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_fio} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155569 2021-06-06 05:15:29 2021-06-23 10:18:52 2021-06-23 10:19:52 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155570 2021-06-06 05:15:30 2021-06-23 10:18:52 2021-06-23 10:21:03 0:02:11 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{prefer-crc} msgr-failures/few tasks/rbd_xfstests} 4
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155571 2021-06-06 05:15:31 2021-06-23 10:20:03 2021-06-23 10:36:21 0:16:18 0:04:06 0:12:12 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/backoff thrashosds-health workloads/rbd_fio} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155572 2021-06-06 05:15:32 2021-06-23 10:21:14 2021-06-23 10:22:15 0:01:01 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} 2
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155573 2021-06-06 05:15:32 2021-06-23 10:21:16 2021-06-23 10:34:12 0:12:56 0:03:51 0:09:05 gibba master krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155574 2021-06-06 05:15:33 2021-06-23 10:21:16 2021-06-23 10:22:26 0:01:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155575 2021-06-06 05:15:34 2021-06-23 10:21:26 2021-06-23 10:23:36 0:02:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155576 2021-06-06 05:15:35 2021-06-23 10:22:36 2021-06-23 10:37:59 0:15:23 0:04:09 0:11:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155577 2021-06-06 05:15:35 2021-06-23 10:22:57 2021-06-23 10:24:49 0:01:52 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155578 2021-06-06 05:15:36 2021-06-23 10:23:47 2021-06-23 10:39:35 0:15:48 0:04:04 0:11:44 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

fail 6155579 2021-06-06 05:15:37 2021-06-23 10:24:28 2021-06-23 10:39:29 0:15:01 0:04:06 0:10:55 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155580 2021-06-06 05:15:38 2021-06-23 10:24:29 2021-06-23 10:25:33 0:01:04 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_discard} 1
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155581 2021-06-06 05:15:38 2021-06-23 10:24:29 2021-06-23 10:26:10 0:01:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155582 2021-06-06 05:15:39 2021-06-23 10:25:00 2021-06-23 10:41:13 0:16:13 0:04:12 0:12:01 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155583 2021-06-06 05:15:40 2021-06-23 10:26:03 2021-06-23 10:27:29 0:01:26 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155584 2021-06-06 05:15:41 2021-06-23 10:26:14 2021-06-23 10:41:51 0:15:37 0:04:10 0:11:27 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155585 2021-06-06 05:15:41 2021-06-23 10:26:36 2021-06-23 10:28:46 0:02:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155586 2021-06-06 05:15:42 2021-06-23 10:27:40 2021-06-23 10:29:53 0:02:13 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155587 2021-06-06 05:15:43 2021-06-23 10:28:51 2021-06-23 10:44:34 0:15:43 0:04:08 0:11:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_symlinks} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155588 2021-06-06 05:15:44 2021-06-23 10:29:32 2021-06-23 10:31:10 0:01:38 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155589 2021-06-06 05:15:44 2021-06-23 10:30:04 2021-06-23 10:46:05 0:16:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

fail 6155590 2021-06-06 05:15:45 2021-06-23 10:31:04 2021-06-23 10:43:56 0:12:52 0:03:47 0:09:05 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_huge_image} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155591 2021-06-06 05:15:46 2021-06-23 10:31:05 2021-06-23 10:32:35 0:01:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155592 2021-06-06 05:15:47 2021-06-23 10:31:35 2021-06-23 10:33:45 0:02:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155593 2021-06-06 05:15:47 2021-06-23 10:32:45 2021-06-23 10:40:59 0:08:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

SSH connection to gibba029 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155594 2021-06-06 05:15:48 2021-06-23 10:33:06 2021-06-23 10:34:56 0:01:50 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_kernel} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155595 2021-06-06 05:15:49 2021-06-23 10:33:56 2021-06-23 10:40:59 0:07:03 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
Failure Reason:

Command failed on gibba010 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155596 2021-06-06 05:15:49 2021-06-23 10:33:56 2021-06-23 10:36:07 0:02:11 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155597 2021-06-06 05:15:50 2021-06-23 10:35:07 2021-06-23 10:54:30 0:19:23 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

fail 6155598 2021-06-06 05:15:51 2021-06-23 10:35:17 2021-06-23 10:48:15 0:12:58 0:03:44 0:09:14 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_msgr_segments} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155599 2021-06-06 05:15:52 2021-06-23 10:35:17 2021-06-23 10:37:09 0:01:52 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_unmap} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155600 2021-06-06 05:15:52 2021-06-23 10:36:09 2021-06-23 10:51:30 0:15:21 0:04:11 0:11:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155601 2021-06-06 05:15:53 2021-06-23 10:36:29 2021-06-23 10:38:20 0:01:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155602 2021-06-06 05:15:54 2021-06-23 10:37:20 2021-06-23 10:45:14 0:07:54 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Command failed on gibba020 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155603 2021-06-06 05:15:55 2021-06-23 10:37:30 2021-06-23 10:45:13 0:07:43 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

SSH connection to gibba028 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155604 2021-06-06 05:15:55 2021-06-23 10:38:11 2021-06-23 10:39:32 0:01:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155605 2021-06-06 05:15:56 2021-06-23 10:38:32 2021-06-23 10:54:09 0:15:37 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

dead 6155606 2021-06-06 05:15:57 2021-06-23 10:39:02 2021-06-23 10:40:34 0:01:32 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155607 2021-06-06 05:15:58 2021-06-23 10:39:32 2021-06-23 10:54:47 0:15:15 0:04:14 0:11:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

fail 6155608 2021-06-06 05:15:58 2021-06-23 10:39:43 2021-06-23 10:52:43 0:13:00 0:03:50 0:09:10 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_parent_overlap} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155609 2021-06-06 05:15:59 2021-06-23 10:39:43 2021-06-23 10:41:35 0:01:52 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155610 2021-06-06 05:16:00 2021-06-23 10:40:35 2021-06-23 10:50:25 0:09:50 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

SSH connection to gibba030 was lost: 'sudo lsb_release -is'

fail 6155611 2021-06-06 05:16:00 2021-06-23 10:40:36 2021-06-23 10:55:52 0:15:16 0:04:14 0:11:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155612 2021-06-06 05:16:01 2021-06-23 10:40:46 2021-06-23 10:50:45 0:09:59 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

SSH connection to gibba029 was lost: 'sudo lsb_release -is'

fail 6155613 2021-06-06 05:16:02 2021-06-23 10:41:06 2021-06-23 10:48:58 0:07:52 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Command failed on gibba030 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155614 2021-06-06 05:16:03 2021-06-23 10:41:27 2021-06-23 10:48:58 0:07:31 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

SSH connection to gibba042 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

fail 6155615 2021-06-06 05:16:03 2021-06-23 10:41:49 2021-06-23 10:54:45 0:12:56 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_read_only} 1
Failure Reason:

Could not reconnect to ubuntu@gibba029.front.sepia.ceph.com

dead 6155616 2021-06-06 05:16:04 2021-06-23 10:41:49 2021-06-23 10:43:00 0:01:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155617 2021-06-06 05:16:05 2021-06-23 10:41:59 2021-06-23 10:44:20 0:02:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155618 2021-06-06 05:16:06 2021-06-23 10:43:20 2021-06-23 10:44:21 0:01:01 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155619 2021-06-06 05:16:06 2021-06-23 10:43:20 2021-06-23 10:45:22 0:02:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_symlinks} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155620 2021-06-06 05:16:07 2021-06-23 10:44:22 2021-06-23 10:55:49 0:11:27 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/pggrow thrashosds-health workloads/rbd_fio} 3
Failure Reason:

SSH connection to gibba040 was lost: 'sudo dpkg -i /tmp/linux-image.deb'

dead 6155621 2021-06-06 05:16:08 2021-06-23 10:44:32 2021-06-23 11:02:02 0:17:30 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

dead 6155622 2021-06-06 05:16:09 2021-06-23 10:44:52 2021-06-23 10:46:23 0:01:31 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155623 2021-06-06 05:16:09 2021-06-23 10:45:23 2021-06-23 10:46:27 0:01:04 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba040

dead 6155624 2021-06-06 05:16:10 2021-06-23 10:45:23 2021-06-23 11:01:31 0:16:08 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

dead 6155625 2021-06-06 05:16:11 2021-06-23 10:46:23 2021-06-23 10:47:24 0:01:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155626 2021-06-06 05:16:12 2021-06-23 10:46:24 2021-06-23 10:59:21 0:12:57 0:03:46 0:09:11 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_whole_object_zeroout} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

fail 6155627 2021-06-06 05:16:12 2021-06-23 10:46:24 2021-06-23 10:59:54 0:13:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_kernel} 3
Failure Reason:

Could not reconnect to ubuntu@gibba040.front.sepia.ceph.com

dead 6155628 2021-06-06 05:16:13 2021-06-23 10:46:44 2021-06-23 10:48:35 0:01:51 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155629 2021-06-06 05:16:14 2021-06-23 10:47:35 2021-06-23 10:48:56 0:01:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155630 2021-06-06 05:16:14 2021-06-23 10:47:55 2021-06-23 10:49:46 0:01:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_unmap} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155631 2021-06-06 05:16:15 2021-06-23 10:48:45 2021-06-23 11:04:01 0:15:16 0:04:10 0:11:06 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155632 2021-06-06 05:16:16 2021-06-23 10:49:06 2021-06-23 10:50:07 0:01:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

pass 6155633 2021-06-06 05:16:17 2021-06-10 19:45:11 2021-06-10 20:02:09 0:16:58 0:07:18 0:09:40 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_deep_flatten} 1
dead 6155634 2021-06-06 05:16:17 2021-06-23 10:49:06 2021-06-23 10:59:43 0:10:37 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
Failure Reason:

SSH connection to gibba010 was lost: 'sudo lsb_release -is'

dead 6155635 2021-06-06 05:16:18 2021-06-23 10:49:36 2021-06-23 10:50:57 0:01:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_fio} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155636 2021-06-06 05:16:19 2021-06-23 10:49:57 2021-06-23 10:51:17 0:01:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155637 2021-06-06 05:16:20 2021-06-23 10:50:17 2021-06-23 11:03:33 0:13:16 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} 2
Failure Reason:

Could not reconnect to ubuntu@gibba010.front.sepia.ceph.com

fail 6155638 2021-06-06 05:16:20 2021-06-23 10:50:37 2021-06-23 10:58:36 0:07:59 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} 3
Failure Reason:

Command failed on gibba005 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155639 2021-06-06 05:16:21 2021-06-23 10:50:48 2021-06-23 10:52:09 0:01:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_exclusive_option} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155640 2021-06-06 05:16:22 2021-06-23 10:51:08 2021-06-23 10:52:29 0:01:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155641 2021-06-06 05:16:23 2021-06-23 10:51:29 2021-06-23 10:58:36 0:07:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_fallocate} 3
Failure Reason:

SSH connection to gibba036 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155642 2021-06-06 05:16:23 2021-06-23 10:51:39 2021-06-23 10:53:20 0:01:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155643 2021-06-06 05:16:24 2021-06-23 10:52:19 2021-06-23 10:53:40 0:01:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155644 2021-06-06 05:16:25 2021-06-23 10:52:40 2021-06-23 10:54:31 0:01:51 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_namespaces} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155645 2021-06-06 05:16:26 2021-06-23 10:53:30 2021-06-23 11:03:07 0:09:37 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
Failure Reason:

Command failed on gibba029 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155646 2021-06-06 05:16:26 2021-06-23 10:53:51 2021-06-23 10:54:51 0:01:00 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_discard} 1
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155647 2021-06-06 05:16:27 2021-06-23 10:53:51 2021-06-23 11:09:14 0:15:23 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_enumerate} 3
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

fail 6155648 2021-06-06 05:16:28 2021-06-23 10:54:11 2021-06-23 11:09:42 0:15:31 0:04:09 0:11:22 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

pass 6155649 2021-06-06 05:16:28 2021-06-10 19:45:11 2021-06-10 20:27:19 0:42:08 0:14:33 0:27:35 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
dead 6155650 2021-06-06 05:16:29 2021-06-23 10:54:42 2021-06-23 10:55:42 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155651 2021-06-06 05:16:30 2021-06-23 10:54:42 2021-06-23 10:55:52 0:01:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155652 2021-06-06 05:16:31 2021-06-23 10:54:52 2021-06-23 11:03:11 0:08:19 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

Command failed on gibba002 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

fail 6155653 2021-06-06 05:16:31 2021-06-23 10:55:12 2021-06-23 11:11:00 0:15:48 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_symlinks} 3
Failure Reason:

Could not reconnect to ubuntu@gibba008.front.sepia.ceph.com

dead 6155654 2021-06-06 05:16:32 2021-06-23 10:55:13 2021-06-23 10:56:53 0:01:40 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155655 2021-06-06 05:16:33 2021-06-23 10:55:53 2021-06-23 11:03:07 0:07:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_concurrent} 3
Failure Reason:

SSH connection to gibba031 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155656 2021-06-06 05:16:34 2021-06-23 10:56:05 2021-06-23 11:03:11 0:07:06 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_huge_image} 1
Failure Reason:

SSH connection to gibba029 was lost: 'sudo dpkg -i /tmp/linux-image.deb'

dead 6155657 2021-06-06 05:16:34 2021-06-23 10:56:05 2021-06-23 11:15:12 0:19:07 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

dead 6155658 2021-06-06 05:16:35 2021-06-23 10:56:05 2021-06-23 10:58:06 0:02:01 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155659 2021-06-06 05:16:36 2021-06-23 10:57:06 2021-06-23 11:05:51 0:08:45 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_image_read} 3
Failure Reason:

Command failed on gibba005 with status 128: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155660 2021-06-06 05:16:37 2021-06-23 10:58:07 2021-06-23 10:59:18 0:01:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_kernel} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155661 2021-06-06 05:16:37 2021-06-23 10:58:18 2021-06-23 11:05:52 0:07:34 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

SSH connection to gibba005 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155662 2021-06-06 05:16:38 2021-06-23 10:58:48 2021-06-23 11:00:28 0:01:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155663 2021-06-06 05:16:39 2021-06-23 10:59:28 2021-06-23 11:14:47 0:15:19 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_msgr_segments} 1
Failure Reason:

Could not reconnect to ubuntu@gibba041.front.sepia.ceph.com

dead 6155664 2021-06-06 05:16:40 2021-06-23 10:59:29 2021-06-23 11:00:49 0:01:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_unmap} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155665 2021-06-06 05:16:40 2021-06-23 10:59:49 2021-06-23 11:09:20 0:09:31 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
dead 6155666 2021-06-06 05:16:41 2021-06-23 10:59:59 2021-06-23 11:01:40 0:01:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155667 2021-06-06 05:16:42 2021-06-23 11:00:39 2021-06-23 11:02:00 0:01:21 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155668 2021-06-06 05:16:42 2021-06-23 11:01:00 2021-06-23 11:10:48 0:09:48 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

SSH connection to gibba041 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155669 2021-06-06 05:16:43 2021-06-23 11:01:40 2021-06-23 11:16:54 0:15:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds

dead 6155670 2021-06-06 05:16:44 2021-06-23 11:01:50 2021-06-23 11:03:11 0:01:21 gibba master krbd/singleton/{bluestore-bitmap conf ms_mode$/{crc} msgr-failures/many tasks/rbd_xfstests} 4
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155671 2021-06-06 05:16:45 2021-06-23 11:02:11 2021-06-23 11:11:01 0:08:50 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{secure} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba029.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6155672 2021-06-06 05:16:46 2021-06-23 11:02:21 2021-06-23 11:09:22 0:07:01 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_exclusive_option} 3
Failure Reason:

Command failed on gibba020 with status 128: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155673 2021-06-06 05:16:46 2021-06-23 11:02:21 2021-06-23 11:04:12 0:01:51 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155674 2021-06-06 05:16:47 2021-06-23 11:03:12 2021-06-23 11:18:15 0:15:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

failed to install new kernel version within timeout

fail 6155675 2021-06-06 05:16:48 2021-06-23 11:03:12 2021-06-23 11:16:12 0:13:00 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_parent_overlap} 1
Failure Reason:

failed to install new kernel version within timeout

dead 6155676 2021-06-06 05:16:49 2021-06-23 11:03:12 2021-06-23 11:04:44 0:01:32 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155677 2021-06-06 05:16:49 2021-06-23 11:03:44 2021-06-23 11:15:17 0:11:33 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba010.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155678 2021-06-06 05:16:50 2021-06-23 11:03:44 2021-06-23 11:05:17 0:01:33 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155679 2021-06-06 05:16:51 2021-06-23 11:04:14 2021-06-23 11:12:16 0:08:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_enumerate} 3
Failure Reason:

SSH connection to gibba042 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155680 2021-06-06 05:16:52 2021-06-23 11:04:25 2021-06-23 11:05:45 0:01:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155681 2021-06-06 05:16:52 2021-06-23 11:04:45 2021-06-23 11:16:27 0:11:42 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

SSH connection to gibba038 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

fail 6155682 2021-06-06 05:16:53 2021-06-23 11:05:15 2021-06-23 11:18:13 0:12:58 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_read_only} 1
Failure Reason:

Could not reconnect to ubuntu@gibba042.front.sepia.ceph.com

dead 6155683 2021-06-06 05:16:54 2021-06-23 11:05:16 2021-06-23 11:06:26 0:01:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_netns} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155684 2021-06-06 05:16:55 2021-06-23 11:05:26 2021-06-23 11:12:52 0:07:26 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
Failure Reason:

SSH connection to gibba023 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155685 2021-06-06 05:16:55 2021-06-23 11:05:56 2021-06-23 11:06:57 0:01:01 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{secure} objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155686 2021-06-06 05:16:56 2021-06-23 11:05:57 2021-06-23 11:07:37 0:01:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155687 2021-06-06 05:16:57 2021-06-23 11:06:37 2021-06-23 11:08:07 0:01:30 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155688 2021-06-06 05:16:57 2021-06-23 11:07:07 2021-06-23 11:08:48 0:01:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_concurrent} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155689 2021-06-06 05:16:58 2021-06-23 11:07:48 2021-06-23 11:09:18 0:01:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155690 2021-06-06 05:16:59 2021-06-23 11:08:18 2021-06-23 11:09:58 0:01:40 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155691 2021-06-06 05:17:00 2021-06-23 11:08:58 2021-06-23 11:10:19 0:01:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_image_read} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155692 2021-06-06 05:17:00 2021-06-23 11:09:19 2021-06-23 11:20:38 0:11:19 0:01:58 0:09:21 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_whole_object_zeroout} 1
Failure Reason:

{'gibba020.front.sepia.ceph.com': {'changed': False, 'msg': 'Data could not be sent to remote host "gibba020.front.sepia.ceph.com". Make sure this host can be reached over ssh: Warning: Permanently added \'gibba020.front.sepia.ceph.com,172.21.2.120\' (ECDSA) to the list of known hosts.\r\nubuntu@gibba020.front.sepia.ceph.com: Permission denied (publickey,password,keyboard-interactive).\r\n', 'unreachable': True}}

dead 6155693 2021-06-06 05:17:01 2021-06-23 11:09:19 2021-06-23 11:16:33 0:07:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

SSH connection to gibba013 was lost: 'cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master'

dead 6155694 2021-06-06 05:17:02 2021-06-23 11:09:29 2021-06-23 11:18:05 0:08:36 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{crc} thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba005.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6155695 2021-06-06 05:17:03 2021-06-23 11:09:29 2021-06-23 11:24:43 0:15:14 0:04:02 0:11:12 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155696 2021-06-06 05:17:03 2021-06-23 11:09:50 2021-06-23 11:11:00 0:01:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155697 2021-06-06 05:17:04 2021-06-23 11:10:00 2021-06-23 11:17:10 0:07:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

SSH connection to gibba027 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155698 2021-06-06 05:17:05 2021-06-23 11:10:10 2021-06-23 11:11:31 0:01:21 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155699 2021-06-06 05:17:06 2021-06-23 11:10:31 2021-06-23 11:18:18 0:07:47 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba029.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6155700 2021-06-06 05:17:06 2021-06-23 11:11:11 2021-06-23 11:24:04 0:12:53 0:03:43 0:09:10 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_deep_flatten} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

fail 6155701 2021-06-06 05:17:07 2021-06-23 11:11:12 2021-06-23 11:30:22 0:19:10 0:04:01 0:15:09 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{legacy} objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155702 2021-06-06 05:17:08 2021-06-23 11:11:12 2021-06-23 11:12:32 0:01:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_fio} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155703 2021-06-06 05:17:09 2021-06-23 11:11:32 2021-06-23 11:12:42 0:01:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155704 2021-06-06 05:17:09 2021-06-23 11:11:43 2021-06-23 11:26:11 0:14:28 gibba master krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} 2
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba010.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155705 2021-06-06 05:17:10 2021-06-23 11:12:23 2021-06-23 11:13:43 0:01:20 gibba master krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155706 2021-06-06 05:17:11 2021-06-23 11:12:43 2021-06-23 11:13:43 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_exclusive_option} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155707 2021-06-06 05:17:12 2021-06-23 11:12:44 2021-06-23 11:20:31 0:07:47 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: 'NoneType' object has no attribute 'get_transport'

dead 6155708 2021-06-06 05:17:13 2021-06-23 11:12:54 2021-06-23 11:18:56 0:06:02 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba023.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155709 2021-06-06 05:17:13 2021-06-23 11:13:14 2021-06-23 11:22:07 0:08:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba035.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155710 2021-06-06 05:17:14 2021-06-23 11:13:55 2021-06-23 11:14:55 0:01:00 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155711 2021-06-06 05:17:15 2021-06-23 11:13:55 2021-06-23 11:16:05 0:02:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155712 2021-06-06 05:17:16 2021-06-23 11:15:05 2021-06-23 11:21:57 0:06:52 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_discard} 1
Failure Reason:

SSH connection to gibba041 was lost: 'sudo dpkg -i /tmp/linux-image.deb'

dead 6155713 2021-06-06 05:17:16 2021-06-23 11:15:05 2021-06-23 11:22:27 0:07:22 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_enumerate} 3
Failure Reason:

Error reimaging machines: 'NoneType' object has no attribute 'get_transport'

dead 6155714 2021-06-06 05:17:17 2021-06-23 11:15:26 2021-06-23 11:34:16 0:18:50 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

dead 6155715 2021-06-06 05:17:18 2021-06-23 11:15:26 2021-06-23 11:17:06 0:01:40 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155716 2021-06-06 05:17:19 2021-06-23 11:16:06 2021-06-23 11:26:18 0:10:12 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_udev_netns} 3
Failure Reason:

SSH connection to gibba013 was lost: 'sudo lsb_release -is'

dead 6155717 2021-06-06 05:17:19 2021-06-23 11:16:17 2021-06-23 11:33:17 0:17:00 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress} 3
dead 6155718 2021-06-06 05:17:20 2021-06-23 11:16:37 2021-06-23 11:18:07 0:01:30 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155719 2021-06-06 05:17:21 2021-06-23 11:17:07 2021-06-23 11:33:49 0:16:42 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

dead 6155720 2021-06-06 05:17:22 2021-06-23 11:17:07 2021-06-23 11:27:57 0:10:50 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{prefer-crc} thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} 3
Failure Reason:

SSH connection to gibba002 was lost: 'sudo lsb_release -is'

dead 6155721 2021-06-06 05:17:22 2021-06-23 11:17:09 2021-06-23 11:24:21 0:07:12 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} 3
Failure Reason:

SSH connection to gibba027 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

fail 6155722 2021-06-06 05:17:23 2021-06-23 11:17:19 2021-06-23 11:31:22 0:14:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_concurrent} 3
Failure Reason:

Stale jobs detected, aborting.

fail 6155723 2021-06-06 05:17:24 2021-06-23 11:18:19 2021-06-23 11:33:36 0:15:17 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_huge_image} 1
Failure Reason:

Could not reconnect to ubuntu@gibba020.front.sepia.ceph.com

dead 6155724 2021-06-06 05:17:25 2021-06-23 11:18:20 2021-06-23 11:19:20 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155725 2021-06-06 05:17:25 2021-06-23 11:18:20 2021-06-23 11:34:06 0:15:46 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Could not reconnect to ubuntu@gibba002.front.sepia.ceph.com

dead 6155726 2021-06-06 05:17:26 2021-06-23 11:18:30 2021-06-23 11:19:40 0:01:10 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_image_read} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

fail 6155727 2021-06-06 05:17:27 2021-06-23 11:18:40 2021-06-23 11:37:43 0:19:03 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_kernel} 1
Failure Reason:

Could not reconnect to ubuntu@gibba035.front.sepia.ceph.com

dead 6155728 2021-06-06 05:17:28 2021-06-23 11:19:11 2021-06-23 11:20:31 0:01:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_iozone} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155729 2021-06-06 05:17:28 2021-06-23 11:19:31 2021-06-23 11:20:51 0:01:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155730 2021-06-06 05:17:29 2021-06-23 11:19:51 2021-06-23 11:21:31 0:01:40 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_msgr_segments} 1
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155731 2021-06-06 05:17:30 2021-06-23 11:20:32 2021-06-23 11:27:46 0:07:14 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba020.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155732 2021-06-06 05:17:31 2021-06-23 11:20:42 2021-06-23 11:21:52 0:01:10 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155733 2021-06-06 05:17:31 2021-06-23 11:20:52 2021-06-23 11:49:34 0:28:42 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds

dead 6155734 2021-06-06 05:17:32 2021-06-23 11:21:03 2021-06-23 11:23:03 0:02:00 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{legacy} objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155735 2021-06-06 05:17:33 2021-06-23 11:22:03 2021-06-23 11:23:23 0:01:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_fio} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155736 2021-06-06 05:17:34 2021-06-23 11:22:23 2021-06-23 11:33:04 0:10:41 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_data_pool} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba027.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155737 2021-06-06 05:17:34 2021-06-23 11:22:45 2021-06-23 11:24:15 0:01:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_exclusive_option} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155738 2021-06-06 05:17:35 2021-06-23 11:23:15 2021-06-23 11:24:35 0:01:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_kernel_untar_build} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155739 2021-06-06 05:17:36 2021-06-23 11:23:35 2021-06-23 11:35:04 0:11:29 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_fallocate} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba015.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6155740 2021-06-06 05:17:37 2021-06-23 11:24:16 2021-06-23 11:37:14 0:12:58 0:03:40 0:09:18 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/crc tasks/krbd_parent_overlap} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

dead 6155741 2021-06-06 05:17:37 2021-06-23 11:24:16 2021-06-23 11:29:42 0:05:26 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/krbd_latest_osdmap_on_map} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba045.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155742 2021-06-06 05:17:38 2021-06-23 11:24:26 2021-06-23 11:31:46 0:07:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_suites_dbench} 3
Failure Reason:

SSH connection to gibba035 was lost: "cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init"

dead 6155743 2021-06-06 05:17:39 2021-06-23 11:24:47 2021-06-23 11:25:47 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/krbd_namespaces} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155744 2021-06-06 05:17:40 2021-06-23 11:24:47 2021-06-23 11:35:22 0:10:35 gibba master krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/upmap thrashosds-health workloads/rbd_fio} 3
Failure Reason:

SSH connection to gibba025 was lost: 'sudo dpkg -i /tmp/linux-image.deb'

dead 6155745 2021-06-06 05:17:40 2021-06-23 11:25:47 2021-06-23 11:27:17 0:01:30 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/krbd_udev_enumerate} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155746 2021-06-06 05:17:41 2021-06-23 11:26:17 2021-06-23 11:35:26 0:09:09 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/few tasks/rbd_workunit_suites_ffsb} 3
Failure Reason:

Command failed on gibba025 with status 2: 'sudo dpkg -i /tmp/linux-image.deb'

dead 6155747 2021-06-06 05:17:42 2021-06-23 11:26:28 2021-06-23 11:28:18 0:01:50 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155748 2021-06-06 05:17:42 2021-06-23 11:27:18 2021-06-23 11:40:26 0:13:08 0:03:39 0:09:29 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/legacy tasks/krbd_read_only} 1
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=e48b5814dafdd4f3719d8c8a2d7f386676cd9bdd

fail 6155749 2021-06-06 05:17:43 2021-06-23 11:27:28 2021-06-23 11:41:12 0:13:44 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netns} 1
Failure Reason:

Command failed on gibba030 with status 1: 'sudo yum upgrade -y linux-firmware'

fail 6155750 2021-06-06 05:17:44 2021-06-23 11:27:59 2021-06-23 11:41:32 0:13:33 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/many tasks/rbd_workunit_suites_fsstress} 1
Failure Reason:

Command failed on gibba010 with status 1: 'sudo yum upgrade -y linux-firmware'

dead 6155751 2021-06-06 05:17:45 2021-06-23 11:27:59 2021-06-23 11:29:30 0:01:31 gibba master krbd/fsx/{ceph/ceph clusters/3-node conf ms_mode$/{crc} objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155752 2021-06-06 05:17:46 2021-06-23 11:28:30 2021-06-23 11:30:50 0:02:20 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/krbd_udev_symlinks} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034

dead 6155753 2021-06-06 05:17:46 2021-06-23 11:29:51 2021-06-23 11:41:57 0:12:06 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} 2
Failure Reason:

SSH connection to gibba015 was lost: 'rm -f /tmp/kernel.x86_64.rpm && echo kernel-5.13.0_rc4_ceph_g88d0c6ef8cc4-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/88d0c6ef8cc498e14e85ea946931b4a50bd04f7b/centos/8/flavors/default/x86_64/ --input-file=-'

dead 6155754 2021-06-06 05:17:47 2021-06-23 11:29:51 2021-06-23 11:43:57 0:14:06 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_concurrent} 1
Failure Reason:

SSH connection to gibba015 was lost: 'sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm'

dead 6155755 2021-06-06 05:17:48 2021-06-23 11:30:01 2021-06-23 11:41:54 0:11:53 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/many tasks/rbd_huge_tickets} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba004.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

dead 6155756 2021-06-06 05:17:49 2021-06-23 11:30:32 2021-06-23 11:32:02 0:01:30 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/legacy msgr-failures/many tasks/rbd_workunit_suites_fsx} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155757 2021-06-06 05:17:49 2021-06-23 11:31:02 2021-06-23 11:36:58 0:05:56 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/few tasks/rbd_image_read} 3
Failure Reason:

Error reimaging machines: 'ssh_keyscan gibba035.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds

fail 6155758 2021-06-06 05:17:50 2021-06-23 11:31:52 2021-06-23 11:46:28 0:14:36 gibba master krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf ms_mode/secure tasks/krbd_whole_object_zeroout}
Failure Reason:

Could not reconnect to ubuntu@gibba005.front.sepia.ceph.com

dead 6155759 2021-06-06 05:17:51 2021-06-23 11:31:52 2021-06-23 11:33:13 0:01:21 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_kernel} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

fail 6155760 2021-06-06 05:17:52 2021-06-23 11:32:13 2021-06-23 11:48:33 0:16:20 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_suites_iozone}
Failure Reason:

Could not reconnect to ubuntu@gibba004.front.sepia.ceph.com

dead 6155761 2021-06-06 05:17:52 2021-06-23 11:33:23 2021-06-23 11:34:23 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/rbd_map_snapshot_io} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba039

dead 6155762 2021-06-06 05:17:53 2021-06-23 11:33:23 2021-06-23 11:42:34 0:09:11 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy msgr-failures/many tasks/rbd_map_unmap} 3
Failure Reason:

SSH connection to gibba005 was lost: 'sudo lsb_release -is'

dead 6155763 2021-06-06 05:17:54 2021-06-23 11:33:34 2021-06-23 11:54:27 0:20:53 0:06:07 0:14:46 gibba master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc msgr-failures/many tasks/rbd_workunit_trivial_sync} 1
Failure Reason:

{'Failure object was': {'gibba025.front.sepia.ceph.com': {'results': [{'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'vg': 'vg_nvme', 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'pesize': '4', 'pv_options': '', 'vg_options': '', 'force': False}}, '_ansible_no_log': False, 'changed': False, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'ansible_loop_var': 'item', '_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}}], 'msg': 'All items completed', 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'key')", 'gibba042.front.sepia.ceph.com': {'changed': False, 'msg': 'Data could not be sent to remote host "gibba042.front.sepia.ceph.com". Make sure this host can be reached over ssh: ssh: connect to host gibba042.front.sepia.ceph.com port 22: No route to host\r\n', 'unreachable': True}}

dead 6155764 2021-06-06 05:17:55 2021-06-23 11:33:54 2021-06-23 11:34:54 0:01:00 gibba master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/few tasks/rbd_simple_big} 3
Failure Reason:

Error reimaging machines: Failed to power on gibba034