Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 4252945 2019-08-26 03:20:26 2019-09-01 03:38:20 2019-09-01 03:56:20 0:18:00 0:09:25 0:08:35 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_blkroset.yaml} 1
Failure Reason:

"2019-09-01T03:53:18.782848+0000 mon.a (mon.0) 98 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252946 2019-08-26 03:20:26 2019-09-01 03:38:21 2019-09-01 04:36:20 0:57:59 0:29:11 0:28:48 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

"2019-09-01T04:31:48.065028+0000 mon.a (mon.0) 444 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252947 2019-08-26 03:20:27 2019-09-01 03:40:38 2019-09-01 04:32:37 0:51:59 0:30:29 0:21:30 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_fio.yaml} 3
Failure Reason:

"2019-09-01T04:23:04.609532+0000 mon.b (mon.0) 202 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252948 2019-08-26 03:20:28 2019-09-01 03:42:13 2019-09-01 04:56:13 1:14:00 0:22:42 0:51:18 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_data_pool.yaml} 3
Failure Reason:

"2019-09-01T04:52:10.754672+0000 mon.a (mon.0) 195 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252949 2019-08-26 03:20:28 2019-09-01 03:42:13 2019-09-01 08:48:18 5:06:05 2:09:33 2:56:32 smithi master krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_xfstests.yaml} 4
Failure Reason:

Command failed on smithi035 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludesuzvD5 -r -- '-g auto -g blockdev -x clone'"

fail 4252950 2019-08-26 03:20:29 2019-09-01 03:42:14 2019-09-01 05:34:15 1:52:01 0:56:01 0:56:00 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} 3
Failure Reason:

Command failed on smithi187 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252951 2019-08-26 03:20:30 2019-09-01 03:42:18 2019-09-01 03:50:17 0:07:59 smithi master krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/pre-single-major.yaml tasks/unmap.yaml} 2
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=nightly_pre-single-major

fail 4252952 2019-08-26 03:20:31 2019-09-01 03:44:26 2019-09-01 04:04:25 0:19:59 0:09:01 0:10:58 smithi master krbd/wac/sysfs/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/stable_pages_required.yaml} 1
Failure Reason:

Command failed on smithi109 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252953 2019-08-26 03:20:31 2019-09-01 03:44:26 2019-09-01 04:32:25 0:47:59 0:14:31 0:33:28 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_exclusive_option.yaml} 3
Failure Reason:

Command failed (workunit test rbd/krbd_exclusive_option.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=732fe579b6f95da40a003c6ff431845c9b7eabba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/krbd_exclusive_option.sh'

fail 4252954 2019-08-26 03:20:32 2019-09-01 03:44:26 2019-09-01 05:00:26 1:16:00 0:15:32 1:00:28 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_kernel_untar_build.yaml} 3
Failure Reason:

Command failed on smithi003 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

dead 4252955 2019-08-26 03:20:33 2019-09-01 03:46:19 2019-09-01 15:48:42 12:02:23 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} 3
fail 4252956 2019-08-26 03:20:33 2019-09-01 03:46:19 2019-09-01 05:02:19 1:16:00 0:11:02 1:04:58 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_fallocate.yaml} 3
Failure Reason:

Command failed on smithi173 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252957 2019-08-26 03:20:34 2019-09-01 03:48:13 2019-09-01 05:48:14 2:00:01 0:41:20 1:18:41 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_dbench.yaml} 3
Failure Reason:

Command failed on smithi151 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252958 2019-08-26 03:20:35 2019-09-01 03:48:13 2019-09-01 04:58:13 1:10:00 0:15:17 0:54:43 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_latest_osdmap_on_map.yaml} 3
Failure Reason:

Command failed on smithi151 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252959 2019-08-26 03:20:36 2019-09-01 06:24:42 2019-09-01 09:46:44 3:22:02 0:36:49 2:45:13 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2019-09-01T09:15:06.065450+0000 mon.b (mon.1) 97 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

fail 4252960 2019-08-26 03:20:37 2019-09-01 06:24:42 2019-09-01 06:44:41 0:19:59 0:10:44 0:09:15 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_deep_flatten.yaml} 1
Failure Reason:

Command failed on smithi086 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252961 2019-08-26 03:20:37 2019-09-01 06:26:41 2019-09-01 07:34:41 1:08:00 0:35:48 0:32:12 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

Command failed on smithi175 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252962 2019-08-26 03:20:38 2019-09-01 06:26:41 2019-09-01 08:58:43 2:32:02 0:45:26 1:46:36 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2019-09-01T08:46:50.786131+0000 mon.a (mon.0) 249 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

fail 4252963 2019-08-26 03:20:39 2019-09-01 06:26:41 2019-09-01 07:16:40 0:49:59 0:15:42 0:34:17 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_namespaces.yaml} 3
Failure Reason:

"2019-09-01T07:13:28.255517+0000 mon.a (mon.0) 143 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252964 2019-08-26 03:20:39 2019-09-01 06:26:41 2019-09-01 07:40:41 1:14:00 0:16:54 0:57:06 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_concurrent.yaml} 3
Failure Reason:

"2019-09-01T07:36:31.706540+0000 mon.b (mon.0) 196 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252965 2019-08-26 03:20:40 2019-09-01 06:26:41 2019-09-01 07:50:41 1:24:00 0:10:32 1:13:28 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress.yaml} 3
Failure Reason:

"2019-09-01T07:48:25.567069+0000 mon.a (mon.0) 134 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252966 2019-08-26 03:20:41 2019-09-01 06:28:44 2019-09-01 07:14:44 0:46:00 0:27:05 0:18:55 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} 3
Failure Reason:

Command failed on smithi172 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252967 2019-08-26 03:20:42 2019-09-01 06:28:45 2019-09-01 07:04:44 0:35:59 0:10:41 0:25:18 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_huge_tickets.yaml} 3
Failure Reason:

"2019-09-01T07:00:11.090005+0000 mon.b (mon.0) 148 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 4252968 2019-08-26 03:20:42 2019-09-01 06:28:45 2019-09-01 07:58:45 1:30:00 1:00:00 0:30:00 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} 3
Failure Reason:

Command failed on smithi088 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252969 2019-08-26 03:20:43 2019-09-01 06:28:45 2019-09-01 07:26:44 0:57:59 0:11:55 0:46:04 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} 3
Failure Reason:

Command failed on smithi137 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252970 2019-08-26 03:20:44 2019-09-01 06:30:08 2019-09-01 06:50:07 0:19:59 0:10:37 0:09:22 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_discard.yaml} 1
Failure Reason:

Command failed on smithi154 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252971 2019-08-26 03:20:45 2019-09-01 06:30:34 2019-09-01 07:16:33 0:45:59 0:11:06 0:34:53 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_image_read.yaml} 3
Failure Reason:

Command failed on smithi103 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252972 2019-08-26 03:20:45 2019-09-01 06:30:40 2019-09-01 07:26:40 0:56:00 0:33:17 0:22:43 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

"2019-09-01T07:18:56.775514+0000 mon.a (mon.0) 561 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

fail 4252973 2019-08-26 03:20:46 2019-09-01 06:31:24 2019-09-01 07:05:23 0:33:59 0:19:20 0:14:39 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsx.yaml} 3
Failure Reason:

Command failed on smithi121 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252974 2019-08-26 03:20:47 2019-09-01 06:34:42 2019-09-01 07:08:41 0:33:59 0:11:42 0:22:17 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_kernel.yaml} 3
Failure Reason:

Command failed on smithi049 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252975 2019-08-26 03:20:48 2019-09-01 06:34:42 2019-09-01 06:56:41 0:21:59 0:10:45 0:11:14 smithi master krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-off.yaml tasks/unmap.yaml} 2
Failure Reason:

Command failed on smithi025 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252976 2019-08-26 03:20:48 2019-09-01 06:34:42 2019-09-01 08:10:43 1:36:01 0:22:49 1:13:12 smithi master krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/many-resets.yaml} 3
Failure Reason:

"2019-09-01T08:05:46.370656+0000 mon.a (mon.0) 174 : cluster [WRN] Health check failed: 6 osds down (OSD_DOWN)" in cluster log

fail 4252977 2019-08-26 03:20:49 2019-09-01 06:34:46 2019-09-01 07:22:45 0:47:59 0:12:00 0:35:59 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_snapshot_io.yaml} 3
Failure Reason:

Command failed on smithi162 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252978 2019-08-26 03:20:50 2019-09-01 06:36:30 2019-09-01 10:30:34 3:54:04 3:41:32 0:12:32 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

Command failed (workunit test suites/ffsb.sh) on smithi016 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=732fe579b6f95da40a003c6ff431845c9b7eabba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

fail 4252979 2019-08-26 03:20:51 2019-09-01 06:36:30 2019-09-01 07:10:30 0:34:00 0:14:19 0:19:41 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_iozone.yaml} 3
Failure Reason:

"2019-09-01T07:07:32.435221+0000 mon.a (mon.0) 161 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252980 2019-08-26 03:20:51 2019-09-01 06:36:30 2019-09-01 07:22:30 0:46:00 0:29:12 0:16:48 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} 3
Failure Reason:

"2019-09-01T07:15:05.049702+0000 mon.a (mon.0) 559 : cluster [WRN] Health check failed: 6 osds down (OSD_DOWN)" in cluster log

fail 4252981 2019-08-26 03:20:52 2019-09-01 06:36:31 2019-09-01 07:36:31 1:00:00 0:17:54 0:42:06 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_unmap.yaml} 3
Failure Reason:

"2019-09-01T07:32:50.782438+0000 mon.a (mon.0) 180 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 4252982 2019-08-26 03:20:53 2019-09-01 06:36:48 2019-09-01 18:39:12 12:02:24 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_huge_image.yaml} 1
fail 4252983 2019-08-26 03:20:54 2019-09-01 06:37:16 2019-09-01 07:33:16 0:56:00 0:13:29 0:42:31 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_trivial_sync.yaml} 3
Failure Reason:

Command failed on smithi041 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252984 2019-08-26 03:20:54 2019-09-01 06:38:53 2019-09-01 07:40:53 1:02:00 0:26:34 0:35:26 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_simple_big.yaml} 3
Failure Reason:

"2019-09-01T07:34:25.698452+0000 mon.b (mon.0) 134 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252985 2019-08-26 03:20:55 2019-09-01 06:38:53 2019-09-01 08:12:54 1:34:01 0:28:02 1:05:59 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

Command failed on smithi136 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252986 2019-08-26 03:20:56 2019-09-01 06:38:53 2019-09-01 07:36:53 0:58:00 0:30:43 0:27:17 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_fio.yaml} 3
Failure Reason:

"2019-09-01T07:28:58.036020+0000 mon.b (mon.0) 174 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252987 2019-08-26 03:20:57 2019-09-01 06:40:44 2019-09-01 08:18:44 1:38:00 0:26:00 1:12:00 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_data_pool.yaml} 3
Failure Reason:

"2019-09-01T08:14:39.867542+0000 mon.a (mon.0) 249 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252988 2019-08-26 03:20:57 2019-09-01 06:40:43 2019-09-01 10:34:47 3:54:04 2:41:54 1:12:10 smithi master krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_xfstests.yaml} 4
Failure Reason:

Command failed on smithi013 with status 1: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeARz8tZ -r -- '-g auto -g blockdev -x clone'"

fail 4252989 2019-08-26 03:20:58 2019-09-01 06:40:44 2019-09-01 08:56:45 2:16:01 0:45:52 1:30:09 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

Command failed on smithi088 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252990 2019-08-26 03:20:59 2019-09-01 06:40:44 2019-09-01 07:08:43 0:27:59 0:15:19 0:12:40 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_exclusive_option.yaml} 3
Failure Reason:

Command failed (workunit test rbd/krbd_exclusive_option.sh) on smithi203 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=732fe579b6f95da40a003c6ff431845c9b7eabba TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/krbd_exclusive_option.sh'

fail 4252991 2019-08-26 03:21:00 2019-09-01 06:40:44 2019-09-01 09:36:46 2:56:02 0:15:44 2:40:18 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_kernel_untar_build.yaml} 3
Failure Reason:

"2019-09-01T09:32:46.702725+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252992 2019-08-26 03:21:01 2019-09-01 06:42:13 2019-09-01 07:02:12 0:19:59 0:09:54 0:10:05 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_msgr_segments.yaml} 1
Failure Reason:

Command failed on smithi044 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252993 2019-08-26 03:21:01 2019-09-01 06:42:38 2019-09-01 08:14:38 1:32:00 0:31:46 1:00:14 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} 3
Failure Reason:

"2019-09-01T08:07:22.002533+0000 mon.a (mon.0) 571 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252994 2019-08-26 03:21:02 2019-09-01 06:42:39 2019-09-01 07:14:38 0:31:59 0:13:31 0:18:28 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_fallocate.yaml} 3
Failure Reason:

Command failed on smithi118 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252995 2019-08-26 03:21:03 2019-09-01 06:42:39 2019-09-01 08:02:40 1:20:01 0:39:19 0:40:42 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_dbench.yaml} 3
Failure Reason:

"2019-09-01T07:56:13.099865+0000 mon.b (mon.0) 169 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4252996 2019-08-26 03:21:04 2019-09-01 06:42:54 2019-09-01 09:46:56 3:04:02 0:16:05 2:47:57 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_latest_osdmap_on_map.yaml} 3
Failure Reason:

Command failed on smithi191 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252997 2019-08-26 03:21:05 2019-09-01 06:44:42 2019-09-01 08:04:42 1:20:00 1:02:28 0:17:32 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} 3
Failure Reason:

"2019-09-01T07:08:11.728617+0000 mon.a (mon.1) 30 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 4252998 2019-08-26 03:21:05 2019-09-01 06:44:42 2019-09-01 07:36:42 0:52:00 0:36:25 0:15:35 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

Command failed on smithi182 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4252999 2019-08-26 03:21:06 2019-09-01 06:44:43 2019-09-01 07:40:42 0:55:59 0:37:41 0:18:18 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

"2019-09-01T07:26:17.826333+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253000 2019-08-26 03:21:07 2019-09-01 06:46:48 2019-09-01 07:18:47 0:31:59 0:17:46 0:14:13 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_namespaces.yaml} 3
Failure Reason:

Command failed on smithi164 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253001 2019-08-26 03:21:08 2019-09-01 06:48:37 2019-09-01 07:10:36 0:21:59 0:11:07 0:10:52 smithi master krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-on.yaml tasks/unmap.yaml} 2
Failure Reason:

Command failed on smithi071 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253002 2019-08-26 03:21:08 2019-09-01 06:48:37 2019-09-01 07:42:37 0:54:00 0:23:56 0:30:04 smithi master krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/no-resets.yaml} 3
Failure Reason:

"2019-09-01T07:38:06.968847+0000 mon.a (mon.0) 198 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)" in cluster log

fail 4253003 2019-08-26 03:21:09 2019-09-01 06:49:03 2019-09-01 08:37:04 1:48:01 0:15:46 1:32:15 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_concurrent.yaml} 3
Failure Reason:

Command failed on smithi049 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253004 2019-08-26 03:21:10 2019-09-01 06:50:23 2019-09-01 07:16:22 0:25:59 0:10:16 0:15:43 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_parent_overlap.yaml} 1
Failure Reason:

"2019-09-01T07:12:30.020633+0000 mon.a (mon.0) 102 : cluster [WRN] Health check failed: 2 osds down (OSD_DOWN)" in cluster log

fail 4253005 2019-08-26 03:21:11 2019-09-01 06:50:23 2019-09-01 07:34:22 0:43:59 0:13:52 0:30:07 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress.yaml} 3
Failure Reason:

"2019-09-01T07:32:10.265636+0000 mon.b (mon.0) 199 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253006 2019-08-26 03:21:11 2019-09-01 06:50:23 2019-09-01 07:50:23 1:00:00 0:23:50 0:36:10 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} 3
Failure Reason:

"2019-09-01T07:46:46.494483+0000 mon.b (mon.0) 447 : cluster [WRN] Health check failed: 6 osds down (OSD_DOWN)" in cluster log

fail 4253007 2019-08-26 03:21:12 2019-09-01 06:50:32 2019-09-01 07:40:32 0:50:00 0:13:04 0:36:56 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_huge_tickets.yaml} 3
Failure Reason:

"2019-09-01T07:36:14.538106+0000 mon.a (mon.0) 159 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253008 2019-08-26 03:21:13 2019-09-01 06:50:37 2019-09-01 08:08:37 1:18:00 0:40:42 0:37:18 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} 3
Failure Reason:

Command failed on smithi145 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253009 2019-08-26 03:21:14 2019-09-01 06:50:39 2019-09-01 07:40:39 0:50:00 0:13:04 0:36:56 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} 3
Failure Reason:

"2019-09-01T07:36:20.945665+0000 mon.b (mon.0) 122 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253010 2019-08-26 03:21:14 2019-09-01 06:51:03 2019-09-01 07:49:03 0:58:00 0:10:54 0:47:06 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_image_read.yaml} 3
Failure Reason:

Command failed on smithi026 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253011 2019-08-26 03:21:15 2019-09-01 06:51:03 2019-09-01 07:43:03 0:52:00 0:35:08 0:16:52 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} 3
Failure Reason:

Command failed on smithi191 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253012 2019-08-26 03:21:16 2019-09-01 06:51:03 2019-09-01 08:43:04 1:52:01 0:18:29 1:33:32 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsx.yaml} 3
Failure Reason:

"2019-09-01T08:38:50.900874+0000 mon.b (mon.0) 172 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253013 2019-08-26 03:21:17 2019-09-01 06:52:45 2019-09-01 08:28:45 1:36:00 0:11:21 1:24:39 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_kernel.yaml} 3
Failure Reason:

Command failed on smithi168 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253014 2019-08-26 03:21:18 2019-09-01 06:52:45 2019-09-01 07:16:44 0:23:59 0:10:20 0:13:39 smithi master krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_whole_object_zeroout.yaml} 1
Failure Reason:

"2019-09-01T07:12:42.931189+0000 mon.a (mon.0) 98 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 4253015 2019-08-26 03:21:18 2019-09-01 06:52:51 2019-09-01 07:44:51 0:52:00 0:11:07 0:40:53 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_snapshot_io.yaml} 3
Failure Reason:

Command failed on smithi176 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253016 2019-08-26 03:21:19 2019-09-01 06:56:45 2019-09-01 09:16:46 2:20:01 0:58:49 1:21:12 smithi master krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} 3
Failure Reason:

Command failed on smithi026 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253017 2019-08-26 03:21:20 2019-09-01 06:56:45 2019-09-01 07:30:45 0:34:00 0:13:53 0:20:07 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_iozone.yaml} 3
Failure Reason:

Command failed on smithi198 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253018 2019-08-26 03:21:21 2019-09-01 06:58:31 2019-09-01 09:32:33 2:34:02 0:26:10 2:07:52 smithi master krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} 3
Failure Reason:

"2019-09-01T09:28:57.450079+0000 mon.b (mon.0) 577 : cluster [WRN] Health check failed: 4 osds down (OSD_DOWN)" in cluster log

fail 4253019 2019-08-26 03:21:21 2019-09-01 06:58:38 2019-09-01 08:36:39 1:38:01 0:15:50 1:22:11 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_unmap.yaml} 3
Failure Reason:

Command failed on smithi104 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253020 2019-08-26 03:21:22 2019-09-01 07:17:51 2019-09-01 08:25:51 1:08:00 0:11:02 0:56:58 smithi master krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_trivial_sync.yaml} 3
Failure Reason:

Command failed on smithi067 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog"

fail 4253021 2019-08-26 03:21:23 2019-09-01 07:17:51 2019-09-01 09:53:52 2:36:01 0:23:39 2:12:22 smithi master krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_simple_big.yaml} 3
Failure Reason:

"2019-09-01T09:48:47.436859+0000 mon.b (mon.0) 194 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log