Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 1418157 2017-07-19 04:23:31 2017-07-20 08:46:11 2017-07-20 20:51:07 12:04:56 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
fail 1418160 2017-07-19 04:23:32 2017-07-20 08:46:20 2017-07-20 10:08:21 1:22:01 0:47:21 0:34:40 smithi master centos 7.3 upgrade:jewel-x/point-to-point-x/{distros/centos_7.3.yaml point-to-point-upgrade.yaml} 3
Failure Reason:

Command failed on smithi034 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client luminous'"

fail 1418163 2017-07-19 04:23:32 2017-07-20 08:46:25 2017-07-20 11:52:28 3:06:03 2:03:34 1:02:29 smithi master centos 7.3 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:52:46.783098 mon.a mon.0 172.21.15.1:6789/0 40 : cluster [WRN] Health check update: 2 osds down (OSD_DOWN)" in cluster log

fail 1418166 2017-07-19 04:23:33 2017-07-20 08:48:15 2017-07-20 09:58:17 1:10:02 0:53:57 0:16:05 smithi master centos 7.3 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/centos_latest.yaml} 3
Failure Reason:

Scrubbing terminated -- not all pgs were active and clean.

fail 1418169 2017-07-19 04:23:34 2017-07-20 08:48:21 2017-07-20 09:34:22 0:46:01 0:05:26 0:40:35 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi044 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418171 2017-07-19 04:23:34 2017-07-20 08:49:00 2017-07-20 09:14:59 0:25:59 0:05:09 0:20:50 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi202 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418174 2017-07-19 04:23:35 2017-07-20 08:49:12 2017-07-20 11:09:14 2:20:02 0:58:04 1:21:58 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 10:20:18.766611 mon.a mon.0 172.21.15.34:6789/0 351 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418177 2017-07-19 04:23:36 2017-07-20 08:50:34 2017-07-20 10:18:33 1:27:59 1:14:28 0:13:31 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:08:39.266380 mon.b mon.0 172.21.15.51:6789/0 70 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418180 2017-07-19 04:23:37 2017-07-20 08:50:59 2017-07-20 10:16:58 1:25:59 0:58:19 0:27:40 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:27:03.656524 mon.a mon.0 172.21.15.117:6789/0 556 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418183 2017-07-19 04:23:37 2017-07-20 08:51:18 2017-07-20 09:25:18 0:34:00 0:06:50 0:27:10 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi135 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418186 2017-07-19 04:23:38 2017-07-20 08:51:21 2017-07-20 09:15:20 0:23:59 0:05:19 0:18:40 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi072 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418189 2017-07-19 04:23:39 2017-07-20 08:52:21 2017-07-20 10:38:23 1:46:02 1:21:06 0:24:56 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:21:14.238870 mon.b mon.0 172.21.15.184:6789/0 82 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418192 2017-07-19 04:23:39 2017-07-20 08:52:21 2017-07-20 10:10:22 1:18:01 1:10:07 0:07:54 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:08:11.060145 mon.b mon.0 172.21.15.18:6789/0 348 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418195 2017-07-19 04:23:40 2017-07-20 08:52:46 2017-07-20 10:26:46 1:34:00 1:22:14 0:11:46 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:08:58.515255 mon.b mon.0 172.21.15.73:6789/0 71 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418198 2017-07-19 04:23:41 2017-07-20 08:52:47 2017-07-20 10:18:48 1:26:01 1:11:30 0:14:31 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:17:45.726333 mon.b mon.0 172.21.15.87:6789/0 349 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418201 2017-07-19 04:23:41 2017-07-20 08:53:13 2017-07-20 10:05:13 1:12:00 1:00:14 0:11:46 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:09:41.533811 mon.a mon.0 172.21.15.4:6789/0 66 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418204 2017-07-19 04:23:42 2017-07-20 08:53:44 2017-07-20 10:21:45 1:28:01 1:05:11 0:22:50 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:22:00.276351 mon.b mon.0 172.21.15.79:6789/0 324 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418207 2017-07-19 04:23:43 2017-07-20 08:53:49 2017-07-20 10:11:50 1:18:01 0:05:37 1:12:24 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi097 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418210 2017-07-19 04:23:44 2017-07-20 08:53:59 2017-07-20 11:00:01 2:06:02 1:58:16 0:07:46 smithi master ubuntu 14.04 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:03:42.289036 mon.a mon.0 172.21.15.169:6789/0 28 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1418213 2017-07-19 04:23:44 2017-07-20 08:55:02 2017-07-20 10:07:03 1:12:01 1:01:22 0:10:39 smithi master ubuntu 14.04 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:07:21.372328 mon.a mon.0 172.21.15.64:6789/0 28 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1418216 2017-07-19 04:23:45 2017-07-20 08:55:07 2017-07-20 09:21:07 0:26:00 0:06:10 0:19:50 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi010 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418219 2017-07-19 04:23:46 2017-07-20 08:55:17 2017-07-20 10:23:18 1:28:01 1:07:54 0:20:07 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:17:52.661076 mon.a mon.0 172.21.15.45:6789/0 64 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1418222 2017-07-19 04:23:47 2017-07-20 08:55:40 2017-07-20 10:19:41 1:24:01 0:56:48 0:27:13 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:30:01.533307 mon.b mon.0 172.21.15.90:6789/0 267 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418225 2017-07-19 04:23:47 2017-07-20 08:56:19 2017-07-20 10:04:17 1:07:58 0:06:07 1:01:51 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi137 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418228 2017-07-19 04:23:48 2017-07-20 08:56:37 2017-07-20 10:06:38 1:10:01 0:57:21 0:12:40 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:14:43.426267 mon.a mon.0 172.21.15.67:6789/0 535 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418231 2017-07-19 04:23:49 2017-07-20 08:56:45 2017-07-20 09:14:45 0:18:00 0:05:52 0:12:08 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi104 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418234 2017-07-19 04:23:49 2017-07-20 08:56:53 2017-07-20 10:10:53 1:14:00 0:59:52 0:14:08 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:18:25.253503 mon.a mon.0 172.21.15.3:6789/0 156 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 1418237 2017-07-19 04:23:50 2017-07-20 08:57:13 2017-07-20 21:04:37 12:07:24 smithi master ubuntu 14.04 upgrade:jewel-x/point-to-point-x/{distros/ubuntu_14.04.yaml point-to-point-upgrade.yaml} 3
fail 1418240 2017-07-19 04:23:51 2017-07-20 08:57:18 2017-07-20 09:29:18 0:32:00 0:12:04 0:19:56 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi160 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'

fail 1418243 2017-07-19 04:23:52 2017-07-20 08:57:49 2017-07-20 09:13:48 0:15:59 0:05:34 0:10:25 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi110 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418245 2017-07-19 04:23:52 2017-07-20 08:57:58 2017-07-20 10:27:59 1:30:01 1:21:31 0:08:30 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:11:21.170717 mon.b mon.0 172.21.15.15:6789/0 56 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418248 2017-07-19 04:23:53 2017-07-20 08:58:11 2017-07-20 11:22:13 2:24:02 1:11:30 1:12:32 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 10:21:46.337954 mon.a mon.0 172.21.15.53:6789/0 484 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418251 2017-07-19 04:23:54 2017-07-20 08:58:18 2017-07-20 10:10:19 1:12:01 0:56:54 0:15:07 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:17:21.795614 mon.b mon.0 172.21.15.190:6789/0 87 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418254 2017-07-19 04:23:54 2017-07-20 08:58:27 2017-07-20 10:12:28 1:14:01 1:02:26 0:11:35 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:18:23.967680 mon.a mon.0 172.21.15.11:6789/0 339 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 1418257 2017-07-19 04:23:55 2017-07-20 08:59:00 2017-07-20 21:05:40 12:06:40 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
fail 1418260 2017-07-19 04:23:56 2017-07-20 08:59:00 2017-07-20 12:25:05 3:26:05 2:03:19 1:22:46 smithi master ubuntu 16.04 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 10:23:32.047576 mon.a mon.0 172.21.15.97:6789/0 25 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1418262 2017-07-19 04:23:56 2017-07-20 08:59:30 2017-07-20 10:01:31 1:02:01 0:49:12 0:12:49 smithi master ubuntu 16.04 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:14:49.462091 mon.a mon.0 172.21.15.194:6789/0 25 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1418265 2017-07-19 04:23:57 2017-07-20 09:01:17 2017-07-20 09:21:16 0:19:59 0:05:38 0:14:21 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi089 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418268 2017-07-19 04:23:58 2017-07-20 09:01:17 2017-07-20 10:19:18 1:18:01 1:08:27 0:09:34 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:16:03.524311 mon.b mon.0 172.21.15.27:6789/0 88 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418271 2017-07-19 04:23:58 2017-07-20 09:01:22 2017-07-20 11:59:25 2:58:03 0:56:57 2:01:06 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 11:07:27.266824 mon.a mon.0 172.21.15.62:6789/0 343 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418275 2017-07-19 04:23:59 2017-07-20 09:01:39 2017-07-20 10:31:40 1:30:01 1:13:27 0:16:34 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:22:03.927661 mon.b mon.0 172.21.15.85:6789/0 71 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418278 2017-07-19 04:24:00 2017-07-20 09:02:16 2017-07-20 11:44:21 2:42:05 0:59:33 1:42:32 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 10:53:24.746145 mon.a mon.0 172.21.15.24:6789/0 593 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418281 2017-07-19 04:24:00 2017-07-20 09:02:17 2017-07-20 10:28:19 1:26:02 1:06:39 0:19:23 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:25:34.860676 mon.b mon.0 172.21.15.49:6789/0 88 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418283 2017-07-19 04:24:01 2017-07-20 09:03:02 2017-07-20 10:15:02 1:12:00 0:56:31 0:15:29 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:25:54.062095 mon.b mon.0 172.21.15.107:6789/0 170 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418286 2017-07-19 04:24:02 2017-07-20 09:03:15 2017-07-20 10:25:15 1:22:00 1:06:07 0:15:53 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:24:26.522214 mon.b mon.0 172.21.15.24:6789/0 139 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418289 2017-07-19 04:24:02 2017-07-20 09:03:37 2017-07-20 09:21:37 0:18:00 0:06:12 0:11:48 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi176 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1418292 2017-07-19 04:24:03 2017-07-20 09:04:32 2017-07-20 10:34:33 1:30:01 1:20:52 0:09:09 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-20 09:17:07.759823 mon.a mon.0 172.21.15.91:6789/0 64 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418295 2017-07-19 04:24:04 2017-07-20 09:04:41 2017-07-20 10:46:42 1:42:01 1:12:48 0:29:13 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-20 09:46:44.874152 mon.a mon.0 172.21.15.135:6789/0 493 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418298 2017-07-19 04:24:05 2017-07-20 09:04:41 2017-07-20 10:20:42 1:16:01 0:55:03 0:20:58 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-20 09:27:39.761456 mon.b mon.0 172.21.15.40:6789/0 91 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1418301 2017-07-19 04:24:05 2017-07-20 09:06:34 2017-07-20 10:18:34 1:12:00 0:05:32 1:06:28 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi109 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'