Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
dead 1398817 2017-07-14 04:24:03 2017-07-14 11:21:34 2017-07-14 23:28:53 12:07:19 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
dead 1398818 2017-07-14 04:24:03 2017-07-14 11:21:29 2017-07-14 23:27:22 12:05:53 smithi master centos 7.3 upgrade:jewel-x/point-to-point-x/{distros/centos_7.3.yaml point-to-point-upgrade.yaml} 3
fail 1398821 2017-07-14 04:24:04 2017-07-14 11:22:02 2017-07-14 11:35:59 0:13:57 0:07:16 0:06:41 smithi master centos 7.3 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi029 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.3 1.0 host=localhost root=default'

fail 1398824 2017-07-14 04:24:05 2017-07-14 11:22:04 2017-07-14 11:38:03 0:15:59 0:07:40 0:08:19 smithi master centos 7.3 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi131 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.3 1.0 host=localhost root=default'

fail 1398827 2017-07-14 04:24:06 2017-07-14 11:22:08 2017-07-14 12:48:09 1:26:01 1:18:05 0:07:56 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 11:35:45.104471 mon.a mon.0 172.21.15.46:6789/0 540 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398830 2017-07-14 04:24:06 2017-07-14 11:22:18 2017-07-14 11:40:17 0:17:59 0:05:37 0:12:22 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi093 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398833 2017-07-14 04:24:07 2017-07-14 11:22:39 2017-07-14 12:40:40 1:18:01 1:02:54 0:15:07 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:48:04.198914 mon.a mon.0 172.21.15.20:6789/0 354 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398836 2017-07-14 04:24:08 2017-07-14 11:22:40 2017-07-14 12:56:43 1:34:03 1:16:37 0:17:26 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 11:43:48.525411 mon.b mon.0 172.21.15.77:6789/0 76 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398839 2017-07-14 04:24:08 2017-07-14 11:23:06 2017-07-14 14:09:05 2:45:59 1:00:54 1:45:05 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:14:46.329879 mon.b mon.0 172.21.15.93:6789/0 318 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398842 2017-07-14 04:24:09 2017-07-14 11:23:49 2017-07-14 12:55:48 1:31:59 1:17:42 0:14:17 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:48:28.898562 mon.b mon.0 172.21.15.31:6789/0 171 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

dead 1398846 2017-07-14 04:24:10 2017-07-14 11:23:55 2017-07-14 23:30:27 12:06:32 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
fail 1398848 2017-07-14 04:24:10 2017-07-14 11:24:05 2017-07-14 14:26:01 3:01:56 1:12:15 1:49:41 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:18:02.341861 mon.b mon.0 172.21.15.97:6789/0 221 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398852 2017-07-14 04:24:11 2017-07-14 11:25:11 2017-07-14 11:47:09 0:21:58 0:09:24 0:12:34 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi151 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398855 2017-07-14 04:24:12 2017-07-14 11:25:10 2017-07-14 13:05:11 1:40:01 1:27:43 0:12:18 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 11:40:21.147240 mon.b mon.0 172.21.15.63:6789/0 59 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398858 2017-07-14 04:24:12 2017-07-14 11:25:49 2017-07-14 14:33:50 3:08:01 1:30:04 1:37:57 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:15:25.137469 mon.a mon.0 172.21.15.82:6789/0 426 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398861 2017-07-14 04:24:13 2017-07-14 11:26:22 2017-07-14 13:02:21 1:35:59 1:25:20 0:10:39 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:42:22.143489 mon.b mon.0 172.21.15.12:6789/0 70 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398864 2017-07-14 04:24:13 2017-07-14 11:26:22 2017-07-14 12:54:21 1:27:59 1:12:33 0:15:26 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 11:49:53.051706 mon.a mon.0 172.21.15.55:6789/0 547 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

dead 1398867 2017-07-14 04:24:14 2017-07-14 11:26:22 2017-07-14 23:32:33 12:06:11 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
fail 1398869 2017-07-14 04:24:15 2017-07-14 11:26:30 2017-07-14 11:56:30 0:30:00 0:05:45 0:24:15 smithi master ubuntu 14.04 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi172 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.3 1.0 host=localhost root=default'

fail 1398872 2017-07-14 04:24:15 2017-07-14 11:27:45 2017-07-14 12:57:44 1:29:59 1:05:56 0:24:03 smithi master ubuntu 14.04 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 11:52:33.868559 mon.a mon.0 172.21.15.45:6789/0 25 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1398875 2017-07-14 04:24:16 2017-07-14 11:27:48 2017-07-14 11:51:47 0:23:59 0:10:45 0:13:14 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi144 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398878 2017-07-14 04:24:17 2017-07-14 11:28:06 2017-07-14 11:45:59 0:17:53 0:09:02 0:08:51 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi172 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398881 2017-07-14 04:24:17 2017-07-14 11:28:22 2017-07-14 14:30:21 3:01:59 1:00:15 2:01:44 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:36:16.066958 mon.b mon.0 172.21.15.107:6789/0 253 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398884 2017-07-14 04:24:18 2017-07-14 11:28:24 2017-07-14 13:06:25 1:38:01 1:15:53 0:22:08 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:55:23.700974 mon.a mon.0 172.21.15.149:6789/0 78 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398887 2017-07-14 04:24:19 2017-07-14 11:28:32 2017-07-14 14:16:35 2:48:03 0:59:15 1:48:48 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 13:22:27.870569 mon.a mon.0 172.21.15.45:6789/0 563 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398890 2017-07-14 04:24:19 2017-07-14 11:29:11 2017-07-14 13:25:12 1:56:01 0:05:57 1:50:04 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi107 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398893 2017-07-14 04:24:20 2017-07-14 11:29:11 2017-07-14 12:39:11 1:10:00 1:03:52 0:06:08 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:44:02.625951 mon.b mon.0 172.21.15.133:6789/0 172 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398896 2017-07-14 04:24:21 2017-07-14 11:29:38 2017-07-14 12:35:38 1:06:00 1:01:20 0:04:40 smithi master ubuntu 14.04 upgrade:jewel-x/point-to-point-x/{distros/ubuntu_14.04.yaml point-to-point-upgrade.yaml} 3
Failure Reason:

Command failed (workunit test rados/test-upgrade-v11.0.0.sh) on smithi160 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=jewel TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test-upgrade-v11.0.0.sh'

fail 1398899 2017-07-14 04:24:21 2017-07-14 11:29:49 2017-07-14 14:29:50 3:00:01 1:09:01 1:51:00 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 13:25:56.490500 mon.b mon.0 172.21.15.163:6789/0 137 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398902 2017-07-14 04:24:22 2017-07-14 11:29:51 2017-07-14 12:59:48 1:29:57 1:14:03 0:15:54 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 11:53:12.404336 mon.a mon.0 172.21.15.82:6789/0 520 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398905 2017-07-14 04:24:22 2017-07-14 11:30:27 2017-07-14 12:48:27 1:18:00 1:04:05 0:13:55 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:54:55.084287 mon.a mon.0 172.21.15.9:6789/0 79 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398909 2017-07-14 04:24:23 2017-07-14 11:31:57 2017-07-14 12:09:57 0:38:00 0:05:38 0:32:22 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi065 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398912 2017-07-14 04:24:24 2017-07-14 11:31:58 2017-07-14 12:43:57 1:11:59 1:03:13 0:08:46 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 11:42:59.056884 mon.b mon.0 172.21.15.84:6789/0 94 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398915 2017-07-14 04:24:25 2017-07-14 11:32:04 2017-07-14 13:06:04 1:34:00 1:15:55 0:18:05 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 12:02:54.535235 mon.b mon.0 172.21.15.130:6789/0 363 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

dead 1398918 2017-07-14 04:24:25 2017-07-14 11:32:04 2017-07-14 23:38:59 12:06:55 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
fail 1398921 2017-07-14 04:24:26 2017-07-14 11:34:14 2017-07-14 15:16:18 3:42:04 2:11:12 1:30:52 smithi master ubuntu 16.04 upgrade:jewel-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:30:09.591723 osd.2 osd.2 172.21.15.100:6815/29340 231 : cluster [ERR] 9.13 required past_interval bounds are empty [685,685) but past_intervals is not: {656=interval(656-657 up [0,1](0) acting [0,1](0)),658=interval(658-662 up [1](1) acting [1](1) maybe_went_rw),663=interval(663-673 up [0,1](0) acting [0,1](0) maybe_went_rw),674=interval(674-676 up [3,1](3) acting [0,1](0)),677=interval(677-679 up [3,1](3) acting [3,1](3) maybe_went_rw),680=interval(680-680 up [2,1](2) acting [3,1](3)),681=interval(681-684 up [2,1](2) acting [2,1](2) maybe_went_rw)}" in cluster log

fail 1398924 2017-07-14 04:24:27 2017-07-14 11:34:14 2017-07-14 14:36:17 3:02:03 0:59:54 2:02:09 smithi master ubuntu 16.04 upgrade:jewel-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/ec-rados-default.yaml 5-finish-upgrade.yaml 6-luminous.yaml 7-final-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 13:37:44.550270 mon.a mon.0 172.21.15.194:6789/0 25 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 1398927 2017-07-14 04:24:27 2017-07-14 11:34:14 2017-07-14 13:58:16 2:24:02 1:10:57 1:13:05 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 12:53:39.933657 mon.a mon.0 172.21.15.84:6789/0 540 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398930 2017-07-14 04:24:28 2017-07-14 11:34:14 2017-07-14 11:46:14 0:12:00 0:06:16 0:05:44 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi038 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398933 2017-07-14 04:24:29 2017-07-14 11:34:33 2017-07-14 14:10:33 2:36:00 0:59:44 1:36:16 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 13:15:29.485516 mon.b mon.0 172.21.15.63:6789/0 253 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398936 2017-07-14 04:24:29 2017-07-14 11:34:43 2017-07-14 13:34:46 2:00:03 0:05:04 1:54:59 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

Command failed on smithi196 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398939 2017-07-14 04:24:30 2017-07-14 11:35:55 2017-07-14 12:09:55 0:34:00 0:06:30 0:27:30 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi038 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398941 2017-07-14 04:24:30 2017-07-14 11:35:56 2017-07-14 13:43:57 2:08:01 1:11:31 0:56:30 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 12:34:31.185422 mon.b mon.0 172.21.15.68:6789/0 92 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398944 2017-07-14 04:24:31 2017-07-14 11:36:03 2017-07-14 13:04:02 1:27:59 1:20:12 0:07:47 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 11:51:25.477977 mon.b mon.0 172.21.15.93:6789/0 175 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398948 2017-07-14 04:24:32 2017-07-14 11:36:34 2017-07-14 12:58:34 1:22:00 1:12:01 0:09:59 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/cache-pool-snaps.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

"2017-07-14 11:57:01.689791 mon.b mon.0 172.21.15.131:6789/0 83 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398950 2017-07-14 04:24:32 2017-07-14 11:38:13 2017-07-14 13:46:17 2:08:04 0:05:55 2:02:09 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/ec-rados-default.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

Command failed on smithi176 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398953 2017-07-14 04:24:33 2017-07-14 11:38:14 2017-07-14 12:56:15 1:18:01 1:13:47 0:04:14 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 11:47:32.004374 mon.b mon.0 172.21.15.100:6789/0 103 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398957 2017-07-14 04:24:34 2017-07-14 11:40:02 2017-07-14 12:02:01 0:21:59 0:07:30 0:14:29 smithi master centos 7.3 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/rados_loadgenbig.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/centos_latest.yaml} 3
Failure Reason:

Command failed on smithi086 with status 2: 'sudo ceph --cluster ceph osd crush create-or-move osd.2 1.0 host=localhost root=default'

fail 1398960 2017-07-14 04:24:34 2017-07-14 11:40:27 2017-07-14 14:06:21 2:25:54 1:25:30 1:00:24 smithi master ubuntu 14.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_api.yaml 3-upgrade-sequence/upgrade-all.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml} 3
Failure Reason:

"2017-07-14 12:42:34.655946 mon.a mon.0 172.21.15.73:6789/0 59 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log

fail 1398963 2017-07-14 04:24:35 2017-07-14 11:41:27 2017-07-14 13:39:27 1:58:00 1:05:59 0:52:01 smithi master ubuntu 16.04 upgrade:jewel-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 1.5-final-scrub.yaml 2-workload/test_rbd_python.yaml 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-luminous.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_latest.yaml} 3
Failure Reason:

"2017-07-14 12:39:22.506741 mon.a mon.0 172.21.15.182:6789/0 490 : cluster [ERR] Health check failed: 1 mds daemon down (MDS_FAILED)" in cluster log