User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail | Dead |
---|---|---|---|---|---|---|---|---|---|---|---|
teuthology | 2019-08-27 02:30:03 | 2019-08-27 02:31:16 | 2019-08-27 15:26:44 | 12:55:28 | upgrade:mimic-x | nautilus | smithi | 49a6c9a | 20 | 39 | 5 |
Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
fail | 4255905 | 2019-08-27 02:30:29 | 2019-08-27 02:31:16 | 2019-08-27 03:21:15 | 0:49:59 | 0:36:57 | 0:13:02 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
pass | 4255906 | 2019-08-27 02:30:30 | 2019-08-27 02:33:04 | 2019-08-27 06:17:07 | 3:44:03 | 3:31:21 | 0:12:42 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
pass | 4255907 | 2019-08-27 02:30:31 | 2019-08-27 02:33:04 | 2019-08-27 04:41:05 | 2:08:01 | 0:57:54 | 1:10:07 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
pass | 4255908 | 2019-08-27 02:30:31 | 2019-08-27 02:35:03 | 2019-08-27 08:49:09 | 6:14:06 | 2:21:24 | 3:52:42 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml} | 4 | |
fail | 4255909 | 2019-08-27 02:30:32 | 2019-08-27 02:35:03 | 2019-08-27 04:27:04 | 1:52:01 | 0:30:52 | 1:21:09 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255910 | 2019-08-27 02:30:33 | 2019-08-27 02:36:31 | 2019-08-27 06:36:34 | 4:00:03 | 3:19:10 | 0:40:53 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi155 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats' |
||||||||||||||
fail | 4255911 | 2019-08-27 02:30:34 | 2019-08-27 02:36:31 | 2019-08-27 03:32:30 | 0:55:59 | 0:30:30 | 0:25:29 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi026 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
pass | 4255912 | 2019-08-27 02:30:35 | 2019-08-27 02:37:48 | 2019-08-27 04:51:50 | 2:14:02 | 2:04:32 | 0:09:30 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
fail | 4255913 | 2019-08-27 02:30:36 | 2019-08-27 02:38:19 | 2019-08-27 03:28:19 | 0:50:00 | 0:36:13 | 0:13:47 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
pass | 4255914 | 2019-08-27 02:30:36 | 2019-08-27 02:38:31 | 2019-08-27 08:48:36 | 6:10:05 | 3:22:33 | 2:47:32 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
pass | 4255915 | 2019-08-27 02:30:37 | 2019-08-27 02:38:42 | 2019-08-27 05:04:43 | 2:26:01 | 0:43:33 | 1:42:28 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
pass | 4255916 | 2019-08-27 02:30:38 | 2019-08-27 02:39:11 | 2019-08-27 05:29:13 | 2:50:02 | 2:17:58 | 0:32:04 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml} | 4 | |
fail | 4255917 | 2019-08-27 02:30:39 | 2019-08-27 02:41:39 | 2019-08-27 03:05:38 | 0:23:59 | 0:13:23 | 0:10:36 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
Command failed on smithi117 with status 124: "sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_health_to_clog=true" |
||||||||||||||
pass | 4255918 | 2019-08-27 02:30:39 | 2019-08-27 02:42:55 | 2019-08-27 13:37:06 | 10:54:11 | 3:39:50 | 7:14:21 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
dead | 4255919 | 2019-08-27 02:30:40 | 2019-08-27 02:42:55 | 2019-08-27 14:45:28 | 12:02:33 | 8:42:09 | 3:20:24 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255920 | 2019-08-27 02:30:41 | 2019-08-27 02:43:41 | 2019-08-27 05:19:42 | 2:36:01 | 2:07:34 | 0:28:27 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
"2019-08-27 03:20:43.783194 mon.c (mon.0) 25 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
pass | 4255921 | 2019-08-27 02:30:42 | 2019-08-27 02:45:43 | 2019-08-27 07:13:47 | 4:28:04 | 2:12:09 | 2:15:55 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} | 4 | |
fail | 4255922 | 2019-08-27 02:30:43 | 2019-08-27 02:46:29 | 2019-08-27 03:26:28 | 0:39:59 | 0:26:44 | 0:13:15 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi172 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255923 | 2019-08-27 02:30:43 | 2019-08-27 02:46:37 | 2019-08-27 03:26:36 | 0:39:59 | 0:28:06 | 0:11:53 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi052 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
dead | 4255924 | 2019-08-27 02:30:44 | 2019-08-27 02:47:04 | 2019-08-27 14:49:36 | 12:02:32 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml} | 4 | |||
fail | 4255925 | 2019-08-27 02:30:45 | 2019-08-27 02:47:04 | 2019-08-27 03:05:03 | 0:17:59 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |||
Failure Reason:
Command failed on smithi146 with status 100: 'sudo apt-get clean' |
||||||||||||||
pass | 4255926 | 2019-08-27 02:30:46 | 2019-08-27 02:48:41 | 2019-08-27 08:16:46 | 5:28:05 | 3:55:14 | 1:32:51 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
pass | 4255927 | 2019-08-27 02:30:47 | 2019-08-27 02:48:41 | 2019-08-27 05:26:43 | 2:38:02 | 0:54:43 | 1:43:19 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
fail | 4255928 | 2019-08-27 02:30:47 | 2019-08-27 02:48:41 | 2019-08-27 03:28:41 | 0:40:00 | 0:29:54 | 0:10:06 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255929 | 2019-08-27 02:30:48 | 2019-08-27 02:50:43 | 2019-08-27 06:16:45 | 3:26:02 | 2:17:09 | 1:08:53 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
"2019-08-27 04:17:46.091428 mon.c (mon.0) 40 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255930 | 2019-08-27 02:30:49 | 2019-08-27 02:50:45 | 2019-08-27 08:22:49 | 5:32:04 | 0:21:25 | 5:10:39 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi018 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255931 | 2019-08-27 02:30:50 | 2019-08-27 02:52:20 | 2019-08-27 03:40:20 | 0:48:00 | 0:20:41 | 0:27:19 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi085 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255932 | 2019-08-27 02:30:51 | 2019-08-27 02:52:25 | 2019-08-27 05:10:26 | 2:18:01 | 0:39:28 | 1:38:33 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255933 | 2019-08-27 02:30:52 | 2019-08-27 02:52:28 | 2019-08-27 05:14:29 | 2:22:01 | 2:06:01 | 0:16:00 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
"2019-08-27 03:18:09.236723 mon.b (mon.0) 83 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255934 | 2019-08-27 02:30:52 | 2019-08-27 02:52:46 | 2019-08-27 14:52:58 | 12:00:12 | 4:59:37 | 7:00:35 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
failed to recover before timeout expired |
||||||||||||||
pass | 4255935 | 2019-08-27 02:30:53 | 2019-08-27 02:56:49 | 2019-08-27 03:48:48 | 0:51:59 | 0:44:36 | 0:07:23 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
dead | 4255936 | 2019-08-27 02:30:54 | 2019-08-27 02:56:59 | 2019-08-27 03:04:58 | 0:07:59 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |||
Failure Reason:
SSH connection to smithi146 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' |
||||||||||||||
fail | 4255937 | 2019-08-27 02:30:55 | 2019-08-27 02:57:42 | 2019-08-27 04:03:42 | 1:06:00 | 0:38:49 | 0:27:11 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255938 | 2019-08-27 02:30:56 | 2019-08-27 02:59:14 | 2019-08-27 09:29:19 | 6:30:05 | 0:27:12 | 6:02:53 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
pass | 4255939 | 2019-08-27 02:30:56 | 2019-08-27 03:00:01 | 2019-08-27 04:16:01 | 1:16:00 | 0:51:07 | 0:24:53 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
fail | 4255940 | 2019-08-27 02:30:57 | 2019-08-27 03:02:09 | 2019-08-27 05:50:11 | 2:48:02 | 2:15:31 | 0:32:31 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml} | 4 | |
Failure Reason:
"2019-08-27 03:53:34.490940 mon.a (mon.0) 112 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255941 | 2019-08-27 02:30:58 | 2019-08-27 03:02:46 | 2019-08-27 05:44:48 | 2:42:02 | 0:29:40 | 2:12:22 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255942 | 2019-08-27 02:30:59 | 2019-08-27 03:05:14 | 2019-08-27 08:25:19 | 5:20:05 | 0:30:54 | 4:49:11 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi086 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
pass | 4255943 | 2019-08-27 02:31:00 | 2019-08-27 03:05:14 | 2019-08-27 07:31:18 | 4:26:04 | 0:53:22 | 3:32:42 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
fail | 4255944 | 2019-08-27 02:31:01 | 2019-08-27 03:05:40 | 2019-08-27 08:31:45 | 5:26:05 | 2:11:40 | 3:14:25 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
"2019-08-27 06:28:16.546434 mon.a (mon.0) 158 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255945 | 2019-08-27 02:31:01 | 2019-08-27 03:07:02 | 2019-08-27 03:53:01 | 0:45:59 | 0:35:43 | 0:10:16 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
pass | 4255946 | 2019-08-27 02:31:02 | 2019-08-27 03:08:59 | 2019-08-27 07:07:02 | 3:58:03 | 3:34:57 | 0:23:06 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
fail | 4255947 | 2019-08-27 02:31:03 | 2019-08-27 03:10:59 | 2019-08-27 14:55:11 | 11:44:12 | 0:03:01 | 11:41:11 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 4 | |
Failure Reason:
data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'sudo') |
||||||||||||||
fail | 4255948 | 2019-08-27 02:31:04 | 2019-08-27 03:10:59 | 2019-08-27 08:35:03 | 5:24:04 | 2:18:04 | 3:06:00 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml} | 4 | |
Failure Reason:
"2019-08-27 06:36:25.744954 mon.c (mon.0) 35 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255949 | 2019-08-27 02:31:05 | 2019-08-27 03:12:18 | 2019-08-27 06:22:20 | 3:10:02 | 0:30:39 | 2:39:23 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255950 | 2019-08-27 02:31:06 | 2019-08-27 03:12:19 | 2019-08-27 11:20:26 | 8:08:07 | 3:57:50 | 4:10:17 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
failed to recover before timeout expired |
||||||||||||||
fail | 4255951 | 2019-08-27 02:31:07 | 2019-08-27 03:12:55 | 2019-08-27 06:46:58 | 3:34:03 | 0:20:49 | 3:13:14 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi117 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255952 | 2019-08-27 02:31:07 | 2019-08-27 03:14:44 | 2019-08-27 06:16:45 | 3:02:01 | 2:06:38 | 0:55:23 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
"2019-08-27 04:18:08.904154 mon.b (mon.0) 54 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
pass | 4255953 | 2019-08-27 02:31:08 | 2019-08-27 03:16:42 | 2019-08-27 05:54:44 | 2:38:02 | 2:12:59 | 0:25:03 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} | 4 | |
dead | 4255954 | 2019-08-27 02:31:09 | 2019-08-27 03:16:47 | 2019-08-27 15:19:09 | 12:02:22 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |||
pass | 4255955 | 2019-08-27 02:31:10 | 2019-08-27 03:16:58 | 2019-08-27 08:07:02 | 4:50:04 | 0:48:53 | 4:01:11 | smithi | master | centos | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} | 5 | |
fail | 4255956 | 2019-08-27 02:31:11 | 2019-08-27 03:20:43 | 2019-08-27 04:12:43 | 0:52:00 | 0:40:09 | 0:11:51 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
pass | 4255957 | 2019-08-27 02:31:12 | 2019-08-27 03:21:17 | 2019-08-27 06:05:18 | 2:44:01 | 2:09:25 | 0:34:36 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
dead | 4255958 | 2019-08-27 02:31:12 | 2019-08-27 03:22:06 | 2019-08-27 15:26:44 | 12:04:38 | 3:09:18 | 8:55:20 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | — | |
Failure Reason:
failed to recover before timeout expired |
||||||||||||||
pass | 4255959 | 2019-08-27 02:31:13 | 2019-08-27 03:23:01 | 2019-08-27 08:33:06 | 5:10:05 | 0:55:02 | 4:15:03 | smithi | master | rhel | 7.6 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml} | 5 | |
fail | 4255960 | 2019-08-27 02:31:14 | 2019-08-27 03:23:25 | 2019-08-27 05:07:26 | 1:44:01 | 0:30:47 | 1:13:14 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255961 | 2019-08-27 02:31:15 | 2019-08-27 03:26:32 | 2019-08-27 06:44:34 | 3:18:02 | 2:10:26 | 1:07:36 | smithi | master | centos | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} | 4 | |
Failure Reason:
"2019-08-27 04:47:12.468549 mon.a (mon.0) 145 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255962 | 2019-08-27 02:31:16 | 2019-08-27 03:26:32 | 2019-08-27 06:22:34 | 2:56:02 | 0:21:06 | 2:34:56 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi002 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
pass | 4255963 | 2019-08-27 02:31:17 | 2019-08-27 03:26:38 | 2019-08-27 09:04:42 | 5:38:04 | 0:42:08 | 4:55:56 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml} | 5 | |
fail | 4255964 | 2019-08-27 02:31:18 | 2019-08-27 03:28:34 | 2019-08-27 04:26:34 | 0:58:00 | 0:38:45 | 0:19:15 | smithi | master | rhel | 7.6 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |
||||||||||||||
fail | 4255965 | 2019-08-27 02:31:19 | 2019-08-27 03:28:42 | 2019-08-27 06:38:45 | 3:10:03 | 2:05:54 | 1:04:09 | smithi | master | ubuntu | 16.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml} | 4 | |
Failure Reason:
"2019-08-27 04:42:14.775055 mon.b (mon.0) 70 : cluster [ERR] Health check failed: Module 'rbd_support' has failed: Not found or unloadable (MGR_MODULE_ERROR)" in cluster log |
||||||||||||||
fail | 4255966 | 2019-08-27 02:31:19 | 2019-08-27 03:32:46 | 2019-08-27 05:12:46 | 1:40:00 | 0:20:33 | 1:19:27 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi161 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255967 | 2019-08-27 02:31:20 | 2019-08-27 03:36:19 | 2019-08-27 07:10:21 | 3:34:02 | 0:21:07 | 3:12:55 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} | 5 | |
Failure Reason:
Command failed on smithi150 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' |
||||||||||||||
fail | 4255968 | 2019-08-27 02:31:21 | 2019-08-27 03:40:35 | 2019-08-27 04:38:35 | 0:58:00 | 0:31:57 | 0:26:03 | smithi | master | ubuntu | 18.04 | upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} | 4 | |
Failure Reason:
'wait_until_healthy' reached maximum tries (150) after waiting for 900 seconds |