Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 5359807 2020-08-19 14:26:10 2020-08-19 15:50:10 2020-08-19 19:12:15 3:22:05 3:14:54 0:07:11 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} 4
Failure Reason:

Command failed (ragweed tests against rgw) on smithi013 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

dead 5359808 2020-08-19 14:26:10 2020-08-19 15:58:00 2020-08-19 16:05:59 0:07:59 0:01:31 0:06:28 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} 5
Failure Reason:

{'Failure object was': {'smithi136.front.sepia.ceph.com': {'results': [{'msg': 'Failed to download key at http://download.ceph.com/keys/autobuild.asc: Request failed: <urlopen error timed out>', 'failed': True, 'invocation': {'module_args': {'file': 'None', 'keyserver': 'None', 'url': 'http://download.ceph.com/keys/autobuild.asc', 'validate_certs': True, 'keyring': 'None', 'state': 'present', 'key': 'None', 'data': 'None', 'id': 'None'}}, '_ansible_no_log': False, 'changed': False, 'item': 'http://download.ceph.com/keys/autobuild.asc', 'ansible_loop_var': 'item', '_ansible_item_label': 'http://download.ceph.com/keys/autobuild.asc'}, {'msg': 'Failed to download key at http://download.ceph.com/keys/release.asc: Request failed: <urlopen error timed out>', 'failed': True, 'invocation': {'module_args': {'file': 'None', 'keyserver': 'None', 'url': 'http://download.ceph.com/keys/release.asc', 'validate_certs': True, 'keyring': 'None', 'state': 'present', 'key': 'None', 'data': 'None', 'id': 'None'}}, '_ansible_no_log': False, 'changed': False, 'item': 'http://download.ceph.com/keys/release.asc', 'ansible_loop_var': 'item', '_ansible_item_label': 'http://download.ceph.com/keys/release.asc'}], 'msg': 'All items completed', 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 199, in represent_list return self.represent_sequence(\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 92, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', 'http://download.ceph.com/keys/autobuild.asc')"}

dead 5359809 2020-08-19 14:26:11 2020-08-19 16:00:36 2020-08-20 04:03:14 12:02:38 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} 5
fail 5359810 2020-08-19 14:26:12 2020-08-19 16:02:24 2020-08-19 19:08:28 3:06:04 2:58:46 0:07:18 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} 4
Failure Reason:

Command failed (ragweed tests against rgw) on smithi045 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

fail 5359811 2020-08-19 14:26:13 2020-08-19 16:02:24 2020-08-19 19:24:28 3:22:04 3:14:33 0:07:31 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} 4
Failure Reason:

Command failed (ragweed tests against rgw) on smithi029 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

pass 5359812 2020-08-19 14:26:14 2020-08-19 16:04:24 2020-08-19 19:36:29 3:32:05 3:24:00 0:08:05 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} 5
dead 5359813 2020-08-19 14:26:14 2020-08-19 16:04:24 2020-08-20 04:08:47 12:04:23 11:56:53 0:07:30 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04}
Failure Reason:

psutil.NoSuchProcess process no longer exists (pid=6159)

fail 5359814 2020-08-19 14:26:15 2020-08-19 16:06:16 2020-08-19 19:12:20 3:06:04 2:59:49 0:06:15 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} 4
Failure Reason:

Command failed (ragweed tests against rgw) on smithi172 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

fail 5359815 2020-08-19 14:26:16 2020-08-19 16:12:25 2020-08-19 19:36:29 3:24:04 3:17:25 0:06:39 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} 4
Failure Reason:

Command failed (ragweed tests against rgw) on smithi037 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

dead 5359816 2020-08-19 14:26:17 2020-08-19 16:12:25 2020-08-20 04:16:16 12:03:51 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} 5
fail 5359817 2020-08-19 14:26:18 2020-08-19 16:16:25 2020-08-19 16:28:24 0:11:59 0:04:05 0:07:54 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} 5
Failure Reason:

ceph version 14.1.0-1bionic was not installed, found 14.2.11-1bionic.

fail 5359818 2020-08-19 14:26:19 2020-08-19 16:16:25 2020-08-19 16:28:24 0:11:59 0:03:50 0:08:09 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} 4
Failure Reason:

ceph version 14.1.0-1bionic was not installed, found 14.2.11-1bionic.

fail 5359819 2020-08-19 14:26:19 2020-08-19 16:18:35 2020-08-19 16:32:34 0:13:59 0:06:36 0:07:23 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} 4
Failure Reason:

Command failed on smithi141 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5359820 2020-08-19 14:26:20 2020-08-19 16:20:33 2020-08-19 16:34:33 0:14:00 0:06:24 0:07:36 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} 5
Failure Reason:

Command failed on smithi157 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5359821 2020-08-19 14:26:21 2020-08-19 16:20:34 2020-08-19 16:34:33 0:13:59 0:06:44 0:07:15 smithi master ubuntu 18.04 upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} 5
Failure Reason:

Command failed on smithi129 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'

fail 5359822 2020-08-19 14:26:22 2020-08-19 16:20:34 2020-08-19 16:34:33 0:13:59 0:06:03 0:07:56 smithi master ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} 4
Failure Reason:

Command failed on smithi086 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json'