Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi197.front.sepia.ceph.com smithi True True 2019-10-20 23:43:59.329980 bhubbard@teuthology ubuntu 18.04 x86_64 /home/bhubbard/working/archive/
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version
fail 4427261 2019-10-20 05:30:31 2019-10-20 23:13:48 2019-10-20 23:39:46 0:25:58 0:02:49 0:23:09 smithi master ubuntu 14.04
Failure Reason:

urn self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")Failure object was: {'smithi197.front.sepia.ceph.com': {'msg': u'All items completed', 'changed': False, 'results': [{'ansible_loop_var': u'item', '_ansible_no_log': False, u'err': u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n", 'item': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, 'changed': False, u'failed': True, '_ansible_item_label': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, u'rc': 3, u'invocation': {u'module_args': {u'vg_options': u'', u'pvs': [u'/dev/nvme0n1'], u'force': False, u'vg': u'vg_nvme', u'pesize': u'4', u'pv_options': u'', u'state': u'present'}}, u'msg': u'Failed executing pvs command.'}], 'warnings': [u"The value 4 (type int) in a string field was converted to u'4' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")Failure object was: {'smithi157.front.sepia.ceph.com': {'msg': u'All items completed', 'changed': False, 'results': [{'ansible_loop_var': u'item', '_ansible_no_log': False, u'err': u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n", 'item': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, 'changed': False, u'failed': True, '_ansible_item_label': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, u'rc': 3, u'invocation': {u'module_args': {u'vg_options': u'', u'pvs': [u'/dev/nvme0n1'], u'force': False, u'vg': u'vg_nvme', u'pesize': u'4', u'pv_options': u'', u'state': u'present'}}, u'msg': u'Failed executing pvs command.'}], 'warnings': [u"The value 4 (type int) in a string field was converted to u'4' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")

pass 4427174 2019-10-20 04:11:49 2019-10-20 22:19:20 2019-10-20 23:19:20 1:00:00 0:46:03 0:13:57 smithi master centos 7.5
pass 4427097 2019-10-20 04:10:53 2019-10-20 21:14:47 2019-10-20 22:20:47 1:06:00 0:18:16 0:47:44 smithi master rhel 7.5
pass 4427036 2019-10-20 04:01:18 2019-10-20 20:41:23 2019-10-20 21:03:22 0:21:59 0:11:50 0:10:09 smithi master
fail 4426965 2019-10-20 03:17:22 2019-10-20 19:35:06 2019-10-20 20:31:06 0:56:00 0:25:15 0:30:45 smithi master
Failure Reason:

"2019-10-20 20:10:00.247322 mon.a mon.0 172.21.15.197:6789/0 60 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log

pass 4425571 2019-10-19 09:50:39 2019-10-19 17:36:15 2019-10-19 18:04:14 0:27:59 0:11:29 0:16:30 smithi master ubuntu 18.04
pass 4425459 2019-10-19 09:49:12 2019-10-19 16:15:58 2019-10-19 17:37:58 1:22:00 0:12:19 1:09:41 smithi master ubuntu 18.04
pass 4425452 2019-10-19 09:49:06 2019-10-19 16:13:10 2019-10-19 17:13:10 1:00:00 0:48:48 0:11:12 smithi master ubuntu 18.04
pass 4425358 2019-10-19 09:47:54 2019-10-19 15:10:21 2019-10-19 16:12:21 1:02:00 0:40:41 0:21:19 smithi master centos 7.6
pass 4425339 2019-10-19 09:47:39 2019-10-19 14:51:49 2019-10-19 15:19:49 0:28:00 0:18:28 0:09:32 smithi master rhel 7.7
fail 4423113 2019-10-19 02:31:08 2019-10-19 02:56:18 2019-10-19 14:56:30 12:00:12 0:24:23 11:35:49 smithi master ubuntu 16.04
Failure Reason:

Command failed on smithi118 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1'

fail 4422715 2019-10-18 18:03:17 2019-10-19 13:44:55 2019-10-19 14:18:54 0:33:59 0:22:19 0:11:40 smithi master rhel 7.7
Failure Reason:

"2019-10-19T14:07:29.336973+0000 mon.b (mon.0) 95 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log

pass 4422705 2019-10-18 17:19:20 2019-10-18 22:46:49 2019-10-18 23:22:48 0:35:59 0:12:12 0:23:47 smithi master ubuntu
pass 4419322 2019-10-18 03:07:39 2019-10-18 09:52:36 2019-10-18 10:12:35 0:19:59 0:13:22 0:06:37 smithi master rhel 7.7
dead 4419224 2019-10-18 03:06:05 2019-10-18 08:35:27 2019-10-18 20:37:53 12:02:26 smithi master centos 7.6
pass 4419125 2019-10-18 02:25:39 2019-10-18 11:13:07 2019-10-18 23:01:19 11:48:12 2:12:16 9:35:56 smithi master ubuntu 18.04
pass 4419071 2019-10-18 00:51:07 2019-10-19 13:10:36 2019-10-19 13:44:36 0:34:00 0:20:03 0:13:57 smithi master
pass 4419013 2019-10-18 00:50:09 2019-10-19 12:15:34 2019-10-19 13:11:34 0:56:00 0:41:37 0:14:23 smithi master
pass 4418980 2019-10-18 00:49:37 2019-10-19 11:34:52 2019-10-19 12:16:51 0:41:59 0:14:57 0:27:02 smithi master
pass 4418953 2019-10-18 00:49:10 2019-10-19 10:54:53 2019-10-19 11:52:53 0:58:00 0:18:16 0:39:44 smithi master centos 7.4