Pulpito :: Results Dashboard

Posted	Started	Updated	Runtime	Duration	In Waiting	Machine	Teuthology Branch	OS Type	OS Version	Nodes	Status
2020-01-24 13:16:56	2020-01-24 14:53:58	2020-01-24 15:47:58	0:54:00	0:42:30	0:11:30	smithi	master	centos	7.6	3	fail

Nodes: smithi197 smithi155 smithi157

Description: rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml}

Log: http://qa-proxy.ceph.com/teuthology/sage-2020-01-24_13:15:58-rados-wip-sage2-testing-2020-01-23-1953-distro-basic-smithi/4700899/teuthology.log

Failure Reason:

repository, to point to a working', u' upstream. This is most often useful if you are using a newer', u' distribution release than is supported by the repository (and the', u' packages for the previous distribution release still work).', u'', u' 3. Run the command with the repository temporarily disabled', u' yum --disablerepo=epel ...', u'', u" 4. Disable the repository permanently, so yum won't use it by default. Yum", u' will then just ignore the repository until you permanently enable it', u' again or use --enablerepo for temporary usage:', u'', u' yum-config-manager --disable epel', u' or', u' subscription-manager repos --disable=epel', u'', u' 5. Configure the failing repository to be skipped, if it is unavailable.', u' Note that yum will try to contact the repo. when it runs most commands,', u' so will have to try and fail each time (and thus. yum will be be much', u' slower). If it is a very temporary problem though, this is often a nice', u' compromise:', u'', u' yum-config-manager --save --setopt=epel.skip_if_unavailable=true', u'', u'failure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"'], u'changed': True, u'stderr': u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 4733866 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/847ea7712ecf18929572ab1c59abfa4500e779f22773ff681df1d42620aa4fc3-updateinfo.xml.zck: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/847ea7712ecf18929572ab1c59abfa4500e779f22773ff681df1d42620aa4fc3-updateinfo.xml.zck: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 3318310 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"\nTrying other mirror.\n\n\n One of the configured repositories failed (Extra Packages for Enterprise Linux),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=epel ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable epel\n or\n subscription-manager repos --disable=epel\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=epel.skip_if_unavailable=true\n\nfailure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"', u'stdout': u'Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: mirror.cogentco.com\n * updates: mirror.math.princeton.edu', u'msg': u'non-zero return code', u'delta': u'0:25:24.864532', 'stdout_lines': [u'Loaded plugins: fastestmirror, langpacks, priorities', u'Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates', u'Loaded plugins: fastestmirror, langpacks, priorities', u'Determining fastest mirrors', u' * base: packages.oit.ncsu.edu', u' * extras: mirror.cogentco.com', u' * updates: mirror.math.princeton.edu'], u'end': u'2020-01-24 15:32:02.767192', '_ansible_no_log': False, u'cmd': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'start': u'2020-01-24 15:06:37.902660', u'warnings': [u"Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."], u'rc': 1, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rm -rf /var/cache/yum/*; yum clean all; yum makecache')Failure object was: {'smithi155.front.sepia.ceph.com': {'stderr_lines': [u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 6448522 bytes remaining to read"', u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 2044450 bytes remaining to read"', u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"', u'Trying other mirror.', u'', u'', u' One of the configured repositories failed (Extra Packages for Enterprise Linux),', u" and yum doesn't have enough cached data to continue. At this point the only", u' safe thing yum can do is fail. There are a few ways to work "fix" this:', u'', u' 1. Contact the upstream for the repository and get them to fix the problem.', u'', u' 2. Reconfigure the baseurl/etc. for the repository, to point to a working', u' upstream. This is most often useful if you are using a newer', u' distribution release than is supported by the repository (and the', u' packages for the previous distribution release still work).', u'', u' 3. Run the command with the repository temporarily disabled', u' yum --disablerepo=epel ...', u'', u" 4. Disable the repository permanently, so yum won't use it by default. Yum", u' will then just ignore the repository until you permanently enable it', u' again or use --enablerepo for temporary usage:', u'', u' yum-config-manager --disable epel', u' or', u' subscription-manager repos --disable=epel', u'', u' 5. Configure the failing repository to be skipped, if it is unavailable.', u' Note that yum will try to contact the repo. when it runs most commands,', u' so will have to try and fail each time (and thus. yum will be be much', u' slower). If it is a very temporary problem though, this is often a nice', u' compromise:', u'', u' yum-config-manager --save --setopt=epel.skip_if_unavailable=true', u'', u'failure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"'], u'changed': True, u'stderr': u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 6448522 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 2044450 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"\nTrying other mirror.\n\n\n One of the configured repositories failed (Extra Packages for Enterprise Linux),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=epel ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable epel\n or\n subscription-manager repos --disable=epel\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=epel.skip_if_unavailable=true\n\nfailure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"', u'stdout': u'Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: repos-va.psychz.net\n * updates: mirror.math.princeton.edu', u'msg': u'non-zero return code', u'delta': u'0:40:22.284715', 'stdout_lines': [u'Loaded plugins: fastestmirror, langpacks, priorities', u'Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates', u'Loaded plugins: fastestmirror, langpacks, priorities', u'Determining fastest mirrors', u' * base: packages.oit.ncsu.edu', u' * extras: repos-va.psychz.net', u' * updates: mirror.math.princeton.edu'], u'end': u'2020-01-24 15:46:59.083378', '_ansible_no_log': False, u'cmd': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'start': u'2020-01-24 15:06:36.798663', u'warnings': [u"Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."], u'rc': 1, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rm -rf /var/cache/yum/*; yum clean all; yum makecache')

All Details...

log_href: http://qa-proxy.ceph.com/teuthology/sage-2020-01-24_13:15:58-rados-wip-sage2-testing-2020-01-23-1953-distro-basic-smithi/4700899/teuthology.log
archive_path: /home/teuthworker/archive/sage-2020-01-24_13:15:58-rados-wip-sage2-testing-2020-01-23-1953-distro-basic-smithi/4700899
description: rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml}
duration: 0:42:30
email:
failure_reason: repository, to point to a working', u' upstream. This is most often useful if you are using a newer', u' distribution release than is supported by the repository (and the', u' packages for the previous distribution release still work).', u'', u' 3. Run the command with the repository temporarily disabled', u' yum --disablerepo=epel ...', u'', u" 4. Disable the repository permanently, so yum won't use it by default. Yum", u' will then just ignore the repository until you permanently enable it', u' again or use --enablerepo for temporary usage:', u'', u' yum-config-manager --disable epel', u' or', u' subscription-manager repos --disable=epel', u'', u' 5. Configure the failing repository to be skipped, if it is unavailable.', u' Note that yum will try to contact the repo. when it runs most commands,', u' so will have to try and fail each time (and thus. yum will be be much', u' slower). If it is a very temporary problem though, this is often a nice', u' compromise:', u'', u' yum-config-manager --save --setopt=epel.skip_if_unavailable=true', u'', u'failure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"'], u'changed': True, u'stderr': u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 4733866 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/847ea7712ecf18929572ab1c59abfa4500e779f22773ff681df1d42620aa4fc3-updateinfo.xml.zck: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/847ea7712ecf18929572ab1c59abfa4500e779f22773ff681df1d42620aa4fc3-updateinfo.xml.zck: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 3318310 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"\nTrying other mirror.\n\n\n One of the configured repositories failed (Extra Packages for Enterprise Linux),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=epel ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable epel\n or\n subscription-manager repos --disable=epel\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=epel.skip_if_unavailable=true\n\nfailure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 1708046 bytes remaining to read"', u'stdout': u'Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: mirror.cogentco.com\n * updates: mirror.math.princeton.edu', u'msg': u'non-zero return code', u'delta': u'0:25:24.864532', 'stdout_lines': [u'Loaded plugins: fastestmirror, langpacks, priorities', u'Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates', u'Loaded plugins: fastestmirror, langpacks, priorities', u'Determining fastest mirrors', u' * base: packages.oit.ncsu.edu', u' * extras: mirror.cogentco.com', u' * updates: mirror.math.princeton.edu'], u'end': u'2020-01-24 15:32:02.767192', '_ansible_no_log': False, u'cmd': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'start': u'2020-01-24 15:06:37.902660', u'warnings': [u"Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."], u'rc': 1, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rm -rf /var/cache/yum/*; yum clean all; yum makecache')Failure object was: {'smithi155.front.sepia.ceph.com': {'stderr_lines': [u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 6448522 bytes remaining to read"', u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 2044450 bytes remaining to read"', u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'Trying other mirror.', u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"', u'Trying other mirror.', u'', u'', u' One of the configured repositories failed (Extra Packages for Enterprise Linux),', u" and yum doesn't have enough cached data to continue. At this point the only", u' safe thing yum can do is fail. There are a few ways to work "fix" this:', u'', u' 1. Contact the upstream for the repository and get them to fix the problem.', u'', u' 2. Reconfigure the baseurl/etc. for the repository, to point to a working', u' upstream. This is most often useful if you are using a newer', u' distribution release than is supported by the repository (and the', u' packages for the previous distribution release still work).', u'', u' 3. Run the command with the repository temporarily disabled', u' yum --disablerepo=epel ...', u'', u" 4. Disable the repository permanently, so yum won't use it by default. Yum", u' will then just ignore the repository until you permanently enable it', u' again or use --enablerepo for temporary usage:', u'', u' yum-config-manager --disable epel', u' or', u' subscription-manager repos --disable=epel', u'', u' 5. Configure the failing repository to be skipped, if it is unavailable.', u' Note that yum will try to contact the repo. when it runs most commands,', u' so will have to try and fail each time (and thus. yum will be be much', u' slower). If it is a very temporary problem though, this is often a nice', u' compromise:', u'', u' yum-config-manager --save --setopt=epel.skip_if_unavailable=true', u'', u'failure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.', u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u"http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds')", u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"'], u'changed': True, u'stderr': u'http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 6448522 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 2044450 bytes remaining to read"\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nTrying other mirror.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"\nTrying other mirror.\n\n\n One of the configured repositories failed (Extra Packages for Enterprise Linux),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=epel ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable epel\n or\n subscription-manager repos --disable=epel\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=epel.skip_if_unavailable=true\n\nfailure: repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2 from epel: [Errno 256] No more mirrors to try.\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 12] Timeout on http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: (28, \'Operation too slow. Less than 1000 bytes/sec transferred the last 300 seconds\')\nhttp://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0878257236e2d6fa937b34d0f8b5a3551ee8b90f497fd308734bd43c44fe6b4-filelists.sqlite.bz2: [Errno 14] curl#18 - "transfer closed with 820908 bytes remaining to read"', u'stdout': u'Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: repos-va.psychz.net\n * updates: mirror.math.princeton.edu', u'msg': u'non-zero return code', u'delta': u'0:40:22.284715', 'stdout_lines': [u'Loaded plugins: fastestmirror, langpacks, priorities', u'Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates', u'Loaded plugins: fastestmirror, langpacks, priorities', u'Determining fastest mirrors', u' * base: packages.oit.ncsu.edu', u' * extras: repos-va.psychz.net', u' * updates: mirror.math.princeton.edu'], u'end': u'2020-01-24 15:46:59.083378', '_ansible_no_log': False, u'cmd': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'start': u'2020-01-24 15:06:36.798663', u'warnings': [u"Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."], u'rc': 1, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rm -rf /var/cache/yum/*; yum clean all; yum makecache')
flavor:
job_id: 4700899
kernel:
- sha1: distro
- kdb: True
last_in_suite: False
machine_type: smithi
name: sage-2020-01-24_13:15:58-rados-wip-sage2-testing-2020-01-23-1953-distro-basic-smithi
nuke_on_error: True
os_type: centos
os_version: 7.6
overrides:
- ceph-deploy:
  - conf:
    - client:
      - log file: /var/log/ceph/ceph-$name.$pid.log
    - mon:
      - osd default pool size: 2
- workunit:
  - sha1: cc5e8d14f86da9a296c11c7a9ef7c6a8f147941f
  - branch: wip-sage2-testing-2020-01-23-1953
- ceph:
  - log-whitelist:
    - $MDS_ALL_DOWN$
    - $MDS_UP_LESS_THAN_MAX$
    - $MON_DOWN$
    - $OSD_SLOW_PING_TIME
    - but it is still running
    - objects unfound and apparently lost
    - overall HEALTH_
    - \(OSDMAP_FLAGS$
    - $OSD_
    - \(PG_
    - \(POOL_
    - \(CACHE_POOL_
    - \(SMALLER_PGP_NUM$
    - $OBJECT_
    - \(SLOW_OPS$
    - $REQUEST_SLOW$
    - $TOO_FEW_PGS$
    - slow requests
  - mon_bind_msgr2: False
  - conf:
    - global:
      - ms tcp read timeout: 5
      - osd_pool_default_size: 3
      - osd_pool_default_min_size: 2
      - ms type: async
      - ms bind msgr2: False
      - mon client directed command retry: 5
      - ms inject socket failures: 2500
    - mon:
      - debug mon: 20
      - debug paxos: 20
      - mon min osdmap epochs: 50
      - debug ms: 1
      - mon osdmap full prune min: 15
      - mon osdmap full prune interval: 2
      - mon osdmap full prune txsize: 2
      - paxos service trim min: 10
    - osd:
      - osd op queue cut off: debug_random
      - debug ms: 1
      - debug journal: 20
      - osd scrub max interval: 120
      - osd max backfills: 3
      - debug osd: 25
      - osd snap trim sleep: 2
      - osd debug reject backfill probability: 0.3
      - osd debug verify missing on start: True
      - osd op queue: debug_random
      - osd backoff on peering: True
      - osd debug verify cached snaps: True
      - osd scrub min interval: 60
      - debug filestore: 20
      - osd shutdown pgref assert: True
      - osd backoff on degraded: True
  - sha1: cc5e8d14f86da9a296c11c7a9ef7c6a8f147941f
- install:
  - ceph:
    - sha1: cc5e8d14f86da9a296c11c7a9ef7c6a8f147941f
- admin_socket:
  - branch: wip-sage2-testing-2020-01-23-1953
- thrashosds:
  - min_in: 4
owner: scheduled_sage@teuthology
pid:
roles:
- ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
- ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']
- ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2']
sentry_event:
status: fail
success: False
branch: wip-sage2-testing-2020-01-23-1953
seed:
sha1: cc5e8d14f86da9a296c11c7a9ef7c6a8f147941f
subset:
suite:
suite_branch: wip-sage2-testing-2020-01-23-1953
suite_path:
suite_relpath:
suite_repo:
suite_sha1: cc5e8d14f86da9a296c11c7a9ef7c6a8f147941f
targets:
- smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5+HEQLIInE90Olct4wLCmx4l3OtDFplNW9636liNDZ0ZfuFV14in4nV6VF8mkIMm9+vd5Qh0NNPQNP+dbpnUeZw/VEZJNSZ5s9uP/nitvWwecYDH6eVkdoJNImIwpbWrGqeQY7NL26ScV/tuloVKTIqahEQs9Zw0nFPCrToyl9sZAGgfD2YaFm/wqKKSKr7vo9SJd8f4UmAdtd1mUh24AcEF18xyPMretq0ABz83u5T2MJXIehheXgf18dr1pNSTHZw1H3n0gxkHChhmAqhIAFvNzShaDS9SC2nAHGtQc/b1fsRsn2I1qWcOhER5iDCUtRb8H+wqbq6GHvIxTfEFd
- smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPA7KBO/APW0SFV5ue+XHIWdHuGVH3PsO6iF9e5CbWyg0h4VO2V5OxyEfOG0P9HmOArCC2pTp7XcmRR7UUZ/wTT9HeuGQaAVdmw1wHMJOQiveDxkdUKFbnsrXLjjicnP7CyHMVTQBlz+xjEom7QOuHIA/pcbHwDOL2ykWg7i1kKZs7soLSMOOeRe7k+CHmBXCHsGDKbWxPhak/vh5mxeB+Q+31E+ggMe1Zsfsuqb8lu2ag1XFJ/NTha5XrXtPbsV73D0UdqHA+m8KBjVS1MjGHuzrwpfpLlMsebL1Sa/FjIs343+LvwCjLGaCub52rAMCs078u26F9rBQhy8QlpTTh
- smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBjpG3lzCB8ITlnhjw37vDeAd73gK25Re30H9mt4ZWnHcS598Fc1Bc2oSq9przSKK+6uz7wASAGTApRbprXXXg44uKbjYh1dpnNueMA245PdlrIK2JW/nicPuekWGQWbsHgO40BukAPG6ORWG4rFJB9olqxTxBiLM0PRZrEyWgsPHFy1DRkokq5KVa2BIhpuGTgg5zpEOq8P2GdwLi8t92hKIyIC2j+isEvirudXq6o8ylhy6Yg5bo2Zo7oV5MVJD2g2r5jkaHpRG1pQV4qHq1E/mqsqRppDCMZ2W5g6a1mnqu+UhGTBlqSnqNcJBp3yemi2Ii/XOqQGnDXnSNhehH
tasks:
- internal.check_packages:
- internal.buildpackages_prep:
- internal.lock_machines:
  - 3
  - smithi
- internal.save_config:
- internal.check_lock:
- internal.add_remotes:
- console_log:
- internal.connect:
- internal.push_inventory:
- internal.serialize_remote_roles:
- internal.check_conflict:
- internal.check_ceph_data:
- internal.vm_setup:
- kernel:
  - sha1: distro
  - kdb: True
- internal.base:
- internal.archive_upload:
- internal.archive:
- internal.coredump:
- internal.sudo:
- internal.syslog:
- internal.timer:
- pcp:
- selinux:
- ansible.cephlab:
- clock:
- install:
  - extra_packages:
    - librados2
    - python-rados
    - python-rgw
    - python-rbd
    - python-cephfs
  - exclude_packages:
    - librados3
    - ceph-mgr-dashboard
    - ceph-mgr-diskprediction-local
    - ceph-mgr-rook
    - ceph-mgr-cephadm
    - cephadm
    - ceph-immutable-object-cache
    - ceph-base
    - python3-rados
    - python3-rgw
    - python3-rbd
    - python3-cephfs
    - librados-devel
  - branch: luminous
- cephadm:
- thrashosds:
  - chance_pgnum_grow: 1
  - chance_pgpnum_fix: 1
  - timeout: 1200
teuthology_branch: master
verbose: False
pcp_grafana_url:
priority:
user:
queue:
posted: 2020-01-24 13:16:56
started: 2020-01-24 14:53:58
updated: 2020-01-24 15:47:58
status_class: danger
runtime: 0:54:00
wait_time: 0:11:30