Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi144.front.sepia.ceph.com smithi True True 2020-01-26 03:33:07.976546 scheduled_pdonnell@teuthology ubuntu 18.04 x86_64 /home/teuthworker/archive/pdonnell-2020-01-25_00:13:49-multimds-wip-pdonnell-testing-20200124.211519-distro-basic-smithi/4703907
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version
fail 4707678 2020-01-25 20:49:24 2020-01-26 00:12:08 2020-01-26 01:32:08 1:20:00 0:57:59 0:22:01 smithi master centos 8.0
Failure Reason:

reached maximum tries (50) after waiting for 300 seconds

pass 4707601 2020-01-25 20:48:07 2020-01-25 23:19:51 2020-01-25 23:45:50 0:25:59 0:15:23 0:10:36 smithi master centos 8.0
pass 4707550 2020-01-25 20:47:16 2020-01-25 22:45:28 2020-01-25 23:21:27 0:35:59 0:27:48 0:08:11 smithi master rhel 8.0
fail 4707503 2020-01-25 20:46:30 2020-01-25 22:01:04 2020-01-25 22:47:04 0:46:00 0:34:50 0:11:10 smithi master centos 8.0
Failure Reason:

Test failure: test_stale_renew (tasks.cephfs.test_client_recovery.TestClientRecovery)

pass 4707454 2020-01-25 20:45:41 2020-01-25 21:10:08 2020-01-25 22:04:08 0:54:00 0:44:39 0:09:21 smithi master ubuntu 18.04
pass 4707330 2020-01-25 07:00:55 2020-01-25 12:58:52 2020-01-25 13:52:53 0:54:01 0:12:53 0:41:08 smithi master
pass 4706103 2020-01-25 05:00:47 2020-01-25 12:05:41 2020-01-25 13:29:42 1:24:01 0:25:02 0:58:59 smithi master centos 8.0
pass 4706087 2020-01-25 05:00:36 2020-01-25 11:53:11 2020-01-25 12:53:11 1:00:00 0:23:10 0:36:50 smithi master ubuntu 18.04
fail 4706075 2020-01-25 04:59:42 2020-01-25 11:37:21 2020-01-25 14:29:23 2:52:02 0:26:17 2:25:45 smithi master centos 7.6
Failure Reason:

SELinux denials found on ubuntu@smithi144.front.sepia.ceph.com: ['type=AVC msg=audit(1579961941.770:6401): avc: denied { read } for pid=16799 comm="logrotate" name="4bc8dd0c-3f7c-11ea-99dd-001a4aab830c" dev="sda1" ino=527058 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1579961941.770:6402): avc: denied { getattr } for pid=16799 comm="logrotate" path="/var/log/ceph/4bc8dd0c-3f7c-11ea-99dd-001a4aab830c/ceph-mon.c.log" dev="sda1" ino=527091 scontext=system_u:system_r:logrotate_t:s0-s0:c0.c1023 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1']

pass 4706015 2020-01-25 04:58:43 2020-01-25 10:41:01 2020-01-25 11:25:01 0:44:00 0:25:20 0:18:40 smithi master ubuntu 18.04
pass 4704955 2020-01-25 02:31:22 2020-01-25 03:38:19 2020-01-25 10:12:27 6:34:08 3:14:18 3:19:50 smithi master ubuntu 16.04
pass 4704910 2020-01-25 02:30:48 2020-01-25 02:37:52 2020-01-25 06:47:57 4:10:05 3:48:10 0:21:55 smithi master ubuntu 18.04
fail 4703868 2020-01-25 00:17:27 2020-01-26 03:02:30 2020-01-26 03:33:11 0:30:41 0:02:27 0:28:14 smithi master
Failure Reason:

t_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rpm')

fail 4703840 2020-01-25 00:16:59 2020-01-26 02:38:49 2020-01-26 02:54:48 0:15:59 smithi master
Failure Reason:

Command failed on smithi057 with status 1: 'sudo yum install -y kernel'

fail 4703824 2020-01-25 00:16:42 2020-01-26 02:20:10 2020-01-26 03:18:10 0:58:00 0:18:01 0:39:59 smithi master rhel 8.0
Failure Reason:

{'Failure object was': {'smithi123.front.sepia.ceph.com': {'changed': False, "u'invocation'": {"u'module_args'": {"u'list'": 'None', "u'disable_plugin'": [], "u'install_weak_deps'": True, "u'name'": ["u'@core'", "u'@base'", "u'dnf-utils'", "u'git-all'", "u'sysstat'", "u'libedit'", "u'boost-thread'", "u'xfsprogs'", "u'gdisk'", "u'parted'", "u'libgcrypt'", "u'fuse-libs'", "u'openssl'", "u'libuuid'", "u'attr'", "u'ant'", "u'lsof'", "u'gettext'", "u'bc'", "u'xfsdump'", "u'blktrace'", "u'usbredir'", "u'podman'", "u'podman-docker'", "u'libev-devel'", "u'valgrind'", "u'nfs-utils'", "u'ncurses-devel'", "u'gcc'", "u'git'", "u'python3-nose'", "u'python3-virtualenv'", "u'genisoimage'", "u'qemu-img'", "u'qemu-kvm-core'", "u'qemu-kvm-block-rbd'", "u'libacl-devel'", "u'dbench'", "u'autoconf'"], "u'releasever'": 'None', "u'security'": False, "u'conf_file'": 'None', "u'autoremove'": False, "u'update_cache'": False, "u'disablerepo'": [], "u'skip_broken'": False, "u'disable_gpg_check'": False, "u'lock_timeout'": 0, "u'download_only'": False, "u'enablerepo'": [], "u'disable_excludes'": 'None', "u'download_dir'": 'None', "u'state'": "u'present'", "u'install_repoquery'": True, "u'allow_downgrade'": False, "u'validate_certs'": True, "u'exclude'": [], "u'bugfix'": False, "u'enable_plugin'": [], "u'update_only'": False, "u'installroot'": "u'/'"}}, "u'rc'": 1, "u'failures'": ["u'No package dbench available.'"], "u'msg'": ["u'Failed to install some of the specified packages'"], "u'results'": [], '_ansible_no_log': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u\'tag:yaml.org,2002:seq\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'RepresenterError': "('cannot represent an object', u'No package dbench available.')"}

fail 4703815 2020-01-25 00:16:33 2020-01-26 02:14:45 2020-01-26 02:42:45 0:28:00 0:03:36 0:24:24 smithi master ubuntu 18.04
Failure Reason:

l then just ignore the repository until you permanently enable it', u' again or use --enablerepo for temporary usage:', u'', u' yum-config-manager --disable <repoid>', u' or', u' subscription-manager repos --disable=<repoid>', u'', u' 5. Configure the failing repository to be skipped, if it is unavailable.', u' Note that yum will try to contact the repo. when it runs most commands,', u' so will have to try and fail each time (and thus. yum will be be much', u' slower). If it is a very temporary problem though, this is often a nice', u' compromise:', u'', u' yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true', u'', u'base: Check uncompressed DB failed'], u'changed': True, u'stderr': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/filelists.xml.gz: [Errno 16] error opening local file from http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/filelists.xml.gz, IOError: [Errno 2] No such file or directory: \'/var/cache/yum/x86_64/7/centos7-fcgi-ceph/filelists.xml.gz\'\nTrying other mirror.\nhttp://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/primary.xml.gz: [Errno 16] error opening local file from http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/primary.xml.gz, IOError: [Errno 2] No such file or directory: \'/var/cache/yum/x86_64/7/centos7-fcgi-ceph/primary.xml.gz\'\nTrying other mirror.\nhttp://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/other.xml.gz: [Errno 16] error opening local file from http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/repodata/other.xml.gz, IOError: [Errno 2] No such file or directory: \'/var/cache/yum/x86_64/7/centos7-fcgi-ceph/other.xml.gz\'\nTrying other mirror.\n\n\n One of the configured repositories failed (Unknown),\n and yum doesn\'t have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work "fix" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=<repoid> ...\n\n 4. Disable the repository permanently, so yum won\'t use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable <repoid>\n or\n subscription-manager repos --disable=<repoid>\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true\n\nbase: Check uncompressed DB failed', u'stdout': u'Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.math.princeton.edu\n * extras: mirror.math.princeton.edu\n * updates: mirror.math.princeton.edu', u'msg': u'non-zero return code', u'delta': u'0:00:03.767377', 'stdout_lines': [u'Loaded plugins: fastestmirror, langpacks, priorities', u'Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates', u'Loaded plugins: fastestmirror, langpacks, priorities', u'Determining fastest mirrors', u' * base: mirror.math.princeton.edu', u' * extras: mirror.math.princeton.edu', u' * updates: mirror.math.princeton.edu'], u'end': u'2020-01-26 02:40:36.287636', '_ansible_no_log': False, u'cmd': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'start': u'2020-01-26 02:40:32.520259', u'warnings': [u"Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."], u'rc': 1, u'invocation': {u'module_args': {u'warn': True, u'executable': None, u'_uses_shell': True, u'strip_empty_ends': True, u'_raw_params': u'rm -rf /var/cache/yum/*; yum clean all; yum makecache', u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin_add_newline': True, u'stdin': None}}}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'rm -rf /var/cache/yum/*; yum clean all; yum makecache')

fail 4703805 2020-01-25 00:16:22 2020-01-26 02:04:32 2020-01-26 02:20:31 0:15:59 smithi master
Failure Reason:

Command failed on smithi057 with status 100: 'sudo apt-get clean'

pass 4703754 2020-01-25 00:15:29 2020-01-26 01:24:24 2020-01-26 02:12:24 0:48:00 0:32:43 0:15:17 smithi master rhel 8.0
pass 4703741 2020-01-25 00:15:15 2020-01-25 20:35:51 2020-01-25 21:09:50 0:33:59 0:16:17 0:17:42 smithi master
pass 4703723 2020-01-25 00:14:56 2020-01-25 20:12:42 2020-01-25 20:42:42 0:30:00 0:11:22 0:18:38 smithi master