Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 4399546 2019-10-10 14:24:58 2019-10-10 14:26:15 2019-10-10 14:44:14 0:17:59 0:01:07 0:16:52 smithi master rhel 7.6 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-low-osd-mem-target.yaml overrides.yaml rgw_pool_type/ec.yaml supported-random-distro$/{rhel_7.yaml}} 2
Failure Reason:

/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'/')Failure object was: {'smithi083.front.sepia.ceph.com': {'_ansible_no_log': False, u'warnings': [u'Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually'], u'invocation': {u'module_args': {u'install_weak_deps': True, u'autoremove': False, u'lock_timeout': 0, u'download_dir': None, u'install_repoquery': True, u'enable_plugin': [], u'update_cache': False, u'conf_file': None, u'exclude': [], u'update_only': False, u'installroot': u'/', u'allow_downgrade': False, u'name': [u'http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'], u'download_only': False, u'bugfix': False, u'list': None, u'disable_gpg_check': False, u'disable_excludes': None, u'use_backend': u'auto', u'state': u'present', u'disablerepo': [], u'releasever': None, u'disable_plugin': [], u'enablerepo': [], u'skip_broken': False, u'security': False, u'validate_certs': False}}, 'changed': False, u'msg': u'Failure downloading http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm, Request failed: <urlopen error [Errno 111] Connection refused>'}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u'/')

pass 4399547 2019-10-10 14:24:59 2019-10-10 14:26:16 2019-10-10 15:04:15 0:37:59 0:24:39 0:13:20 smithi master rgw/crypt/{0-cluster/fixed-1.yaml 1-ceph-install/install.yaml 2-kms/barbican.yaml 3-rgw/rgw.yaml 4-tests/{s3tests.yaml}} 1
pass 4399548 2019-10-10 14:25:00 2019-10-10 14:26:19 2019-10-10 14:56:19 0:30:00 0:17:00 0:13:00 smithi master centos 7.6 rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/default.yaml s3a-hadoop.yaml supported-distros/centos_latest.yaml} 2
pass 4399549 2019-10-10 14:25:01 2019-10-10 14:26:31 2019-10-10 14:50:31 0:24:00 0:10:22 0:13:38 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-low-osd-mem-target.yaml overrides.yaml rgw_pool_type/ec-profile.yaml tasks/rgw_roundtrip.yaml} 2
fail 4399550 2019-10-10 14:25:02 2019-10-10 14:27:04 2019-10-10 17:21:06 2:54:02 2:38:05 0:15:57 smithi master centos rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} 2
Failure Reason:

rgw multisite, rgw multisite pubsub, test failures

fail 4399551 2019-10-10 14:25:02 2019-10-10 14:27:40 2019-10-10 15:15:40 0:48:00 0:27:25 0:20:35 smithi master ubuntu 18.04 rgw/tempest/{clusters/fixed-1.yaml distro/supported-random-distro$/{ubuntu_latest.yaml} frontend/beast.yaml tasks/rgw_tempest.yaml} 1
Failure Reason:

Command failed on smithi151 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '(tempest.api.object_storage)(?!.*test_list_containers_reverse_order.*)(?!.*test_list_container_contents_with_end_marker.*)(?!.*test_delete_non_empty_container.*)(?!.*test_container_synchronization.*)(?!.*test_get_object_after_expiration_time.*)(?!.*test_create_object_with_transfer_encoding.*)'"

fail 4399552 2019-10-10 14:25:03 2019-10-10 14:27:48 2019-10-10 15:29:48 1:02:00 0:18:58 0:43:02 smithi master centos rgw/website/{clusters/fixed-2.yaml frontend/beast.yaml http.yaml overrides.yaml tasks/s3tests-website.yaml} 2
Failure Reason:

Command failed (s3 tests against rgw) on smithi202 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'"

pass 4399553 2019-10-10 14:25:04 2019-10-10 14:29:48 2019-10-10 15:23:47 0:53:59 0:30:38 0:23:21 smithi master rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/filestore-xfs.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_readwrite.yaml} 2
pass 4399554 2019-10-10 14:25:05 2019-10-10 14:29:48 2019-10-10 15:17:47 0:47:59 0:13:36 0:34:23 smithi master ubuntu 18.04 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-stupid.yaml overrides.yaml rgw_pool_type/replicated.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
fail 4399555 2019-10-10 14:25:06 2019-10-10 14:29:48 2019-10-10 15:35:48 1:06:00 0:24:44 0:41:16 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-stupid.yaml overrides.yaml rgw_pool_type/ec.yaml tasks/rgw_s3tests.yaml} 2
Failure Reason:

Command failed (s3 tests against rgw) on smithi103 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'"

pass 4399556 2019-10-10 14:25:06 2019-10-10 14:29:54 2019-10-10 15:01:53 0:31:59 0:13:43 0:18:16 smithi master rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-bitmap.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_roundtrip.yaml} 2
pass 4399557 2019-10-10 14:25:07 2019-10-10 14:31:25 2019-10-10 15:07:25 0:36:00 0:18:44 0:17:16 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/filestore-xfs.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_swift.yaml} 2
pass 4399558 2019-10-10 14:25:08 2019-10-10 14:31:27 2019-10-10 14:57:27 0:26:00 0:13:33 0:12:27 smithi master centos 7.6 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/filestore-xfs.yaml overrides.yaml rgw_pool_type/ec-profile.yaml supported-random-distro$/{centos_7.yaml}} 2
fail 4399559 2019-10-10 14:25:09 2019-10-10 14:33:34 2019-10-10 15:13:33 0:39:59 0:15:38 0:24:21 smithi master centos 7.6 rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/trunk.yaml s3a-hadoop.yaml supported-distros/centos_latest.yaml} 2
Failure Reason:

Command failed on smithi046 with status 1: 'cd /home/ubuntu/cephtest/hadoop/hadoop-tools/hadoop-aws/ && rm src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java && /home/ubuntu/cephtest/apache-maven-3.6.2/bin/mvn test -Dtest=S3a*,TestS3A*'

pass 4399560 2019-10-10 14:25:10 2019-10-10 14:33:43 2019-10-10 14:57:43 0:24:00 0:10:59 0:13:01 smithi master ubuntu rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-bitmap.yaml overrides.yaml rgw_pool_type/ec-profile.yaml tasks/rgw_user_quota.yaml} 2
fail 4399561 2019-10-10 14:25:11 2019-10-10 14:33:50 2019-10-10 15:29:50 0:56:00 0:43:41 0:12:19 smithi master centos rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/two-zonegroup.yaml tasks/test_multi.yaml valgrind.yaml} 2
Failure Reason:

rgw multisite, rgw multisite pubsub, test failures

fail 4399562 2019-10-10 14:25:11 2019-10-10 14:35:00 2019-10-10 14:54:59 0:19:59 0:11:10 0:08:49 smithi master centos rgw/tempest/{clusters/fixed-1.yaml distro/valgrind.yaml frontend/civetweb.yaml tasks/rgw_tempest.yaml} 1
Failure Reason:

Command failed on smithi190 with status 1: 'sudo yum -y install python34-devel'

fail 4399563 2019-10-10 14:25:12 2019-10-10 14:35:00 2019-10-10 15:20:59 0:45:59 0:25:00 0:20:59 smithi master rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-comp.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_s3tests.yaml} 2
Failure Reason:

Command failed (s3 tests against rgw) on smithi083 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'"

pass 4399564 2019-10-10 14:25:13 2019-10-10 14:35:41 2019-10-10 15:03:41 0:28:00 0:16:29 0:11:31 smithi master rgw/crypt/{0-cluster/fixed-1.yaml 1-ceph-install/install.yaml 2-kms/testing.yaml 3-rgw/rgw.yaml 4-tests/{s3tests.yaml}} 1
pass 4399565 2019-10-10 14:25:14 2019-10-10 14:35:42 2019-10-10 15:35:42 1:00:00 0:29:58 0:30:02 smithi master ubuntu rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-comp.yaml overrides.yaml rgw_pool_type/ec.yaml tasks/rgw_bucket_quota.yaml} 2
pass 4399566 2019-10-10 14:25:15 2019-10-10 14:36:25 2019-10-10 15:04:24 0:27:59 0:13:32 0:14:27 smithi master ubuntu 18.04 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-bitmap.yaml overrides.yaml rgw_pool_type/ec.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
pass 4399567 2019-10-10 14:25:16 2019-10-10 14:36:25 2019-10-10 15:20:24 0:43:59 0:13:27 0:30:32 smithi master rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-low-osd-mem-target.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_swift.yaml} 2
pass 4399568 2019-10-10 14:25:16 2019-10-10 14:36:25 2019-10-10 15:24:24 0:47:59 0:25:06 0:22:53 smithi master ubuntu rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-low-osd-mem-target.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_multipart_upload.yaml} 2
pass 4399569 2019-10-10 14:25:17 2019-10-10 14:36:37 2019-10-10 15:16:37 0:40:00 0:26:36 0:13:24 smithi master ubuntu 18.04 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-comp.yaml overrides.yaml rgw_pool_type/replicated.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
pass 4399570 2019-10-10 14:25:18 2019-10-10 14:37:43 2019-10-10 15:17:43 0:40:00 0:25:15 0:14:45 smithi master ubuntu rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-stupid.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_user_quota.yaml} 2
fail 4399571 2019-10-10 14:25:19 2019-10-10 14:38:58 2019-10-10 16:20:59 1:42:01 1:28:35 0:13:26 smithi master centos 7.6 rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/v28.yaml s3a-hadoop.yaml supported-distros/centos_latest.yaml} 2
Failure Reason:

Command failed on smithi080 with status 1: 'cd /home/ubuntu/cephtest/hadoop/hadoop-tools/hadoop-aws/ && rm src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java && /home/ubuntu/cephtest/apache-maven-3.6.2/bin/mvn -Dit.test=ITestS3A* -Dparallel-tests -Dscale -Dfs.s3a.scale.test.timeout=1200 -Dfs.s3a.scale.test.huge.filesize=256M verify'

pass 4399572 2019-10-10 14:25:20 2019-10-10 14:39:38 2019-10-10 15:01:37 0:21:59 0:11:02 0:10:57 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-stupid.yaml overrides.yaml rgw_pool_type/ec-profile.yaml tasks/rgw_ragweed.yaml} 2
fail 4399573 2019-10-10 14:25:21 2019-10-10 14:39:48 2019-10-10 17:31:50 2:52:02 2:39:54 0:12:08 smithi master centos rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} 2
Failure Reason:

rgw multisite, rgw multisite pubsub, test failures

fail 4399574 2019-10-10 14:25:22 2019-10-10 14:39:55 2019-10-10 14:57:54 0:17:59 0:11:17 0:06:42 smithi master centos rgw/tempest/{clusters/fixed-1.yaml distro/valgrind.yaml frontend/beast.yaml tasks/rgw_tempest.yaml} 1
Failure Reason:

Command failed on smithi018 with status 1: 'sudo yum -y install python34-devel'

fail 4399575 2019-10-10 14:25:23 2019-10-10 14:40:13 2019-10-10 15:12:13 0:32:00 0:19:58 0:12:02 smithi master centos rgw/website/{clusters/fixed-2.yaml frontend/civetweb.yaml http.yaml overrides.yaml tasks/s3tests-website.yaml} 2
Failure Reason:

Command failed (s3 tests against rgw) on smithi044 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'"

fail 4399576 2019-10-10 14:25:23 2019-10-10 14:40:24 2019-10-10 15:42:24 1:02:00 0:31:22 0:30:38 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/filestore-xfs.yaml overrides.yaml rgw_pool_type/ec.yaml tasks/rgw_readwrite.yaml} 2
Failure Reason:

"2019-10-10T15:34:28.965193+0000 mon.a (mon.0) 159 : cluster [WRN] Health check failed: 1 slow ops, oldest one blocked for 1570721666 sec, osd.2 has slow ops (SLOW_OPS)" in cluster log

pass 4399577 2019-10-10 14:25:24 2019-10-10 14:42:29 2019-10-10 15:10:28 0:27:59 0:11:45 0:16:14 smithi master ubuntu 18.04 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-low-osd-mem-target.yaml overrides.yaml rgw_pool_type/ec-profile.yaml supported-random-distro$/{ubuntu_latest.yaml}} 2
pass 4399578 2019-10-10 14:25:25 2019-10-10 14:42:31 2019-10-10 15:14:30 0:31:59 0:15:14 0:16:45 smithi master ubuntu rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/filestore-xfs.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_bucket_quota.yaml} 2
pass 4399579 2019-10-10 14:25:26 2019-10-10 14:42:33 2019-10-10 15:18:33 0:36:00 0:16:35 0:19:25 smithi master rgw/crypt/{0-cluster/fixed-1.yaml 1-ceph-install/install.yaml 2-kms/vault.yaml 3-rgw/rgw.yaml 4-tests/{s3tests.yaml}} 1
pass 4399580 2019-10-10 14:25:27 2019-10-10 14:43:10 2019-10-10 15:41:10 0:58:00 0:09:50 0:48:10 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-bitmap.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_roundtrip.yaml} 2
pass 4399581 2019-10-10 14:25:28 2019-10-10 14:43:27 2019-10-10 15:45:27 1:02:00 0:24:09 0:37:51 smithi master ubuntu rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-bitmap.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_multipart_upload.yaml} 2
pass 4399582 2019-10-10 14:25:28 2019-10-10 14:44:08 2019-10-10 15:12:07 0:27:59 0:14:23 0:13:36 smithi master centos 7.6 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/bluestore-stupid.yaml overrides.yaml rgw_pool_type/ec.yaml supported-random-distro$/{centos_7.yaml}} 2
pass 4399583 2019-10-10 14:25:29 2019-10-10 14:44:15 2019-10-10 15:14:15 0:30:00 0:16:06 0:13:54 smithi master centos 7.6 rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/v32.yaml s3a-hadoop.yaml supported-distros/centos_latest.yaml} 2
fail 4399584 2019-10-10 14:25:30 2019-10-10 14:45:49 2019-10-10 15:27:49 0:42:00 0:25:38 0:16:22 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-comp.yaml overrides.yaml rgw_pool_type/ec-profile.yaml tasks/rgw_s3tests.yaml} 2
Failure Reason:

Command failed (s3 tests against rgw) on smithi194 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'"

fail 4399585 2019-10-10 14:25:31 2019-10-10 14:48:07 2019-10-10 15:48:07 1:00:00 0:46:15 0:13:45 smithi master centos rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/two-zonegroup.yaml tasks/test_multi.yaml valgrind.yaml} 2
Failure Reason:

rgw multisite, rgw multisite pubsub, test failures

fail 4399586 2019-10-10 14:25:32 2019-10-10 14:48:14 2019-10-10 14:58:13 0:09:59 0:02:05 0:07:54 smithi master rhel 7.6 rgw/tempest/{clusters/fixed-1.yaml distro/supported-random-distro$/{rhel_7.yaml} frontend/civetweb.yaml tasks/rgw_tempest.yaml} 1
Failure Reason:

{'smithi107.front.sepia.ceph.com': {'attempts': 5, 'changed': True, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}}

pass 4399587 2019-10-10 14:25:33 2019-10-10 14:49:25 2019-10-10 15:25:24 0:35:59 0:24:41 0:11:18 smithi master rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-comp.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_readwrite.yaml} 2
pass 4399588 2019-10-10 14:25:34 2019-10-10 14:49:25 2019-10-10 15:15:24 0:25:59 0:12:25 0:13:34 smithi master rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-low-osd-mem-target.yaml overrides.yaml rgw_pool_type/ec.yaml tasks/rgw_swift.yaml} 2
fail 4399589 2019-10-10 14:25:35 2019-10-10 14:49:39 2019-10-10 15:03:38 0:13:59 0:02:10 0:11:49 smithi master rhel 7.6 rgw/singleton/{all/radosgw-admin.yaml frontend/civetweb.yaml objectstore/filestore-xfs.yaml overrides.yaml rgw_pool_type/replicated.yaml supported-random-distro$/{rhel_7.yaml}} 2
Failure Reason:

{'smithi154.front.sepia.ceph.com': {'attempts': 5, 'changed': True, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}, 'smithi042.front.sepia.ceph.com': {'attempts': 5, 'changed': True, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}}

pass 4399590 2019-10-10 14:25:35 2019-10-10 14:50:11 2019-10-10 15:26:10 0:35:59 0:16:27 0:19:32 smithi master ubuntu rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-stupid.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_user_quota.yaml} 2