Pulpito

yuriw-2019-10-25_01:34:23-rados-wip-yuri5-testing-2019-10-24-2334-luminous-distro-basic-smithi

User Scheduled Started Updated Runtime Suite Branch Machine Type Revision Pass Fail Dead
yuriw 2019-10-25 01:34:23 2019-10-25 01:36:23 2019-10-25 09:09:19 7:32:56 rados wip-yuri5-testing-2019-10-24-2334-luminous smithi 87d1abc 148 9 2

Average wait time: 0:13:49

See other runs of suite 'rados' on branch 'wip-yuri5-testing-2019-10-24-2334-luminous'?

See other runs on branch 'wip-yuri5-testing-2019-10-24-2334-luminous'?

See other runs scheduled on 2019-10-25?

See detail view?

Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Description Nodes
pass 4442164 2019-10-25 01:34:40 2019-10-25 01:35:14 2019-10-25 01:57:13 0:21:59 0:07:31 0:14:28 smithi master rados/singleton-nomsgr/{all/full-tiering.yaml rados.yaml} 1
pass 4442165 2019-10-25 01:34:41 2019-10-25 01:35:31 2019-10-25 01:57:30 0:21:59 0:07:42 0:14:17 smithi master rados/singleton/{all/mon-seesaw.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
pass 4442166 2019-10-25 01:34:43 2019-10-25 01:35:50 2019-10-25 02:09:50 0:34:00 0:20:02 0:13:58 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 4442167 2019-10-25 01:34:44 2019-10-25 01:36:23 2019-10-25 02:04:22 0:27:59 0:14:52 0:13:07 smithi master rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442168 2019-10-25 01:34:45 2019-10-25 01:37:05 2019-10-25 09:09:19 7:32:14 7:18:00 0:14:14 smithi master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
pass 4442169 2019-10-25 01:34:46 2019-10-25 01:40:16 2019-10-25 02:06:20 0:26:04 0:16:03 0:10:01 smithi master rados/standalone/misc.yaml 1
pass 4442170 2019-10-25 01:34:47 2019-10-25 01:40:17 2019-10-25 02:16:21 0:36:04 0:23:26 0:12:38 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
pass 4442171 2019-10-25 01:34:48 2019-10-25 01:40:17 2019-10-25 02:02:20 0:22:03 0:12:30 0:09:33 smithi master rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442172 2019-10-25 01:34:49 2019-10-25 01:42:20 2019-10-25 02:02:11 0:19:51 0:08:53 0:10:58 smithi master rados/singleton-nomsgr/{all/health-warnings.yaml rados.yaml} 1
pass 4442173 2019-10-25 01:34:50 2019-10-25 01:42:20 2019-10-25 02:14:11 0:31:51 0:21:10 0:10:41 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
pass 4442174 2019-10-25 01:34:51 2019-10-25 01:42:20 2019-10-25 02:06:11 0:23:51 0:11:13 0:12:38 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-stupid.yaml tasks/module_selftest.yaml} 2
pass 4442175 2019-10-25 01:34:52 2019-10-25 01:43:01 2019-10-25 03:21:01 1:38:00 0:11:00 1:27:00 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 4442176 2019-10-25 01:34:53 2019-10-25 01:44:35 2019-10-25 04:00:36 2:16:01 1:59:27 0:16:34 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 4442177 2019-10-25 01:34:54 2019-10-25 01:45:13 2019-10-25 02:13:17 0:28:04 0:16:57 0:11:07 smithi master rados/singleton/{all/osd-recovery.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442178 2019-10-25 01:34:55 2019-10-25 01:47:40 2019-10-25 02:15:44 0:28:04 0:14:46 0:13:18 smithi master rados/multimon/{clusters/9.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/mon_recovery.yaml} 3
pass 4442179 2019-10-25 01:34:56 2019-10-25 01:48:06 2019-10-25 02:14:06 0:26:00 0:13:17 0:12:43 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml d-thrash/none.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml} 2
fail 4442180 2019-10-25 01:34:57 2019-10-25 01:49:40 2019-10-25 02:05:39 0:15:59 0:02:44 0:13:15 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported/ubuntu_14.04.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

urn self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")Failure object was: {'smithi039.front.sepia.ceph.com': {'msg': u'All items completed', 'changed': False, 'results': [{'ansible_loop_var': u'item', '_ansible_no_log': False, u'err': u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n", 'item': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, 'changed': False, u'failed': True, '_ansible_item_label': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, u'rc': 3, u'invocation': {u'module_args': {u'vg_options': u'', u'pvs': [u'/dev/nvme0n1'], u'force': False, u'vg': u'vg_nvme', u'pesize': u'4', u'pv_options': u'', u'state': u'present'}}, u'msg': u'Failed executing pvs command.'}], 'warnings': [u"The value 4 (type int) in a string field was converted to u'4' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")

pass 4442181 2019-10-25 01:34:59 2019-10-25 01:49:43 2019-10-25 02:15:42 0:25:59 0:13:17 0:12:42 smithi master rados/thrash-luminous/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 4442182 2019-10-25 01:35:00 2019-10-25 01:50:08 2019-10-25 02:08:07 0:17:59 0:07:34 0:10:25 smithi master rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
pass 4442183 2019-10-25 01:35:01 2019-10-25 01:50:31 2019-10-25 02:40:31 0:50:00 0:23:04 0:26:56 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 4442184 2019-10-25 01:35:02 2019-10-25 01:51:48 2019-10-25 02:09:47 0:17:59 0:08:13 0:09:46 smithi master rados/singleton-nomsgr/{all/large-omap-object-warnings.yaml rados.yaml} 1
pass 4442185 2019-10-25 01:35:03 2019-10-25 01:51:53 2019-10-25 02:35:53 0:44:00 0:22:11 0:21:49 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
pass 4442186 2019-10-25 01:35:04 2019-10-25 01:52:11 2019-10-25 02:12:10 0:19:59 0:07:26 0:12:33 smithi master rados/singleton/{all/pg-removal-interruption.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442187 2019-10-25 01:35:05 2019-10-25 01:54:26 2019-10-25 02:34:26 0:40:00 0:21:21 0:18:39 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/one.yaml workloads/rados_mon_workunits.yaml} 2
pass 4442188 2019-10-25 01:35:06 2019-10-25 01:57:32 2019-10-25 02:49:31 0:51:59 0:28:44 0:23:15 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
pass 4442189 2019-10-25 01:35:07 2019-10-25 01:57:32 2019-10-25 09:05:39 7:08:07 6:54:16 0:13:51 smithi master rados/objectstore/filestore-idempotent.yaml 1
pass 4442190 2019-10-25 01:35:08 2019-10-25 01:57:32 2019-10-25 02:47:32 0:50:00 0:22:45 0:27:15 smithi master rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} 2
pass 4442191 2019-10-25 01:35:09 2019-10-25 01:57:38 2019-10-25 02:29:38 0:32:00 0:20:17 0:11:43 smithi master rados/singleton/{all/radostool.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442192 2019-10-25 01:35:10 2019-10-25 02:01:21 2019-10-25 02:43:20 0:41:59 0:27:14 0:14:45 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 4442193 2019-10-25 01:35:11 2019-10-25 02:02:41 2019-10-25 02:42:45 0:40:04 0:20:18 0:19:46 smithi master rados/singleton/{all/random-eio.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
pass 4442194 2019-10-25 01:35:12 2019-10-25 02:02:41 2019-10-25 02:48:45 0:46:04 0:24:08 0:21:56 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml fast/normal.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml} 2
pass 4442195 2019-10-25 01:35:13 2019-10-25 02:04:09 2019-10-25 02:26:09 0:22:00 0:10:21 0:11:39 smithi master rados/basic-luminous/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} objectstore/bluestore-stupid.yaml rados.yaml scrub_test.yaml} 2
fail 4442196 2019-10-25 01:35:14 2019-10-25 02:04:09 2019-10-25 02:24:09 0:20:00 0:08:14 0:11:46 smithi master rados/rest/rest_test.yaml 2
Failure Reason:

Command failed (workunit test rest/test.py) on smithi156 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=87d1abc0d62fad2694bd21c85f27cfb4a078904a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rest/test.py'

pass 4442197 2019-10-25 01:35:15 2019-10-25 02:04:09 2019-10-25 02:38:09 0:34:00 0:23:19 0:10:41 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442198 2019-10-25 01:35:16 2019-10-25 02:04:46 2019-10-25 02:22:45 0:17:59 0:09:12 0:08:47 smithi master rados/singleton-nomsgr/{all/lazy_omap_stats_output.yaml rados.yaml} 1
pass 4442199 2019-10-25 01:35:17 2019-10-25 02:05:38 2019-10-25 02:27:37 0:21:59 0:11:14 0:10:45 smithi master rados/standalone/mon.yaml 1
pass 4442200 2019-10-25 01:35:19 2019-10-25 02:05:38 2019-10-25 05:33:41 3:28:03 3:16:08 0:11:55 smithi master ubuntu 16.04 rados/upgrade/jewel-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-luminous.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/ubuntu_latest.yaml thrashosds-health.yaml} 3
pass 4442201 2019-10-25 01:35:20 2019-10-25 02:05:44 2019-10-25 02:31:44 0:26:00 0:13:10 0:12:50 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/readwrite.yaml} 2
pass 4442202 2019-10-25 01:35:21 2019-10-25 02:06:13 2019-10-25 02:28:12 0:21:59 0:11:42 0:10:17 smithi master rados/singleton/{all/rebuild-mondb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
pass 4442203 2019-10-25 01:35:22 2019-10-25 02:06:35 2019-10-25 02:34:37 0:28:02 0:15:39 0:12:23 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 4442204 2019-10-25 01:35:23 2019-10-25 02:07:39 2019-10-25 02:25:38 0:17:59 0:06:29 0:11:30 smithi master rados/objectstore/fusestore.yaml 1
pass 4442205 2019-10-25 01:35:24 2019-10-25 02:07:52 2019-10-25 02:34:01 0:26:09 0:14:16 0:11:53 smithi master rados/singleton/{all/recovery-preemption.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442206 2019-10-25 01:35:25 2019-10-25 02:08:05 2019-10-25 03:02:07 0:54:02 0:28:55 0:25:07 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported/ubuntu_latest.yaml thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 4442207 2019-10-25 01:35:26 2019-10-25 02:08:09 2019-10-25 02:36:08 0:27:59 0:12:29 0:15:30 smithi master rados/thrash-luminous/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
fail 4442208 2019-10-25 01:35:27 2019-10-25 02:10:08 2019-10-25 05:30:11 3:20:03 3:08:23 0:11:40 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/filestore-xfs.yaml tasks/workunits.yaml} 2
Failure Reason:

Command failed (workunit test mgr/test_localpool.sh) on smithi060 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=87d1abc0d62fad2694bd21c85f27cfb4a078904a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mgr/test_localpool.sh'

pass 4442209 2019-10-25 01:35:28 2019-10-25 02:10:08 2019-10-25 02:48:08 0:38:00 0:10:15 0:27:45 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 4442210 2019-10-25 01:35:29 2019-10-25 02:10:11 2019-10-25 02:28:29 0:18:18 0:07:47 0:10:31 smithi master rados/singleton-nomsgr/{all/librados_hello_world.yaml rados.yaml} 1
pass 4442211 2019-10-25 01:35:30 2019-10-25 02:11:24 2019-10-25 02:33:23 0:21:59 0:08:17 0:13:42 smithi master rados/multimon/{clusters/21.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 3
pass 4442212 2019-10-25 01:35:31 2019-10-25 02:12:35 2019-10-25 02:32:34 0:19:59 0:10:13 0:09:46 smithi master rados/singleton/{all/reg11184.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442213 2019-10-25 01:35:32 2019-10-25 02:12:35 2019-10-25 02:52:35 0:40:00 0:25:11 0:14:49 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml} 2
pass 4442214 2019-10-25 01:35:33 2019-10-25 02:13:36 2019-10-25 02:59:36 0:46:00 0:12:05 0:33:55 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-end.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
pass 4442215 2019-10-25 01:35:34 2019-10-25 02:13:51 2019-10-25 02:53:50 0:39:59 0:28:28 0:11:31 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml thrashers/sync-many.yaml workloads/snaps-few-objects.yaml} 2
pass 4442216 2019-10-25 01:35:35 2019-10-25 02:13:54 2019-10-25 02:41:53 0:27:59 0:15:50 0:12:09 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/repair_test.yaml} 2
pass 4442217 2019-10-25 01:35:36 2019-10-25 02:13:58 2019-10-25 02:43:57 0:29:59 0:14:58 0:15:01 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 4442218 2019-10-25 01:35:37 2019-10-25 02:14:07 2019-10-25 02:40:07 0:26:00 0:08:01 0:17:59 smithi master rados/singleton/{all/resolve_stuck_peering.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
pass 4442219 2019-10-25 01:35:38 2019-10-25 02:14:13 2019-10-25 03:06:13 0:52:00 0:41:33 0:10:27 smithi master rados/singleton-nomsgr/{all/msgr.yaml rados.yaml} 1
fail 4442220 2019-10-25 01:35:39 2019-10-25 02:14:14 2019-10-25 02:32:42 0:18:28 0:08:06 0:10:22 smithi master rados/singleton/{all/rest-api.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
Failure Reason:

Command failed (workunit test rest/test.py) on smithi141 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=87d1abc0d62fad2694bd21c85f27cfb4a078904a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rest/test.py'

pass 4442221 2019-10-25 01:35:40 2019-10-25 02:16:02 2019-10-25 03:00:02 0:44:00 0:30:16 0:13:44 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
pass 4442222 2019-10-25 01:35:42 2019-10-25 02:16:02 2019-10-25 02:58:02 0:42:00 0:27:13 0:14:47 smithi master rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml} 2
pass 4442223 2019-10-25 01:35:43 2019-10-25 02:16:43 2019-10-25 02:34:42 0:17:59 0:06:48 0:11:11 smithi master rados/objectstore/keyvaluedb.yaml 1
fail 4442224 2019-10-25 01:35:44 2019-10-25 02:16:43 2019-10-25 06:36:47 4:20:04 4:08:39 0:11:25 smithi master rados/standalone/osd.yaml 1
Failure Reason:

Command failed (workunit test osd/osd-rep-recov-eio.sh) on smithi012 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=87d1abc0d62fad2694bd21c85f27cfb4a078904a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-rep-recov-eio.sh'

pass 4442225 2019-10-25 01:35:45 2019-10-25 02:17:41 2019-10-25 02:51:41 0:34:00 0:15:24 0:18:36 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml} 2
pass 4442226 2019-10-25 01:35:46 2019-10-25 02:18:18 2019-10-25 02:40:18 0:22:00 0:12:09 0:09:51 smithi master rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442227 2019-10-25 01:35:47 2019-10-25 02:20:02 2019-10-25 02:56:01 0:35:59 0:17:56 0:18:03 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rgw_snaps.yaml} 2
fail 4442228 2019-10-25 01:35:48 2019-10-25 02:20:02 2019-10-25 03:00:01 0:39:59 0:25:10 0:14:49 smithi master rados/singleton/{all/thrash-eio.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 2
Failure Reason:

"2019-10-25 02:39:37.903076 mon.a mon.0 172.21.15.199:6789/0 291 : cluster [WRN] Health check failed: 3/2875 objects unfound (0.104%) (OBJECT_UNFOUND)" in cluster log

pass 4442229 2019-10-25 01:35:49 2019-10-25 02:20:13 2019-10-25 02:58:12 0:37:59 0:22:41 0:15:18 smithi master rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml} 2
dead 4442230 2019-10-25 01:35:50 2019-10-25 02:23:03 2019-10-25 02:39:03 0:16:00 smithi master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} supported/centos_latest.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} —
Failure Reason:

reached maximum tries (60) after waiting for 900 seconds

pass 4442231 2019-10-25 01:35:51 2019-10-25 02:23:34 2019-10-25 02:47:33 0:23:59 0:12:11 0:11:48 smithi master rados/thrash-luminous/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 4442232 2019-10-25 01:35:52 2019-10-25 02:24:27 2019-10-25 02:52:26 0:27:59 0:16:52 0:11:07 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml} 2
pass 4442233 2019-10-25 01:35:53 2019-10-25 02:25:57 2019-10-25 02:59:56 0:33:59 0:19:30 0:14:29 smithi master rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 2
pass 4442234 2019-10-25 01:35:54 2019-10-25 02:25:57 2019-10-25 02:49:56 0:23:59 0:09:57 0:14:02 smithi master rados/basic-luminous/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} objectstore/filestore-xfs.yaml rados.yaml scrub_test.yaml} 2
pass 4442235 2019-10-25 01:35:55 2019-10-25 02:26:07 2019-10-25 02:58:07 0:32:00 0:07:37 0:24:23 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/filestore-xfs.yaml tasks/dashboard.yaml} 2
pass 4442236 2019-10-25 01:35:56 2019-10-25 02:26:10 2019-10-25 03:00:10 0:34:00 0:24:46 0:09:14 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442237 2019-10-25 01:35:57 2019-10-25 02:27:22 2019-10-25 02:55:22 0:28:00 0:11:17 0:16:43 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 4442238 2019-10-25 01:35:58 2019-10-25 02:27:39 2019-10-25 03:13:38 0:45:59 0:28:35 0:17:24 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 4442239 2019-10-25 01:35:59 2019-10-25 02:28:02 2019-10-25 03:04:02 0:36:00 0:16:44 0:19:16 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/sync-many.yaml workloads/pool-create-delete.yaml} 2
pass 4442240 2019-10-25 01:36:01 2019-10-25 02:28:13 2019-10-25 03:08:13 0:40:00 0:22:14 0:17:46 smithi master rados/singleton/{all/thrash_cache_writeback_proxy_none.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml} 2
pass 4442241 2019-10-25 01:36:02 2019-10-25 02:28:48 2019-10-25 03:04:48 0:36:00 0:20:44 0:15:16 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_api_tests.yaml} 2
pass 4442242 2019-10-25 01:36:03 2019-10-25 02:29:39 2019-10-25 03:07:42 0:38:03 0:27:37 0:10:26 smithi master rados/objectstore/objectcacher-stress.yaml 1
pass 4442243 2019-10-25 01:36:04 2019-10-25 02:32:04 2019-10-25 02:50:03 0:17:59 0:07:31 0:10:28 smithi master rados/singleton-nomsgr/{all/pool-access.yaml rados.yaml} 1
pass 4442244 2019-10-25 01:36:05 2019-10-25 02:32:56 2019-10-25 02:58:53 0:25:57 0:14:42 0:11:15 smithi master rados/multimon/{clusters/3.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/mon_clock_with_skews.yaml} 2
pass 4442245 2019-10-25 01:36:06 2019-10-25 02:32:57 2019-10-25 03:02:57 0:30:00 0:18:19 0:11:41 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml d-thrash/none.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
pass 4442246 2019-10-25 01:36:07 2019-10-25 02:33:42 2019-10-25 03:05:42 0:32:00 0:20:33 0:11:27 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 4442247 2019-10-25 01:36:08 2019-10-25 02:33:42 2019-10-25 02:49:42 0:16:00 0:06:54 0:09:06 smithi master rados/singleton/{all/watch-notify-same-primary.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442248 2019-10-25 01:36:09 2019-10-25 02:34:02 2019-10-25 03:08:01 0:33:59 0:23:15 0:10:44 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml fast/normal.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 4442249 2019-10-25 01:36:10 2019-10-25 02:34:27 2019-10-25 02:50:26 0:15:59 0:07:38 0:08:21 smithi master rados/singleton/{all/admin-socket.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442250 2019-10-25 01:36:11 2019-10-25 02:34:39 2019-10-25 03:28:39 0:54:00 0:44:50 0:09:10 smithi master rados/standalone/scrub.yaml 1
pass 4442251 2019-10-25 01:36:12 2019-10-25 02:35:06 2019-10-25 03:13:05 0:37:59 0:20:47 0:17:12 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml} 2
pass 4442252 2019-10-25 01:36:13 2019-10-25 02:35:42 2019-10-25 02:55:30 0:19:48 0:08:56 0:10:52 smithi master rados/singleton/{all/divergent_priors.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442253 2019-10-25 01:36:14 2019-10-25 02:35:54 2019-10-25 03:09:54 0:34:00 0:23:10 0:10:50 smithi master rados/singleton-nomsgr/{all/recovery-unfound-found.yaml rados.yaml} 1
pass 4442254 2019-10-25 01:36:15 2019-10-25 02:35:58 2019-10-25 03:03:57 0:27:59 0:11:46 0:16:13 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_cls_all.yaml} 2
pass 4442255 2019-10-25 01:36:16 2019-10-25 02:36:28 2019-10-25 03:20:27 0:43:59 0:28:44 0:15:15 smithi master rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml fast/fast.yaml leveldb.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml} 2
pass 4442256 2019-10-25 01:36:17 2019-10-25 02:37:05 2019-10-25 02:59:05 0:22:00 0:10:07 0:11:53 smithi master rados/singleton/{all/divergent_priors2.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442257 2019-10-25 01:36:19 2019-10-25 02:37:57 2019-10-25 08:14:02 5:36:05 5:21:13 0:14:52 smithi master rados/objectstore/objectstore.yaml 1
fail 4442258 2019-10-25 01:36:20 2019-10-25 02:38:11 2019-10-25 02:54:19 0:16:08 0:02:44 0:13:24 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported/ubuntu_14.04.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

urn self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")Failure object was: {'smithi025.front.sepia.ceph.com': {'msg': u'All items completed', 'changed': False, 'results': [{'ansible_loop_var': u'item', '_ansible_no_log': False, u'err': u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n", 'item': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, 'changed': False, u'failed': True, '_ansible_item_label': {'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}, u'rc': 3, u'invocation': {u'module_args': {u'vg_options': u'', u'pvs': [u'/dev/nvme0n1'], u'force': False, u'vg': u'vg_nvme', u'pesize': u'4', u'pv_options': u'', u'state': u'present'}}, u'msg': u'Failed executing pvs command.'}], 'warnings': [u"The value 4 (type int) in a string field was converted to u'4' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}}Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 309, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 219, in represent_list return self.represent_sequence(u'tag:yaml.org,2002:seq', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 102, in represent_sequence node_item = self.represent_data(item) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 227, in represent_dict return self.represent_mapping(u'tag:yaml.org,2002:map', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 125, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined raise RepresenterError("cannot represent an object", data)RepresenterError: ('cannot represent an object', u"/sbin/pvs: unrecognized option '--select'\n Error during parsing of command line.\n")

dead 4442259 2019-10-25 01:36:21 2019-10-25 02:38:48 2019-10-25 02:56:47 0:17:59 smithi master rados/thrash-luminous/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} —
Failure Reason:

reached maximum tries (60) after waiting for 900 seconds

pass 4442260 2019-10-25 01:36:22 2019-10-25 02:39:21 2019-10-25 03:15:21 0:36:00 0:23:23 0:12:37 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-snaps.yaml} 2
pass 4442261 2019-10-25 01:36:23 2019-10-25 02:40:20 2019-10-25 02:58:20 0:18:00 0:08:52 0:09:08 smithi master rados/singleton/{all/dump-stuck.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
fail 4442262 2019-10-25 01:36:24 2019-10-25 02:40:20 2019-10-25 03:02:20 0:22:00 0:10:37 0:11:23 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml thrashers/sync.yaml workloads/rados_5925.yaml} 2
Failure Reason:

"2019-10-25 02:56:22.325247 mon.b mon.0 172.21.15.114:6789/0 157 : cluster [WRN] Health check failed: 1 slow requests are blocked > 32 sec. Implicated osds 5 (REQUEST_SLOW)" in cluster log

pass 4442263 2019-10-25 01:36:25 2019-10-25 02:40:48 2019-10-25 03:04:48 0:24:00 0:10:05 0:13:55 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
pass 4442264 2019-10-25 01:36:26 2019-10-25 02:42:11 2019-10-25 03:04:11 0:22:00 0:14:22 0:07:38 smithi master centos rados/singleton-nomsgr/{all/valgrind-leaks.yaml rados.yaml} 1
pass 4442265 2019-10-25 01:36:27 2019-10-25 02:42:47 2019-10-25 03:04:46 0:21:59 0:09:45 0:12:14 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-bitmap.yaml tasks/failover.yaml} 2
pass 4442266 2019-10-25 01:36:28 2019-10-25 02:43:40 2019-10-25 03:17:40 0:34:00 0:12:29 0:21:31 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} d-require-luminous/at-end.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 4442267 2019-10-25 01:36:29 2019-10-25 02:43:40 2019-10-25 03:09:40 0:26:00 0:12:33 0:13:27 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/rados_python.yaml} 2
pass 4442268 2019-10-25 01:36:30 2019-10-25 02:43:40 2019-10-25 03:25:40 0:42:00 0:31:32 0:10:28 smithi master rados/singleton/{all/ec-lost-unfound.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442269 2019-10-25 01:36:31 2019-10-25 02:43:59 2019-10-25 03:07:58 0:23:59 0:12:12 0:11:47 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/cache.yaml} 2
pass 4442270 2019-10-25 01:36:32 2019-10-25 02:47:49 2019-10-25 03:03:48 0:15:59 0:06:14 0:09:45 smithi master rados/singleton/{all/erasure-code-nonregression.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442271 2019-10-25 01:36:33 2019-10-25 02:47:49 2019-10-25 03:07:49 0:20:00 0:09:33 0:10:27 smithi master rados/multimon/{clusters/3.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/mon_recovery.yaml} 2
pass 4442272 2019-10-25 01:36:34 2019-10-25 02:47:59 2019-10-25 03:23:58 0:35:59 0:24:51 0:11:08 smithi master centos rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml d-thrash/default/{default.yaml thrashosds-health.yaml} mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
pass 4442273 2019-10-25 01:36:35 2019-10-25 02:48:09 2019-10-25 05:46:11 2:58:02 2:47:51 0:10:11 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml fast/fast.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} 2
pass 4442274 2019-10-25 01:36:36 2019-10-25 02:49:07 2019-10-25 03:10:46 0:21:39 0:10:36 0:11:03 smithi master rados/basic-luminous/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml rados.yaml scrub_test.yaml} 2
pass 4442275 2019-10-25 01:36:37 2019-10-25 02:49:49 2019-10-25 03:05:48 0:15:59 0:07:16 0:08:43 smithi master rados/objectstore/alloc-hint.yaml 1
pass 4442276 2019-10-25 01:36:38 2019-10-25 02:49:49 2019-10-25 03:07:49 0:18:00 0:08:11 0:09:49 smithi master rados/rest/mgr-restful.yaml 1
pass 4442277 2019-10-25 01:36:39 2019-10-25 02:49:55 2019-10-25 03:23:54 0:33:59 0:23:05 0:10:54 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442278 2019-10-25 01:36:40 2019-10-25 02:49:57 2019-10-25 03:09:57 0:20:00 0:10:05 0:09:55 smithi master rados/singleton-nomsgr/{all/admin_socket_output.yaml rados.yaml} 1
pass 4442279 2019-10-25 01:36:41 2019-10-25 02:50:05 2019-10-25 03:08:04 0:17:59 0:07:55 0:10:04 smithi master rados/standalone/crush.yaml 1
pass 4442280 2019-10-25 01:36:42 2019-10-25 02:50:30 2019-10-25 06:12:32 3:22:02 3:05:17 0:16:45 smithi master centos 7.4 rados/upgrade/jewel-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-jewel-install/jewel.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-luminous.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} distros/centos_latest.yaml thrashosds-health.yaml} 3
pass 4442281 2019-10-25 01:36:43 2019-10-25 02:51:59 2019-10-25 03:41:59 0:50:00 0:39:47 0:10:13 smithi master rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442282 2019-10-25 01:36:44 2019-10-25 02:52:01 2019-10-25 03:18:00 0:25:59 0:14:43 0:11:16 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_stress_watch.yaml} 2
pass 4442283 2019-10-25 01:36:45 2019-10-25 02:52:28 2019-10-25 03:32:27 0:39:59 0:26:43 0:13:16 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 4442284 2019-10-25 01:36:46 2019-10-25 02:52:36 2019-10-25 03:44:36 0:52:00 0:39:39 0:12:21 smithi master rados/singleton/{all/lost-unfound.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
pass 4442285 2019-10-25 01:36:48 2019-10-25 02:53:42 2019-10-25 03:27:41 0:33:59 0:23:00 0:10:59 smithi master ubuntu 16.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml leveldb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported/ubuntu_latest.yaml thrashers/none.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 4442286 2019-10-25 01:36:49 2019-10-25 02:53:52 2019-10-25 03:17:51 0:23:59 0:10:44 0:13:15 smithi master rados/thrash-luminous/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/redirect.yaml} 2
pass 4442287 2019-10-25 01:36:50 2019-10-25 02:54:21 2019-10-25 03:32:20 0:37:59 0:26:29 0:11:30 smithi master rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml} 2
pass 4442288 2019-10-25 01:36:51 2019-10-25 02:55:40 2019-10-25 03:13:39 0:17:59 0:07:57 0:10:02 smithi master rados/singleton/{all/max-pg-per-osd.from-mon.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442289 2019-10-25 01:36:52 2019-10-25 02:55:40 2019-10-25 03:13:39 0:17:59 0:07:32 0:10:27 smithi master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml rados.yaml} 1
pass 4442290 2019-10-25 01:36:53 2019-10-25 02:55:43 2019-10-25 03:27:42 0:31:59 0:20:22 0:11:37 smithi master rados/monthrash/{ceph.yaml clusters/9-mons.yaml d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml thrashers/force-sync-many.yaml workloads/rados_api_tests.yaml} 2
pass 4442291 2019-10-25 01:36:54 2019-10-25 02:56:02 2019-10-25 03:30:02 0:34:00 0:21:45 0:12:15 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-upmap.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml rocksdb.yaml thrashers/default.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml} 2
pass 4442292 2019-10-25 01:36:55 2019-10-25 02:57:06 2019-10-25 03:33:05 0:35:59 0:23:32 0:12:27 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml} 3
pass 4442293 2019-10-25 01:36:56 2019-10-25 02:58:03 2019-10-25 03:18:03 0:20:00 0:10:55 0:09:05 smithi master rados/singleton/{all/max-pg-per-osd.from-primary.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442294 2019-10-25 01:36:57 2019-10-25 02:58:25 2019-10-25 03:18:24 0:19:59 0:08:02 0:11:57 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_striper.yaml} 2
pass 4442295 2019-10-25 01:36:58 2019-10-25 02:58:25 2019-10-25 03:26:25 0:28:00 0:13:09 0:14:51 smithi master rados/objectstore/ceph_objectstore_tool.yaml 1
pass 4442296 2019-10-25 01:36:59 2019-10-25 02:58:25 2019-10-25 03:22:24 0:23:59 0:10:23 0:13:36 smithi master rados/mgr/{clusters/2-node-mgr.yaml debug/mgr.yaml objectstore/bluestore-comp.yaml tasks/module_selftest.yaml} 2
pass 4442297 2019-10-25 01:37:00 2019-10-25 02:58:25 2019-10-25 03:34:25 0:36:00 0:11:43 0:24:17 smithi master rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml leveldb.yaml msgr-failures/fastclose.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} 4
pass 4442298 2019-10-25 01:37:01 2019-10-25 02:58:55 2019-10-25 03:22:54 0:23:59 0:13:29 0:10:30 smithi master rados/singleton/{all/max-pg-per-osd.from-replica.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442299 2019-10-25 01:37:02 2019-10-25 02:59:06 2019-10-25 03:17:05 0:17:59 0:06:14 0:11:45 smithi master rados/singleton-nomsgr/{all/ceph-post-file.yaml rados.yaml} 1
pass 4442300 2019-10-25 01:37:03 2019-10-25 02:59:55 2019-10-25 03:51:55 0:52:00 0:39:38 0:12:22 smithi master rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml msgr-failures/osd-delay.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml} 2
pass 4442301 2019-10-25 01:37:04 2019-10-25 02:59:57 2019-10-25 03:35:57 0:36:00 0:23:16 0:12:44 smithi master rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml fast/normal.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml} 2
pass 4442302 2019-10-25 01:37:05 2019-10-25 03:00:03 2019-10-25 03:20:02 0:19:59 0:07:48 0:12:11 smithi master rados/singleton/{all/mon-auth-caps.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1
fail 4442303 2019-10-25 01:37:06 2019-10-25 03:00:03 2019-10-25 06:24:06 3:24:03 3:11:02 0:13:01 smithi master rados/standalone/erasure-code.yaml 1
Failure Reason:

Command failed (workunit test erasure-code/test-erasure-eio.sh) on smithi086 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=87d1abc0d62fad2694bd21c85f27cfb4a078904a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh'

pass 4442304 2019-10-25 01:37:07 2019-10-25 03:00:11 2019-10-25 03:22:10 0:21:59 0:07:42 0:14:17 smithi master rados/multimon/{clusters/6.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/mon_clock_no_skews.yaml} 2
pass 4442305 2019-10-25 01:37:08 2019-10-25 03:02:25 2019-10-25 03:24:25 0:22:00 0:11:24 0:10:36 smithi master rados/singleton/{all/mon-config-key-caps.yaml msgr-failures/many.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml} 1
pass 4442306 2019-10-25 01:37:09 2019-10-25 03:02:25 2019-10-25 03:24:25 0:22:00 0:11:16 0:10:44 smithi master rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml d-thrash/none.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
pass 4442307 2019-10-25 01:37:10 2019-10-25 03:02:58 2019-10-25 03:44:58 0:42:00 0:31:36 0:10:24 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_workunit_loadgen_big.yaml} 2
pass 4442308 2019-10-25 01:37:11 2019-10-25 03:04:08 2019-10-25 03:40:07 0:35:59 0:25:18 0:10:41 smithi master rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-end.yaml msgr-failures/fastclose.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml} 2
pass 4442309 2019-10-25 01:37:12 2019-10-25 03:04:08 2019-10-25 03:22:07 0:17:59 0:06:59 0:11:00 smithi master rados/singleton-nomsgr/{all/export-after-evict.yaml rados.yaml} 1
pass 4442310 2019-10-25 01:37:13 2019-10-25 03:04:08 2019-10-25 03:28:07 0:23:59 0:12:31 0:11:28 smithi master rados/singleton/{all/mon-config-keys.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-bitmap.yaml rados.yaml} 1
pass 4442311 2019-10-25 01:37:14 2019-10-25 03:04:12 2019-10-25 03:50:12 0:46:00 0:30:01 0:15:59 smithi master centos 7.4 rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported/centos_latest.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 4442312 2019-10-25 01:37:15 2019-10-25 03:04:47 2019-10-25 03:28:46 0:23:59 0:12:59 0:11:00 smithi master rados/thrash-luminous/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml rocksdb.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml} 2
pass 4442313 2019-10-25 01:37:16 2019-10-25 03:04:49 2019-10-25 03:22:48 0:17:59 0:07:10 0:10:49 smithi master rados/objectstore/filejournal.yaml 1
pass 4442314 2019-10-25 01:37:17 2019-10-25 03:04:49 2019-10-25 03:32:48 0:27:59 0:16:44 0:11:15 smithi master rados/monthrash/{ceph.yaml clusters/3-mons.yaml d-require-luminous/at-end.yaml mon_kv_backend/leveldb.yaml msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-stupid.yaml rados.yaml thrashers/many.yaml workloads/rados_mon_workunits.yaml} 2
pass 4442315 2019-10-25 01:37:18 2019-10-25 03:06:01 2019-10-25 03:22:00 0:15:59 0:07:28 0:08:31 smithi master rados/singleton/{all/mon-seesaw.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442316 2019-10-25 01:37:19 2019-10-25 03:06:02 2019-10-25 03:28:01 0:21:59 0:10:30 0:11:29 smithi master rados/basic-luminous/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} objectstore/bluestore-comp.yaml rados.yaml scrub_test.yaml} 2
pass 4442317 2019-10-25 01:37:20 2019-10-25 03:06:14 2019-10-25 03:40:13 0:33:59 0:23:07 0:10:52 smithi master rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml} 1
pass 4442318 2019-10-25 01:37:21 2019-10-25 03:08:00 2019-10-25 03:44:00 0:36:00 0:25:48 0:10:12 smithi master rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml fast/fast.yaml leveldb.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml} 2
pass 4442319 2019-10-25 01:37:22 2019-10-25 03:08:00 2019-10-25 03:58:00 0:50:00 0:12:01 0:37:59 smithi master rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} d-require-luminous/at-end.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 3
pass 4442320 2019-10-25 01:37:23 2019-10-25 03:08:00 2019-10-25 03:40:00 0:32:00 0:21:29 0:10:31 smithi master rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs-balancer-crush-compat.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} 2
pass 4442321 2019-10-25 01:37:24 2019-10-25 03:08:00 2019-10-25 03:40:00 0:32:00 0:19:49 0:12:11 smithi master rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-require-luminous/at-mkfs.yaml mon_kv_backend/rocksdb.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
pass 4442322 2019-10-25 01:37:25 2019-10-25 03:08:03 2019-10-25 03:42:02 0:33:59 0:24:38 0:09:21 smithi master rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/random.yaml objectstore/bluestore-stupid.yaml rados.yaml} 1