sage-2017-03-18_23:56:28-rados-wip-bluestore-dw---basic-smithi

SSH connection to smithi082 was lost: 'sudo lsb_release -is'

ceph version 12.0.0-1529-gb2879f6-1xenial was not installed, found None.

Command failed on smithi171 with status 1: "dpkg-query -W -f '${Version}' ceph"

Socket is closed

{'smithi171.front.sepia.ceph.com': {'_ansible_parsed': False, 'exception': 'Traceback (most recent call last):\n File "/tmp/ansible_oBqMt1/ansible_module_apt.py", line 903, in <module>\n main()\n File "/tmp/ansible_oBqMt1/ansible_module_apt.py", line 799, in main\n cache = apt.Cache()\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 107, in __init__\n self.open(progress)\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 153, in open\n self._records = apt_pkg.PackageRecords(self._cache)\nSystemError: E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages - open (2: No such file or directory)\n', '_ansible_no_log': False, 'module_stderr': 'Traceback (most recent call last):\n File "/tmp/ansible_oBqMt1/ansible_module_apt.py", line 903, in <module>\n main()\n File "/tmp/ansible_oBqMt1/ansible_module_apt.py", line 799, in main\n cache = apt.Cache()\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 107, in __init__\n self.open(progress)\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 153, in open\n self._records = apt_pkg.PackageRecords(self._cache)\nSystemError: E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_binary-i386_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_multiverse_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_universe_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_restricted_binary-amd64_Packages - open (2: No such file or directory), E:Could not open file /var/lib/apt/lists/apt-mirror.front.sepia.ceph.com_archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages - open (2: No such file or directory)\n', 'changed': False, 'module_stdout': '', 'failed': True, 'invocation': {'module_name': 'apt'}, 'msg': 'MODULE FAILURE'}}

{'smithi082.front.sepia.ceph.com': {'_ansible_parsed': True, 'cmd': ['apt-get', 'clean'], 'end': '2017-03-19 00:28:09.655166', '_ansible_no_log': False, 'stdout': '', 'changed': True, 'start': '2017-03-19 00:28:09.594994', 'delta': '0:00:00.060172', 'stderr': 'E: Could not get lock /var/cache/apt/archives/lock - open (11: Resource temporarily unavailable)\nE: Unable to lock directory /var/cache/apt/archives/', 'rc': 100, 'invocation': {'module_name': 'command', 'module_args': {'warn': True, 'creates': None, 'executable': None, 'chdir': None, '_uses_shell': False, '_raw_params': 'apt-get clean', 'removes': None}}, 'stdout_lines': [], 'warnings': ['Consider using apt module rather than running apt-get'], 'failed': True}}

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op copy_from 50 --op write_excl 50 --op delete 50 --pool base'

Command failed on smithi082 with status 1: 'sudo python -c \'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))\' /home/ubuntu/cephtest/valgrind.supp'

Command failed on smithi082 with status 127: 'sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap'

Command failed (workunit test rados/test.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b2879f665bd3c9876e817e9588630236e59494c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

timed out waiting for admin_socket to appear after osd.2 restart

Command failed on smithi129 with status 1: "sudo yum -y install '' ceph-radosgw"

SSH connection to smithi082 was lost: u'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=12.0.0-1529-gb2879f6-1xenial ceph-mds=12.0.0-1529-gb2879f6-1xenial ceph-mgr=12.0.0-1529-gb2879f6-1xenial ceph-common=12.0.0-1529-gb2879f6-1xenial ceph-fuse=12.0.0-1529-gb2879f6-1xenial ceph-test=12.0.0-1529-gb2879f6-1xenial radosgw=12.0.0-1529-gb2879f6-1xenial python-ceph=12.0.0-1529-gb2879f6-1xenial libcephfs2=12.0.0-1529-gb2879f6-1xenial libcephfs-dev=12.0.0-1529-gb2879f6-1xenial libcephfs-java=12.0.0-1529-gb2879f6-1xenial libcephfs-jni=12.0.0-1529-gb2879f6-1xenial librados2=12.0.0-1529-gb2879f6-1xenial librbd1=12.0.0-1529-gb2879f6-1xenial rbd-fuse=12.0.0-1529-gb2879f6-1xenial'

Command failed on smithi019 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0'

Command failed on smithi082 with status 127: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

SSH connection to smithi171 was lost: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0'

failed to become clean before timeout expired

Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi201 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b2879f665bd3c9876e817e9588630236e59494c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

Command failed (workunit test rados/test.sh) on smithi179 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b2879f665bd3c9876e817e9588630236e59494c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

Command failed on smithi045 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1'

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool base'

{'smithi082.front.sepia.ceph.com': {'msg': 'One or more items failed', 'failed': True, 'changed': False}}

Command failed (workunit test rados/test.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b2879f665bd3c9876e817e9588630236e59494c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

SSH connection to smithi082 was lost: 'sudo lsb_release -is'

SSH connection to smithi171 was lost: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph rmpool unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it'

Command failed on smithi075 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0'

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --pool-snaps --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op snap_remove 50 --op snap_create 50 --op rollback 50 --op read 100 --op copy_from 50 --op write 50 --op write_excl 50 --op delete 50 --pool unique_pool_0'

reached maximum tries (130) after waiting for 780 seconds