Pulpito

teuthology-2016-07-22_03:00:03-rados-hammer-distro-basic-vps/328515

Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Nodes Status
2016-07-22 10:03:18 2016-07-23 21:57:27 2016-07-23 22:27:15 0:29:48 0:20:43 0:09:05 vps master ubuntu 2 fail
Nodes: vpm097 vpm047

Description: rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml}

Log: http://qa-proxy.ceph.com/teuthology/teuthology-2016-07-22_03:00:03-rados-hammer-distro-basic-vps/328515/teuthology.log

Sentry event: http://sentry.ceph.com/sepia/teuthology/?q=dd73bc2b927b46428acd70fed6c12e66

Performance graphs: http://pcp.front.sepia.ceph.com:44323/grafana/index.html#/dashboard/script/index.js?time_to=2016-07-23T22%3A20%3A47&time_from=2016-07-23T22%3A02%3A47&hosts=vpm047%2Cvpm097

Failure Reason:

{'vpm047.front.sepia.ceph.com': {'changed': False, '_ansible_no_log': False, 'stdout': '', '_ansible_delegated_vars': {'ansible_host': 'localhost'}, 'failed': True, 'stderr': "fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.\n\nIf no other git process is currently running, this probably means a\ngit process crashed in this repository earlier. Make sure no other git\nprocess is running and remove the file manually to continue.\n", 'rc': 128, 'invocation': {'module_name': 'git', 'module_args': {'force': True, 'track_submodules': False, 'reference': None, 'dest': '/var/lib/teuthworker/.cache/src/keys', 'verify_commit': False, 'clone': True, 'update': True, 'ssh_opts': None, 'repo': 'https://github.com/ceph/keys', 'bare': False, 'refspec': None, 'executable': None, 'remote': 'origin', 'recursive': True, 'accept_hostkey': False, 'depth': None, 'version': 'master', 'key_file': None}}, 'stdout_lines': [], 'msg': 'Failed to checkout branch master'}}

All Details...

  • log_href: http://qa-proxy.ceph.com/teuthology/teuthology-2016-07-22_03:00:03-rados-hammer-distro-basic-vps/328515/teuthology.log
  • archive_path: /var/lib/teuthworker/archive/teuthology-2016-07-22_03:00:03-rados-hammer-distro-basic-vps/328515
  • description: rados/thrash/{0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml}
  • duration: 0:20:43
  • email: ceph-qa@ceph.com
  • failure_reason: {'vpm047.front.sepia.ceph.com': {'changed': False, '_ansible_no_log': False, 'stdout': '', '_ansible_delegated_vars': {'ansible_host': 'localhost'}, 'failed': True, 'stderr': "fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.\n\nIf no other git process is currently running, this probably means a\ngit process crashed in this repository earlier. Make sure no other git\nprocess is running and remove the file manually to continue.\n", 'rc': 128, 'invocation': {'module_name': 'git', 'module_args': {'force': True, 'track_submodules': False, 'reference': None, 'dest': '/var/lib/teuthworker/.cache/src/keys', 'verify_commit': False, 'clone': True, 'update': True, 'ssh_opts': None, 'repo': 'https://github.com/ceph/keys', 'bare': False, 'refspec': None, 'executable': None, 'remote': 'origin', 'recursive': True, 'accept_hostkey': False, 'depth': None, 'version': 'master', 'key_file': None}}, 'stdout_lines': [], 'msg': 'Failed to checkout branch master'}}
  • flavor:
  • job_id: 328515
  • kernel:
    • sha1: distro
    • kdb: True
  • last_in_suite: False
  • machine_type: vps
  • name: teuthology-2016-07-22_03:00:03-rados-hammer-distro-basic-vps
  • nuke_on_error: True
  • os_type: ubuntu
  • os_version:
  • overrides:
    • ceph:
      • log-whitelist:
        • slow request
      • fs: ext4
      • conf:
        • global:
          • ms inject delay max: 1
          • osd_pool_default_size: 2
          • osd_pool_default_min_size: 1
          • ms inject delay probability: 0.005
          • ms inject socket failures: 2500
          • ms inject delay type: osd
          • ms inject internal delays: 0.002
        • mon:
          • debug mon: 20
          • debug paxos: 20
          • debug ms: 1
        • osd:
          • debug osd: 25
          • debug filestore: 20
          • debug journal: 20
          • debug ms: 1
      • sha1: 387d5c1ba836833a0cf11ddf9a4fb8690a532878
    • ceph-deploy:
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
          • debug mon: 1
          • debug paxos: 20
          • debug ms: 20
    • workunit:
      • sha1: 387d5c1ba836833a0cf11ddf9a4fb8690a532878
    • install:
      • ceph:
        • sha1: 387d5c1ba836833a0cf11ddf9a4fb8690a532878
    • admin_socket:
      • branch: hammer
  • owner: scheduled_teuthology@teuthology
  • pid:
  • roles:
    • ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2', 'client.0']
    • ['mon.b', 'osd.3', 'osd.4', 'osd.5', 'client.1']
  • sentry_event: http://sentry.ceph.com/sepia/teuthology/?q=dd73bc2b927b46428acd70fed6c12e66
  • status: fail
  • success: False
  • branch: hammer
  • seed:
  • sha1: 387d5c1ba836833a0cf11ddf9a4fb8690a532878
  • subset:
  • suite:
  • suite_branch: hammer
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1:
  • targets:
    • ubuntu@vpm097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqZHMUePK+BKA0FPu9e9qcx9OG639EfRTePLHrPnRQdJaHfTDhRI72yHKa2zgm+1lxVgQLtTOmI1aPz/KrNEuqveyjCfZDJh0rUWFTle3TApi+mnD+N/hlcZkvFokxr2CKjtisXH6vxVhYoCMs6H+UvMscpHMI0SBRGoCxVLKuW7wFaq9bRtBk6U+eqB0MKNXCkyz7j+3dqJcN5MP9izAcWLmA1MUhE78N+OF+XohxhzZU6WH2wlof25w9hEc/QfgzG9XViLwFIeFEmvCKdu9zjyzQrdIQ0yjwSK++gIxnXqYy6TZlX/NXlclD2f2ggOqb7BX8UEbk8zKfxQsoaL4J
    • ubuntu@vpm047.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD37nxaItjDUvA9u2yFKEwHaF2UVHIlDjy2mInwJ189G8L1vwLKRjMdvfCbiCqpbL+VaAHr8QUFkAoJtAMDizZTI34KqfJjytiXzcYDUHMUYT0JWw0Nh6sywtPGJ/Pxq5utWvz8e5DhkbAjUwPoTTxBGWO3wFXA/XUFvC+FTpXvnR7R7nJrKaRCX2hVfh4ZS9Dh9xOOfw/Ks4UJUdO4RmkUdjWp39yxCFrAODuhV09GHNeEWXdWC7lMSBSbj5Dd27MeavaUDnQs42vP0XstM1U8p42xaFQm2Kp4CHXN8eaEroSkaS8+TBr2UP6yXERFvW4s2T4jlwfniBhD2JYTgGgH
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.lock_machines:
      • 2
      • vps
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • sha1: distro
      • kdb: True
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock.check:
    • install:
    • ceph:
      • log-whitelist:
        • wrongly marked me down
        • objects unfound and apparently lost
      • conf:
        • osd:
          • osd max backfills: 1
          • osd scrub min interval: 60
          • osd scrub max interval: 120
    • thrashosds:
      • chance_pgnum_grow: 3
      • chance_pgpnum_fix: 1
      • timeout: 1200
    • rados:
      • op_weights:
        • snap_remove: 50
        • snap_create: 50
        • rollback: 50
        • read: 100
        • copy_from: 50
        • write: 100
        • delete: 50
      • clients:
        • client.0
      • objects: 50
      • ops: 4000
  • teuthology_branch: master
  • verbose: True
  • pcp_grafana_url: http://pcp.front.sepia.ceph.com:44323/grafana/index.html#/dashboard/script/index.js?time_to=2016-07-23T22%3A20%3A47&time_from=2016-07-23T22%3A02%3A47&hosts=vpm047%2Cvpm097
  • priority:
  • user:
  • queue:
  • posted: 2016-07-22 10:03:18
  • started: 2016-07-23 21:57:27
  • updated: 2016-07-23 22:27:15
  • status_class: danger
  • runtime: 0:29:48
  • wait_time: 0:09:05