Pulpito

jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931124

Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Nodes Status
2021-03-03 19:51:33 2021-03-04 01:57:21 2021-03-04 02:39:55 0:42:34 0:24:46 0:17:48 smithi master centos 8.2 4 fail

Description: rbd/singleton-bluestore/{all/issue-20295 objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_8}}

Log: http://qa-proxy.ceph.com/teuthology/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931124/teuthology.log

Failure Reason:

"2021-03-04T02:32:31.910875+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4548.0:42469 2.1 2:853c1948:::rbd_object_map.11bef0ec5e52:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected e32) initiated 2021-03-04T02:32:01.316265+0000 currently delayed" in cluster log

All Details...

  • log_href: http://qa-proxy.ceph.com/teuthology/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931124/teuthology.log
  • archive_path: /home/teuthworker/archive/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931124
  • description: rbd/singleton-bluestore/{all/issue-20295 objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_8}}
  • duration: 0:24:46
  • email:
  • failure_reason: "2021-03-04T02:32:31.910875+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4548.0:42469 2.1 2:853c1948:::rbd_object_map.11bef0ec5e52:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected e32) initiated 2021-03-04T02:32:01.316265+0000 currently delayed" in cluster log
  • flavor: basic
  • job_id: 5931124
  • kernel:
    • flavor: basic
    • sha1: 28aa047c37d1abb529b606e9c1358bd6f0f2e5eb
    • kdb: True
  • last_in_suite: False
  • machine_type: smithi
  • name: jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi
  • nuke_on_error: True
  • os_type: centos
  • os_version: 8.2
  • overrides:
    • ceph-deploy:
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
    • selinux:
      • whitelist:
        • scontext=system_u:system_r:logrotate_t:s0
    • workunit:
      • sha1: 8889feaaec2d94620fc3944e2f6d7a30dbcc44b1
      • branch: wip-qa-objectstore-logs
    • ceph:
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • sha1: f06f3e977d8d68713bf7507af35c9e278d8753b5
      • fs: xfs
      • conf:
        • mgr:
          • debug ms: 1
          • debug mgr: 20
        • mon:
          • debug paxos: 20
          • debug mon: 20
          • debug ms: 1
        • osd:
          • mon osd full ratio: 0.9
          • bluestore compression algorithm: snappy
          • debug ms: 1
          • bluestore fsck on mount: True
          • debug osd: 25
          • bluestore compression mode: aggressive
          • debug bluestore: 1/20
          • debug bluefs: 1/20
          • osd objectstore: bluestore
          • mon osd backfillfull_ratio: 0.85
          • bluestore block size: 96636764160
          • debug filestore: 20
          • debug rocksdb: 4/10
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
          • debug journal: 20
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
    • install:
      • ceph:
        • sha1: f06f3e977d8d68713bf7507af35c9e278d8753b5
    • admin_socket:
      • branch: wip-jd-testing
    • thrashosds:
      • bdev_inject_crash_probability: 0.5
      • bdev_inject_crash: 2
  • owner: scheduled_jdillaman@cube-1
  • pid:
  • roles:
    • ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']
    • ['mon.b', 'mgr.y', 'osd.3', 'osd.4', 'osd.5']
    • ['mon.c', 'mgr.z', 'osd.6', 'osd.7', 'osd.8']
    • ['osd.9', 'osd.10', 'osd.11']
  • sentry_event:
  • status: fail
  • success: False
  • branch: wip-jd-testing
  • seed:
  • sha1: f06f3e977d8d68713bf7507af35c9e278d8753b5
  • subset:
  • suite:
  • suite_branch: wip-qa-objectstore-logs
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 8889feaaec2d94620fc3944e2f6d7a30dbcc44b1
  • targets:
    • smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMA1wO4W1Q1CPITBf514AHABEEWeaRBKpDIeBMprxEIkbxDfEaUqXMOP3etysCBwwghuRmfKfwcMJu+v2405nd6LvaBIPj6s+PHOYkGrnApIsnQaOtPZZ1v1vBzC6OXQviUdGzhT8F/6bc3cT2koBAN3dfa3Q8hE1clzy81SQIZPEBnwnKGgUQVemw4N+7E7Zcja9ElImryf90S0dnirJyN3kmz4iCPVdGCr4MsEvRhpnvSoR6Dnu28scwGFZtm+4yjJzcjdk47ajiY2Akpo/qQ7W9MIrmwQpyxcJrF4/gBzgZYVa81OU3Wfiw1hfqeADYvufggZFLvhrTSxsdCSYjbbB9HTw7VPy+Fcr1hngGCyU5uUS3rHmUtUdIcROLvpesvddOxgsDQsy9oZTi7OxD/IwV2rIENIjbgJsDj15nzLmCxX2TJiLwW/hh/ph42O0Jq2xAUd7yXlrK/627BfFMTQWVMh253slBILKYM9D/Tg49H/+kZqaXnV2a9fOOEt8=
    • smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJZjYKo/IeqhHTdxAd1d9oFXXrhXQPAuXpnJAk3s53Std2J6GPH5MiahKmA+jQizlzAuirTuI4SubmAT3gcW4f95ex3fz6/y7N5lU5xLV2QlQpAySC7vhv/wrxRInDgG7cZ+VSQ8oJJeDNjxKQ84pYwEAjsosWFo1rkGvHZ0FbRf+Qh7jHyz0XF88t3VTcOgKyuxv1f1Qa/aHsqS/6tGvY1CfyXoKt6vNeJcBaAyqS/r0MwOzoTgIGOUmXqNIHxFPTPWCbfDVzeY4LEuzqWQroSoIZbFMZQR2qwGsfuK4TjROo+2JRBZ4QTHsLkl8nxogUbiI2RiO74V2zwj8o1xfd5pXmvq5T+HFJ/AvOHg/zOPHLkn+32FKtmDhdEZZh56XtOtFGU1CO25so2/zYDKmsGe3JjxUO3pluNPwqIO422SgL8Hh7CaKFaIzIac5bl4poVmT8sLjdlBX9smcV9wATAvQpIt73pcdplgou4ahlZE9mSkLv006dyxRqEb8lAaM=
    • smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0FnAym0TvwmwFLGA2uFWNphlgP4YBbwBjiozdZYbojJ2C3jZms52kWix/vPgqeiYs5k2ZxzNvXz00vIo2x6NOftAsyL6ld90Bt5fNATn1uSQUp7w3huXwQNf/zdFU90lt/iE1GeWApcuJMm38iY+wQf6n4UMRvtu3jHWJvEusQnO4syXjcP/IiK9oXfntZYHqAg6JaqJ+R+mri12pFlBacEFT0dKrChJbX4abW4ywi/0LpNYk3l6j3e3mHUD3lw8taF0uT7s6M0Yp6pzjDpQ96go1SSCvFAihSd8PgTdC1CF3C7sQa+GJkPpOQJaM5+48/srRMEhNRjvNha4aQ31+TO0Gyh4V8anAkU3ctkcxOuncrpbGpjWQMQ/qSM1tvb0ZWo/qpjYFUO6sbnj6Jp/ZgDGU7CYCqP4PvIz5X3QV9MEcg4cHu8oOpo9cJI5ZLXUZZLfVpFJkPsWShEcx0RuUcabTNAgoMjCYpXK+gsHTc7X9vz1+hpLxXBfxFTu+4D8=
    • smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCukaD35/nNpxoYiYmdf146rcBN26Pb1aAwHITjsjnsd4p+J9HV+HQXRBKwFmi6l8P0cyPcnz8g21p2zPhmpI5EuogWeFGleHZEzPQYjP98+QF4VP51C+CV2BZjmSvCRVsggxxHwWHOcSIobSo8gE+rlY9UkB68IvNaMGLJbBJPMk+W+E94/7WGoNJwOUnsNb2YmVUdGhC3HGIrtqq4Z31/T4DZlzMjGUPRjGFMQ/P/O5pT0hz/V7SngMN4x9dJ8188Q0nLyl0fdoWibmAVongDNnbdYIc6Z9RLbiG0Mi5IM1Bd9ZyE/ggpmw0xUtaB3usigOexX8b6+eSYy0hV/0PymtVIyDDEcQq6udl/Zy9Py/aTsw2MHhBtFtM6F0tpD9DCqQh/h6wD03XGeKLbN2u0mf6BYKPSHlmojJY0BT3RvrVn99eDDAZ1gLQB2uCcaA5dRRp7TKL4zTp82VVW7khXwPui4Q9eosI0794gk8GFUVsC54n5EnQbXGCPfUse/zk=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • flavor: basic
      • sha1: 28aa047c37d1abb529b606e9c1358bd6f0f2e5eb
      • kdb: True
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • install:
    • ceph:
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • fs: xfs
      • cluster: ceph
      • sha1: f06f3e977d8d68713bf7507af35c9e278d8753b5
      • conf:
        • mgr:
          • debug ms: 1
          • debug mgr: 20
        • mon:
          • debug paxos: 20
          • debug mon: 20
          • debug ms: 1
        • osd:
          • mon osd full ratio: 0.9
          • bluestore compression algorithm: snappy
          • debug ms: 1
          • bluestore fsck on mount: True
          • debug osd: 25
          • bluestore compression mode: aggressive
          • debug bluestore: 1/20
          • debug bluefs: 1/20
          • osd objectstore: bluestore
          • mon osd backfillfull_ratio: 0.85
          • bluestore block size: 96636764160
          • debug filestore: 20
          • debug rocksdb: 4/10
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
          • debug journal: 20
      • log-ignorelist:
        • application not enabled
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
    • workunit:
      • clients:
        • all:
          • rbd/issue-20295.sh
      • branch: wip-qa-objectstore-logs
      • timeout: 30m
      • sha1: 8889feaaec2d94620fc3944e2f6d7a30dbcc44b1
  • teuthology_branch: master
  • verbose: False
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2021-03-03 19:51:33
  • started: 2021-03-04 01:57:21
  • updated: 2021-03-04 02:39:55
  • status_class: danger
  • runtime: 0:42:34
  • wait_time: 0:17:48