Pulpito

yuriw-2021-07-08_23:26:56-kcephfs-wip-yuri3-testing-2021-07-08-1143-octopus-testing-basic-smithi/6259572

Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Nodes Status
2021-07-08 23:27:44 2021-07-09 01:43:03 2021-07-09 02:11:43 0:28:40 0:15:00 0:13:40 smithi master ubuntu 20.04 6 pass

Description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}

Log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-08_23:26:56-kcephfs-wip-yuri3-testing-2021-07-08-1143-octopus-testing-basic-smithi/6259572/teuthology.log

All Details...

  • log_href: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-08_23:26:56-kcephfs-wip-yuri3-testing-2021-07-08-1143-octopus-testing-basic-smithi/6259572/teuthology.log
  • archive_path: /home/teuthworker/archive/yuriw-2021-07-08_23:26:56-kcephfs-wip-yuri3-testing-2021-07-08-1143-octopus-testing-basic-smithi/6259572
  • description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}
  • duration: 0:15:00
  • email: ceph-qa@ceph.io
  • failure_reason:
  • flavor: basic
  • job_id: 6259572
  • kernel:
    • flavor: basic
    • client:
      • branch: testing
    • kdb: True
    • sha1: 64887ecca52b9d754c09837b7242b80463bda63c
  • last_in_suite: False
  • machine_type: smithi
  • name: yuriw-2021-07-08_23:26:56-kcephfs-wip-yuri3-testing-2021-07-08-1143-octopus-testing-basic-smithi
  • nuke_on_error: True
  • os_type: ubuntu
  • os_version: 20.04
  • overrides:
    • ceph-deploy:
      • fs: xfs
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
        • osd:
          • mon osd full ratio: 0.9
          • mon osd backfillfull_ratio: 0.85
          • bluestore fsck on mount: True
          • mon osd nearfull ratio: 0.8
          • debug bluestore: 20
          • debug bluefs: 20
          • osd objectstore: bluestore
          • bluestore block size: 96636764160
          • debug rocksdb: 10
          • osd failsafe full ratio: 0.95
      • bluestore: True
    • workunit:
      • sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
      • branch: wip-yuri3-testing-2021-07-08-1143-octopus
    • ceph:
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • fs: xfs
      • cephfs_ec_profile:
        • m=2
        • k=2
        • crush-failure-domain=osd
      • sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
      • conf:
        • global:
          • ms die on skipped message: False
        • mgr:
          • debug ms: 1
          • debug mgr: 20
        • client:
          • rados osd op timeout: 15m
          • debug ms: 1
          • rados mon op timeout: 15m
          • debug client: 20
          • client mount timeout: 600
        • mon:
          • debug paxos: 20
          • debug mon: 20
          • debug ms: 1
          • mon op complaint time: 120
        • mds:
          • mds bal split bits: 3
          • mds bal split size: 100
          • osd op complaint time: 180
          • debug mds: 20
          • mds bal merge size: 5
          • debug ms: 1
          • mds bal frag: True
          • mds verify scatter: True
          • mds bal fragment size max: 10000
          • mds op complaint time: 180
          • rados mon op timeout: 15m
          • rados osd op timeout: 15m
          • mds debug scatterstat: True
          • mds debug frag: True
        • osd:
          • mon osd full ratio: 0.9
          • debug ms: 1
          • bluestore fsck on mount: True
          • debug osd: 20
          • debug bluestore: 20
          • debug bluefs: 20
          • osd objectstore: bluestore
          • mon osd backfillfull_ratio: 0.85
          • osd op complaint time: 180
          • bluestore block size: 96636764160
          • osd shutdown pgref assert: True
          • debug rocksdb: 10
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • overall HEALTH_
        • \(FS_DEGRADED\)
        • \(MDS_FAILED\)
        • \(MDS_DEGRADED\)
        • \(FS_WITH_FAILED_MDS\)
        • \(MDS_DAMAGE\)
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • \(FS_INLINE_DATA_DEPRECATED\)
        • overall HEALTH_
        • \(OSD_DOWN\)
        • \(OSD_
        • but it is still running
        • is not responding
        • inode wrongly marked free
        • bad backtrace on inode
        • inode table repaired for inode
        • Scrub error on inode
        • Metadata damage detected
    • install:
      • ceph:
        • sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
    • admin_socket:
      • branch: wip-yuri3-testing-2021-07-08-1143-octopus
    • thrashosds:
      • bdev_inject_crash_probability: 0.5
      • bdev_inject_crash: 2
  • owner: scheduled_yuriw@teuthology
  • pid:
  • roles:
    • ['mon.a', 'mgr.y', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
    • ['mon.b', 'mon.c', 'mgr.x', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
    • ['client.0']
    • ['client.1']
    • ['client.2']
    • ['client.3']
  • sentry_event:
  • status: pass
  • success: True
  • branch: wip-yuri3-testing-2021-07-08-1143-octopus
  • seed:
  • sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
  • subset:
  • suite:
  • suite_branch: wip-yuri3-testing-2021-07-08-1143-octopus
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
  • targets:
    • smithi051.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvEi7EMWMwY7PbTHydQ0xYULk1K5YILq0L93ZoEbisObos5z6Dfy8rzwxFxPmOTPNGrLyMxajO5I+KmP9Jca/6bzyMqnWGBKl1DXrCMefK9sw6k4SkJrUShttKvE34vrFd9FqeT+LvqnQg9Wmd/o/nVHwvY2B6QVeayuYfcB0XcTwNqvYNvOm6uEI7DJxuIx0qRBwqmO+n328qDDONZtdvicgTv6604oV2ogTZOoi0qFJPis62KS0sIk5h+PLNYId+M5QHd4m36TmXwFYVw1SSgbJKMfiZt0EzoAPNEM7umVCeRcZ58ageTIJfC/b2H5BC6EDV6SWnBSbPgRsfTH3jFvy4WLDwzvkD1INZaf/5VSIDVeBEQBtTPSqpX6urSpu89ka+5M452Zqi2RQt92LmScnVgRN+etqajNxsbtvvlMqi5Ek2r1zq+3YSw4lc1K7kq6qUSe2c4xpJXm/M9yAtn90XGNiQX3LaLZjK8PXR3lsWs5n/M7A4qtFTHjP/Rf0=
    • smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC05klakNYRm95YDqsOrTN6Gy3eeqsb0ErPLJ736qUKBsUwq1+bodfaNgJ7/oAarZqSIT0b9XNdkFZ5IxZGwWKo2c5mOr0NMb7PiAgb+mxYgYp3cfeKvZEnEj8ONDOjrmVpbcS3x6Z569ARORrcmlO+/osmwnGEdmjYFLPCwZ+J6Tl2sQPhNqG61YfGWROjddya8UpF7+95z09o+w6lU3WMqKporbR71yxbQDde2jfk+tL6tL1nvu2/7wXptJIV4yy6GKef/rHQSb+oGGuhDBkL0EB8UEkYhpMPUpzU5U05nBcLtDE9y3bxypijqarO2ITGWVF4yKpuXdItJ5Ae5gDe40yWmLZ+/RlKsQBYDYU+rrgKtYWWJqUxPLeDnc30FCvnrizQOGr4LVCka9tfEgPu5kEglG0lpJDyoo5Aq8yg6JMM5AqRy3wQuvuT1aKh9bL+abW4m2UZN0hOS1VJEV2GiYR2aL3U1CU2NF4TaPdfiD42UM4gcYfg60+4BDWiGJk=
    • smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+QnjbnmJqMw+XZMTjnbvK0LP1rrocUQ09z3CaIRf5gld5s3xHnO9ab1eyF+pc+6M2H+TEvqdFhOhoig/N+/6pzZtO94vICahm6ml/3AvrPGmbl0Zhv+k+guJs/bspnsipcQKdak4ZWKf88kzrE13H5IySKHL4+AMx6WrW72SeHgr08zb1UhnNgBcNWlSVFpkMS6sR01SQN0rlma1QQsuqTcw9MYxUdKpH1CfmZoy5T0dCBx1fJ2QanJ5MqHkVq5T7/JgsaGLdRVFkNPECaaJTMvR40fldV3QSbGGJCfDVcDoimMPmI8SVjLXmuGYO210bd81VCZsW+D3gduU7HdvMSnCGprwhdEigtxugjNa8QNeKDgmeJP9duy6VRvGFJXyG66y+75NPa34m4L/1f3BAmJJG013ZFOxiU9XJU3VBbwSxGTonwNJ2j2Pf4j/jYQ2yYZtSMIMhZUAHPe7RbaiDe413B67d0h0Z9OTOLjZ59BkB+3QJPKYXrE7I4m31/1k=
    • smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsv1fH61E+hc/RLy/MUkF7J+XB2qzdCXXvoFEKRtS5tf4LZTdnvsYhbzJ1nnn+PykMDQnzEFow+3EXVUk2G3Okd3c5PcsYPfDFB3b5zJXG6vJ/vZ8F2krrFRo95QJcqmrnx24gp/KFUvYnkOUnF6g9jROoqA9lLJR8yDJOI7f1Mz6J7GbdRXnAwqdzuei72Lk2JtWuUJgyXchLIzi8+iwLtb9ongvpfBDLvq+JTtq4w9VeaBIYhVtmyOXU4HeQfMTjT2pvHtisdctNfK3X/gcnoc3Vm29IgyElD4UYlVyQnS/xkgDwQdhpNQ3HRcqn6gKwktSmHu5Yac5l2+/yMiN4nIpbi4QIbLQoeQGkiTZ+d6K+SjGF3ZyiBBP25dlPdSHwpsVMOIN5aQ1g/SHjsW53gcu+Q1KQh4OzP6dVdJn1j0Ff8oLsTQLq0xY1PB+qmv6LUTZl+bCNxIqXQu0V3bIC2OZ5cG3PhR83yAVe1KdeJBnnLkfAc2nszteT15dMKSk=
    • smithi094.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxGqCwlkGIkz4Kjn/EjlydBCIAeq1J+dM4MYxOgk+HqGvDPfxcbDHllJ2qBJi95HZcISwwru/xjmC7iFsdP9hNXBKgZ7NyV/PvI5wKIadOdLKJWhnOS7UfW3a0FHeEotZVWeqeFqo8C66FVohvyr0y6RHDNIKxeZ+jw1Oo/xnQ/igbZH8sc3BQcGi7C0E0TyvVLV6uNywhyCAc09eqc8kHxTYcKTcPSPnpXWVnFBbyrvA+xb1rvZ0uhH7f9xR/AV6ODJSCY/EVpgBq06YYrH1yxdP+kj3dFa9xSf+R6E1xSjtoDBIubLjOEs5MOtVGHXpbVB7RE7FMfpOuf+wYO1esDKKnWfSmrdz0g7hkyEAhHR/RR9QWJMhiXV+gh3GsC1qfDQouV6fDGAI5IoULhZhH0j17uqtJVl34dHN0FQ3CmRZpMiXq08GYG+c6c8lu6kxHsWyr1DdnZgY+5h3iO9lpyIVGIzw9BkRWB/qWDw1nLtF39dasZtN5OFNAnM9E0Gc=
    • smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJo8pFrHcbxRyjZjGuAyvr6A1+ijmtjqk1JIJxBEbh3na8g999mYIqNRuER+PmZJJGM8wlKSVfyBmAkPuBC1gQsPb+pdgnvJz+1Me5XwMoA4kC9aQNyTdPsV7ywTbhB3Ws7nwNpGB1AaA2MhQUO6RmFVRNo27Pwk4qv0uQGVMDEXIowP8U/XCT5FGHnfnakTwQZyrmlPLIuw1ji4Siz/IG5RSAQsuHPHnPgOVz2xmTDSmupNRqgQy6cutxJHd8a0jy1wgM5cnvocZIG23JZG2b+9Gk9d6r6npXJn7+x67Cx0Swz2TLzpOhBdwxtXTqz9Y/dqs/62LF0S6dQvf1pEVSwcF7AJouioNAjDuAi645QV17z9GbiuEdQggokrZxZklnTf1JGvgYRyTmVC42At6bhVEGNHbc7gJmJcNSDUj2C4bgDAni5SBh66/VdK9WhXey2nWG/A/W58sHs5uTpgmbF2fL7N4jTsLRtlGITxvNXliZ7k0XWE+RbL4IqedjpS8=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • flavor: basic
      • client:
        • branch: testing
      • kdb: True
      • sha1: 64887ecca52b9d754c09837b7242b80463bda63c
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • install:
      • extra_system_packages:
        • deb:
          • bison
          • flex
          • libelf-dev
          • libssl-dev
          • dump
          • indent
          • libaio-dev
          • libtool-bin
          • uuid-dev
          • xfslibs-dev
        • rpm:
          • bison
          • flex
          • elfutils-libelf-devel
          • openssl-devel
          • libacl-devel
          • libaio-devel
          • libattr-devel
          • libtool
          • libuuid-devel
          • xfsdump
          • xfsprogs
          • xfsprogs-devel
          • libaio-devel
          • libtool
          • libuuid-devel
          • xfsprogs-devel
      • extra_packages:
        • deb:
          • python3-cephfs
          • cephfs-shell
        • rpm:
          • python3-cephfs
      • sha1: c1c2db705640c4c0cb237fa1f069b791a4e36559
    • ceph:
    • kclient:
    • cephfs_test_runner:
      • modules:
        • tasks.cephfs.test_forward_scrub
  • teuthology_branch: master
  • verbose: True
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2021-07-08 23:27:44
  • started: 2021-07-09 01:43:03
  • updated: 2021-07-09 02:11:43
  • status_class: success
  • runtime: 0:28:40
  • wait_time: 0:13:40