Pulpito

yuriw-2021-07-28_21:00:00-kcephfs-wip-yuri7-testing-2021-07-28-1155-octopus-testing-basic-smithi/6299553

Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Nodes Status
2021-07-28 21:00:49 2021-07-29 00:27:27 2021-07-29 00:58:17 0:30:50 0:14:53 0:15:57 smithi master ubuntu 20.04 6 pass

Description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}

Log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-28_21:00:00-kcephfs-wip-yuri7-testing-2021-07-28-1155-octopus-testing-basic-smithi/6299553/teuthology.log

All Details...

  • log_href: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-28_21:00:00-kcephfs-wip-yuri7-testing-2021-07-28-1155-octopus-testing-basic-smithi/6299553/teuthology.log
  • archive_path: /home/teuthworker/archive/yuriw-2021-07-28_21:00:00-kcephfs-wip-yuri7-testing-2021-07-28-1155-octopus-testing-basic-smithi/6299553
  • description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}
  • duration: 0:14:53
  • email: ceph-qa@ceph.io
  • failure_reason:
  • flavor: basic
  • job_id: 6299553
  • kernel:
    • flavor: basic
    • client:
      • branch: testing
    • kdb: True
    • sha1: 3f5ecabf87d7e3e7a3812fba9af5fd1ee2bfa205
  • last_in_suite: False
  • machine_type: smithi
  • name: yuriw-2021-07-28_21:00:00-kcephfs-wip-yuri7-testing-2021-07-28-1155-octopus-testing-basic-smithi
  • nuke_on_error: True
  • os_type: ubuntu
  • os_version: 20.04
  • overrides:
    • ceph-deploy:
      • fs: xfs
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
        • osd:
          • mon osd full ratio: 0.9
          • mon osd backfillfull_ratio: 0.85
          • bluestore fsck on mount: True
          • mon osd nearfull ratio: 0.8
          • debug bluestore: 20
          • debug bluefs: 20
          • osd objectstore: bluestore
          • bluestore block size: 96636764160
          • debug rocksdb: 10
          • osd failsafe full ratio: 0.95
      • bluestore: True
    • workunit:
      • sha1: 24317f834a194d1f3573de92f0e624646b6b912e
      • branch: wip-yuri7-testing-2021-07-28-1155-octopus
    • ceph:
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • fs: xfs
      • cephfs_ec_profile:
        • m=2
        • k=2
        • crush-failure-domain=osd
      • sha1: 24317f834a194d1f3573de92f0e624646b6b912e
      • conf:
        • global:
          • ms die on skipped message: False
        • mgr:
          • debug ms: 1
          • debug mgr: 20
        • client:
          • rados osd op timeout: 15m
          • debug ms: 1
          • rados mon op timeout: 15m
          • debug client: 20
          • client mount timeout: 600
        • mon:
          • debug paxos: 20
          • debug mon: 20
          • debug ms: 1
          • mon op complaint time: 120
        • mds:
          • mds bal split bits: 3
          • mds bal split size: 100
          • osd op complaint time: 180
          • debug mds: 20
          • mds bal merge size: 5
          • debug ms: 1
          • mds bal frag: True
          • mds verify scatter: True
          • mds bal fragment size max: 10000
          • mds op complaint time: 180
          • rados mon op timeout: 15m
          • rados osd op timeout: 15m
          • mds debug scatterstat: True
          • mds debug frag: True
        • osd:
          • mon osd full ratio: 0.9
          • debug ms: 1
          • bluestore fsck on mount: True
          • debug osd: 20
          • debug bluestore: 20
          • debug bluefs: 20
          • osd objectstore: bluestore
          • mon osd backfillfull_ratio: 0.85
          • osd op complaint time: 180
          • bluestore block size: 96636764160
          • osd shutdown pgref assert: True
          • debug rocksdb: 10
          • mon osd nearfull ratio: 0.8
          • osd failsafe full ratio: 0.95
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • overall HEALTH_
        • \(FS_DEGRADED\)
        • \(MDS_FAILED\)
        • \(MDS_DEGRADED\)
        • \(FS_WITH_FAILED_MDS\)
        • \(MDS_DAMAGE\)
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • \(FS_INLINE_DATA_DEPRECATED\)
        • overall HEALTH_
        • \(OSD_DOWN\)
        • \(OSD_
        • but it is still running
        • is not responding
        • inode wrongly marked free
        • bad backtrace on inode
        • inode table repaired for inode
        • Scrub error on inode
        • Metadata damage detected
    • install:
      • ceph:
        • sha1: 24317f834a194d1f3573de92f0e624646b6b912e
    • admin_socket:
      • branch: wip-yuri7-testing-2021-07-28-1155-octopus
    • thrashosds:
      • bdev_inject_crash_probability: 0.5
      • bdev_inject_crash: 2
  • owner: scheduled_yuriw@teuthology
  • pid:
  • roles:
    • ['mon.a', 'mgr.y', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
    • ['mon.b', 'mon.c', 'mgr.x', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
    • ['client.0']
    • ['client.1']
    • ['client.2']
    • ['client.3']
  • sentry_event:
  • status: pass
  • success: True
  • branch: wip-yuri7-testing-2021-07-28-1155-octopus
  • seed:
  • sha1: 24317f834a194d1f3573de92f0e624646b6b912e
  • subset:
  • suite:
  • suite_branch: wip-yuri7-testing-2021-07-28-1155-octopus
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 24317f834a194d1f3573de92f0e624646b6b912e
  • targets:
    • smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCSgPKHZE4MVY8t3VuvT0eHeMABRMArcut5GaNOePAi0YhjScx6OCvWpxfvnr2FN6x/NmHHsIIPS21Ffp+pEV87Gou0il884ZSuMuXedbEimGJzRMkaAL4v3efBcMjDZiWcTeHgMu9hahmuclYljjG9os+1N43HpGw3e9IhIv1o4b+sx8X8cTybBvHHfmqtO03lZtfIFHHQrZuBz3Fl3JhbLy11o41wBxCsQmmuaAZGCbl5ESjen/GVrtN527xvpL8NFQ159Vl5O6hjtzai6Ia+inRS6jRbz2GAnokz/VEIOkEB+BIaxnu+letmtqcoRUsebmNalBjd86YGWOriH+XKAUSY+/GP+Vk7SRAgjjVD7aDW1fW33yCdVLQCJ96rJf+4daeb1kN9gdpM7umq36SNZI4r9pCKHzlAeoU2umTl5hbZRqpkdb2sNo+CJYYVyFWeQOFBLmmZbZjPW+MR1F8lRXXGn3kYBmIT204dE4BC+85Gg5v507qFFECyXPZPCJM=
    • smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB95tQzqbTVgHY68JGZbzR6afClXph9TM6CzgkN1VWjHH1XLM0Ktx0Ps6SUTd2mafXnsTj4fghh1yWT/a05nPzrzQ3qFTe/1IjKqJ3QEIUO3rn/SYIiz0NAWlPRBlqj18VPigYLMm5gm+VO58ra2UPpjcvYIQwrYr0OUEhRc80MZ3yBKvhUWIyBiQxYG6+mpvAkvq4tVyNqAt9EO/Rhjp660F71oa2QtPQ9ZuVc52PNt49fgKdeCVMdAmhQ39gFYiNuWUi91r6qq5HXP1V8/5Wj0NgJuQhCugRuyQ5gT+MBOwaJBugyndGgdhXL/3qhrizmxMUgdRDLzpFFOlooLY4qSQc00HE/8gaXq9h28m7mXWQTvY6TawH8NjicL9xjVpxnfUcFqGc4/Ap7lrlwiWwp6+ctWT9eFhnIOAdTr03C9xwnpiQ9ydxaRg+spiRKUvopmeDAyC8RhjjHQkSmsS7ejNyJEhnGjEYWMoqWepbXCLLmZeZSH6CSFLSvdWsY2M=
    • smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDARtefzNPD1Coeeq0+i8Imc/K9hzr8yN1UjsuDEuQjtmn660NB2V4l839YdDBAFFElrbD9kQUE6WaLjT9A3evPjz6IL8KJawkmTyDt8ScELGUdK3xqdWGrg0pZnTz8jHPWnlxXscTjXgDr7tex7sfR/7ci8aO55zeYZZq7THckC+jYROK5zLHFquZ+XjX0YxuAOmsh++KjQ7an6PC6CXQtgWHqlAVOw8zmaUTCV+6Q8r900FeWLzdSxIQK76QTQuqiK+kXetQN31OmkLEFzQgPf0mnm5dnM2EY74GN3AX8y/oIqfP9Mg2pG/j65dqdWsMPLVhjs2c3QExku16x278h0ohTaFLCCNNTrSQNW1fT4ByktGGXVvCjLUo9x6YD9Dt9/TzY9u9yp49UrLCG33G2UHmktNPYLDYjW7lTu1XCoUbxRw2S6OUuq5gJR3VNXMo2ql9TJBlSsamaRHr/h8fIt/vhRwNx2q/ArlBtyt5iKjzUI2cmPEksWpQhFivEqgc=
    • smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcNcQt45gQ0KdsvsuKrqIF/7aHkzcJZhR5Mm9vdGTmPOaWG02YNaLItnU7Ew1mHJfczH0PCBhhUo/pevlbApSHilU0rYEe6y5+IGtKpdqh6cX/J5fmsFFfPmfJkrpz+JZcTUTnooWqHgw9G6h6NYxkRYKmy0RFZ+DW6fa/ITL+gikcyYC946bMq/LlNDrZfkz0Ez3lNLS0VFJsCNo8mR144YAsvd/EF/4JdF3lDmJ2FwmcdcsoNWORZYnGxJ/BGnICefxF5c1UjQvx3Y/sC9+3E9ZY4tcA83eLcqHo0IqUb0K2nHeFClHxJvZTPthzTm4Qidcb4Uh8oamyrRiBehjCCETREkAuuhtKuQ64an+jsjt4fJslDneLkvRGASBSyyx3FqmA9OYvJbe5MueiqCleqdm9yPNqJilbq/UE6tgE6QJiGOJSiMbKx0CQkAt7Uvd+XxODF5rL+D1hsPTyHCqzKBYKyfL5kDcIqh2+iuVUCEYGZYNMcsVPVMLH5mhTxT8=
    • smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGWIfIPvoRJhn364Ht9eC6VQG5gW3r2OvwEnH+Ju5AaTLUb7rA9G/vR9c0a8sPZeOcYk7HQVdOCPC/jwuzbKVAs0iUpgT5u9EYorAV1xjD6WuPlh5gVrykWzgkriF4qTx1crUkZ2ymf8x8Wg7jg4Xw7tF1SCphSlQxpT5d9eydCOtt1FneicdnzNzbNSAQk/ap0b6er5gSsIww3LUNG3z266mbCNCDdUdhkvsuJZsgYbIVVK2N4Zuz0RH6ryBI3mpuvQezb402Kvbtnqo5zJxRU5Bc7Gl0aFaMOOizDWH+RK4VDehhz/Wd8PdG9vKTKffE9D9um52Ck8a109QJXB4ieVEBoCmqPZR746TD5ZX3+JiUDIVK99LdZfW5JCsFq9mXB3WssmdW4L4w1/fq9YOa7KQbcU8ke8SR7vYj4YFbWG0NKbEub7Av4PwmJDyf/FBUJm13ax4Q1TCkgJrvbbgsEbiKrJ409b703wOvAlAYQycWUlmHEXn5vAbdowg7tQc=
    • smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7P2edf5cLuJ82wEH49QMEuRBs659XoueEhKuHCkKxhHvyYRDFwb+TflFxK5AAfU01nNfHx52NABvmHv6G+92TFRxWo6HdjihicW9p6D1ufVPZZhPZEyLq0ag/DLkZFZufYVBo/ULvuy3hu2671EVbCi/HiP2idwO58NrDV1C/rBp+WoKMLIGAfPS0dkyGIVr7DTHldYGbzTSEyBvWX9WC5psFJa0WgdV4HTeFPqseIfxEHf+plvqflMxKI85P5ZlIR1vfe3wr0LOXWEJUcm9FMNFm6mMTcjw8Y/JQG14FpxA0zXV+uRCfVmn0SgPxRGqeZDLz0z18kUrldYPb6uRKshB4RViYBHO/DDjI5qVeOUH7SQsupzrmYncNmELjKTLD4iZaCWZJfKUfPsEjlb+FV1ob7vJrniPenWdAitR3FPCv/uRjEYUVMPsxNmRwz0OUGVo8OyAgd0gQaBWR1gQbpmKLRfD66+Fb/+sVHFQHsnmVyEgcgbeIL1yBzhml3c=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • flavor: basic
      • client:
        • branch: testing
      • kdb: True
      • sha1: 3f5ecabf87d7e3e7a3812fba9af5fd1ee2bfa205
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • install:
      • extra_system_packages:
        • deb:
          • bison
          • flex
          • libelf-dev
          • libssl-dev
          • dump
          • indent
          • libaio-dev
          • libtool-bin
          • uuid-dev
          • xfslibs-dev
        • rpm:
          • bison
          • flex
          • elfutils-libelf-devel
          • openssl-devel
          • libacl-devel
          • libaio-devel
          • libattr-devel
          • libtool
          • libuuid-devel
          • xfsdump
          • xfsprogs
          • xfsprogs-devel
          • libaio-devel
          • libtool
          • libuuid-devel
          • xfsprogs-devel
      • extra_packages:
        • deb:
          • python3-cephfs
          • cephfs-shell
        • rpm:
          • python3-cephfs
      • sha1: 24317f834a194d1f3573de92f0e624646b6b912e
    • ceph:
    • kclient:
    • cephfs_test_runner:
      • modules:
        • tasks.cephfs.test_forward_scrub
  • teuthology_branch: master
  • verbose: True
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2021-07-28 21:00:49
  • started: 2021-07-29 00:27:27
  • updated: 2021-07-29 00:58:17
  • status_class: success
  • runtime: 0:30:50
  • wait_time: 0:15:57