Pulpito

teuthology-2020-09-17_10:25:02-kcephfs-master-distro-basic-smithi/5443610

Posted Started Updated
Runtime
Duration
In Waiting
 Machine Teuthology Branch OS Type OS Version Nodes Status
2020-09-17 10:25:55 2020-09-17 10:37:24 2020-09-17 11:07:24 0:30:00 0:05:36 0:24:24 smithi master centos 8.1 6 fail

Description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery}

Log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-17_10:25:02-kcephfs-master-distro-basic-smithi/5443610/teuthology.log

Sentry event: https://sentry.ceph.com/sentry/sepia/?query=c75ab8a073c44386bf7b551418cc0c41

Failure Reason:

Command failed on smithi023 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'

All Details...

  • log_href: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-17_10:25:02-kcephfs-master-distro-basic-smithi/5443610/teuthology.log
  • archive_path: /home/teuthworker/archive/teuthology-2020-09-17_10:25:02-kcephfs-master-distro-basic-smithi/5443610
  • description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery}
  • duration: 0:05:36
  • email: ceph-qa@ceph.io
  • failure_reason: Command failed on smithi023 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'
  • flavor: basic
  • job_id: 5443610
  • kernel:
    • client:
      • branch: testing
    • kdb: True
    • sha1: distro
  • last_in_suite: False
  • machine_type: smithi
  • name: teuthology-2020-09-17_10:25:02-kcephfs-master-distro-basic-smithi
  • nuke_on_error: True
  • os_type: centos
  • os_version: 8.1
  • overrides:
    • ceph-deploy:
      • fs: xfs
      • filestore: True
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
        • osd:
          • osd sloppy crc: True
          • osd objectstore: filestore
    • selinux:
      • whitelist:
        • scontext=system_u:system_r:logrotate_t:s0
    • workunit:
      • sha1: 499d0c7f2b021f292dffb4210e78d3b2db47094d
      • branch: master
    • ceph:
      • log-whitelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • sha1: 36d67cf59e365a1802bd6cbd0c4cc744d7183ec8
      • fs: xfs
      • conf:
        • global:
          • ms die on skipped message: False
        • mgr:
          • debug ms: 1
          • debug mgr: 20
        • client:
          • debug ms: 1
          • debug client: 20
          • client mount timeout: 600
        • mon:
          • debug paxos: 20
          • debug mon: 20
          • debug ms: 1
          • mon op complaint time: 120
        • mds:
          • mds bal split bits: 3
          • mds bal split size: 100
          • osd op complaint time: 180
          • debug mds: 20
          • mds bal merge size: 5
          • debug ms: 1
          • mds bal frag: True
          • mds verify scatter: True
          • mds bal fragment size max: 10000
          • mds op complaint time: 180
          • mds debug scatterstat: True
          • mds debug frag: True
        • osd:
          • debug ms: 20
          • debug journal: 20
          • debug osd: 25
          • osd objectstore: filestore
          • osd sloppy crc: True
          • debug filestore: 20
          • osd shutdown pgref assert: True
          • osd op complaint time: 180
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • overall HEALTH_
        • \(FS_DEGRADED\)
        • \(MDS_FAILED\)
        • \(MDS_DEGRADED\)
        • \(FS_WITH_FAILED_MDS\)
        • \(MDS_DAMAGE\)
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
        • \(FS_INLINE_DATA_DEPRECATED\)
        • overall HEALTH_
        • \(OSD_DOWN\)
        • \(OSD_
        • but it is still running
        • is not responding
        • but it is still running
        • slow request
        • evicting unresponsive client
    • install:
      • ceph:
        • sha1: 36d67cf59e365a1802bd6cbd0c4cc744d7183ec8
    • admin_socket:
      • branch: master
  • owner: scheduled_teuthology@teuthology
  • pid:
  • roles:
    • ['mon.a', 'mgr.y', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
    • ['mon.b', 'mon.c', 'mgr.x', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
    • ['client.0']
    • ['client.1']
    • ['client.2']
    • ['client.3']
  • sentry_event: https://sentry.ceph.com/sentry/sepia/?query=c75ab8a073c44386bf7b551418cc0c41
  • status: fail
  • success: False
  • branch: master
  • seed:
  • sha1: 36d67cf59e365a1802bd6cbd0c4cc744d7183ec8
  • subset:
  • suite:
  • suite_branch: master
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 499d0c7f2b021f292dffb4210e78d3b2db47094d
  • targets:
    • smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkPx1OfJ8RU4bfSgqRvu2cjI0BqbyhkddwjwKS86bMjLqkgSnihYi6wbAPgizxMSr0Yk6zjKfQk/povXwZ5U1MoV9rl0mpawKTw/6hXqG9/yn4zhP/jWTsgmPIC1b93pXW3L9FBTIH8vgKsj6ZllFUS6s8TVo0pA7+MSMHsNIv9zxggIKCEK41eR9m7W5HMgcpFCzsEaSon6tRT0n8yML9a90eyTQPkbwVcPscRdu3D4ccNXecBPT1X49MCZjwMGZuzA9SQofkTa2GEbEO4bE6lKnyL9NNQ1wI85IIo8xHr6CV4GqYZvDF1M/CeKs8kqGcugjQ1ClqNfc7ac+LBxtiq3EYmR+y8Pvl+BzrIAYL+mPePsknDk73Ynk27F5LFZp7HmyK3XGbiB+X2mZcPkK6gNCPqRvjq+culg75Ay/fxEqxrGgflI7Nj2WWEe1h37lzs2Y5rp2najAiiW+21rj/Z7FzktyCO4OTqK72UqnulTvUZUswSEIrGiSLJv6Ui4k=
    • smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHWEWFXXkp/HPhSrv3w8sb4AP6xq/E6loiAomVGo7wNODpJw+NH4QeD4ZrJIwvOhOFW7834tL6yUaDKQFIWxPj0Nhj0UfVj5yHUut/cZd2Z9aU6XluXUs8d75TJ3/PoFKJZ775Aj/Ou8uD6JENwZhSvHTeY0FMyo8PGGUXJg2wcB/9bTkBfJshXh9C9faSXobU2NEaSKizyil1oy4tbS18Sr0Qa8DJoMmfHVi0f1Hm0PHdftJFCpikZv+HNrS/U/RXLj6KgTO+823CIP1iC8pvvPvPPBQkdIFafimswF3N2fbzVlgAOJkuShV5QQyLq3B+QYTWfCagnBtnMF6ZJ+kqpiuAKiVd0eN68xiqnNwzyKNjToxF3rD0aHViONqkOZ6MKmSMwAsfcTxxM6sDCYFSiX54XrOqDT3wfpkDZPrP90Khoc8O7W7U2d2xzj0tUA8yBolTdJUnXdmxFDc75NLFJAGPJfil2oJXsqlSi4LPcIjv59suAE80f5xQexmuBEs=
    • smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfejked6YajhgkIttddWgPy1ma2Htx728nVXBsdYt4UyiEqUMbAgn/t0Rl8KY8i7DEZ/JQd2Xnnudl42slyevRBBy9pII2TSoQsWtKYn6TpViswUU7xOYQrjSLHAqjwOWQaV6/W18/TD3+/KydLB1BjrK+dTxhpW0PYJptWSbUptERqedikXsQ+GHTM44iwlXv1bXQHdKeS7hQWlk3Zrkr0l3jnT/RO0PuF414WGD4ONsLAHbI9b/qY0IAcCbdB/OnvQVliiwdnGAfm3UGHVHsSxHDYGqZtj245ShR8m+g8HvPOusfz2mp0ATHv3sh9oxyyNKp94dzgaHcHTr3850EISk8dFZokL7iBNYf4V6MS8uJ0t27T6WHjvODT//wCWi66EYlBHpGVU3CemsvKOIEaFVLbSo4K2boFnTUMu33RB954Oe59XZJfLoeTOWmxmHuGq8snKIV7e3FliDZ8028ZpMz6ljDAVmT6ruBfx9qrF8Em9WyMAGXi3NEX9ICEfs=
    • smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCoqCMKfmFT23k0Tpr1QQF6+PqWm0u8jd5X8arYTdRYDUDwZRjJ/9CXha48U287yVqKu9+c+iKKnugfvx+NMUcvu0T2t5MHNY0FZ7lMAYdM68RBBWS6XjYpfQ3juvoLdaAwJcRag7DUmwPuKcmagMh1+GnSSJ/uRfsip6KA4oq8vCQlrpZsNd6SELeowpTOXRiyB/mtV0Lcty5qkr3/xPC1f0uUBjakqfNRL0JTbIY+WfqRJ1jUffKTl5ZNjLa333mEG46TaoEww2iP5HdTmYmO6sNofyPc6Q/uf5mSl6uGJ5/m6UvU332Tu9HSgMFEeaZzsgV2PXDvd0UGLeDDbwTdJEb7F7817eH17rbQwZSLJYW2HL43wPu37OMImor/qtH0kXDlv9Sc5O/k2fJpsYchOPxjUPA7Tez/+wJhXrjCGkLA6Pbh7TPLbM8H5udx8g1S4t1DMU20v7Wh1g6eYmBYZYlrBd10fHHnS9i1OdHtnEwfP8CNYTj7Z+PJXZnQo8=
    • smithi123.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNTrTxja0DdLWczk/dyQ2J61UO/sl1wTx7dZFPY+WJ8tsgfWpCFsKKA6nRko2CZ7m9EGd6i4+DXMuR2Xw/Qhs2zJvijPnMfN/QpnFUB5Nq6ai35nkQ4Aod4ykwKzodn0iRrC/rd/3HfU2lbe3uwqyMNkiUpM01kLiFgVSVkNpz9SjMOIe1f9HOz3z/ud7gXNwlXtaviSMWu15xfs1Dkma11thZBVgyRbynUEIMHOydF88UpLzhx4dtWqIpfyfbf4LhpD+ofHgLUw5JZUwu5/vnh0oew6oH3Yb5lZi01hVfj0LQRV9JbYztThwe2k6gBruD29mzTUqUQAL2nlJFWt3AIUOHe6HoU1UMFjp/+F8Ob1MBcb06xQONIUwJv3Zr9fJ0FiQ0iY41EqTGWIL2uiKKwIvnwSZeV+pLsGt0XwsmtCwpY6xxRbmgbDTzBBObtfXNrbIEoSc6dYDaBPXFNTPbvzAYPwFA+KlGFlB4Gapmd0RYDV8MCQJAcidNNyLhfZk=
    • smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU6X6W22N0NCoHrF8eqCY7UHVUe7b/+MkDf/zxXiY4deWz7cBbR+aslhPqjXaBzpgyMJJp5fPVLRvIHa47E6PrusooVBBuhudUx1As4BeaDn0g/SCLWGtbg1oMS1Ui20l67Hx1Ah/2p2pjaNrSNNxDMUGPTimmwu6DGWiSZUqvmgaA/+u2jNGuHuA2hofFn+r2nCRiAlSe8YfKnUBQb/rBpIQvSZI+sNH27HtZWg5qoSvvvS55w/m+unqq/oNkzW6Bi1IctfeDrI7XQiTjOXDbsE7GsgamORD4B4d3rt4l3lVMFNsbynQmafOnLMjCwy07NxhStB+CvYtaye0bulGuXTVlD3tG0iLRHh190NVSKIh6mDKXJ2sJ+fxhwxrWEftTAHmjbN9t4d14KGB5rsu6VgkBHGTlyQaqXiUde9veCVC0JxKW3XQCCM3/Qr3Ka6uAq/4ULalS1j6AxvWj1CO+lRhMEa+q5gxPJFcMTMKCFr4U4MrSpunEfoVeU31Maes=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.lock_machines:
      • 6
      • smithi
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • client:
        • branch: testing
      • kdb: True
      • sha1: distro
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • install:
      • extra_system_packages:
        • deb:
          • bison
          • flex
          • libelf-dev
          • libssl-dev
          • network-manager
          • iproute2
          • util-linux
          • dump
          • indent
          • libaio-dev
          • libtool-bin
          • uuid-dev
          • xfslibs-dev
        • rpm:
          • bison
          • flex
          • elfutils-libelf-devel
          • openssl-devel
          • NetworkManager
          • iproute
          • util-linux
          • libacl-devel
          • libaio-devel
          • libattr-devel
          • libtool
          • libuuid-devel
          • xfsdump
          • xfsprogs
          • xfsprogs-devel
          • libaio-devel
          • libtool
          • libuuid-devel
          • xfsprogs-devel
      • extra_packages:
        • deb:
          • python3-cephfs
          • cephfs-shell
        • rpm:
          • python3-cephfs
      • sha1: 36d67cf59e365a1802bd6cbd0c4cc744d7183ec8
    • ceph:
    • kclient:
    • cephfs_test_runner:
      • modules:
        • tasks.cephfs.test_client_recovery
      • fail_on_skip: False
  • teuthology_branch: master
  • verbose: True
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2020-09-17 10:25:55
  • started: 2020-09-17 10:37:24
  • updated: 2020-09-17 11:07:24
  • status_class: danger
  • runtime: 0:30:00
  • wait_time: 0:24:24