Pulpito :: Results Dashboard

Posted	Started	Updated	Runtime	Duration	In Waiting	Machine	Teuthology Branch	OS Type	OS Version	Nodes	Status
2020-08-03 22:43:38	2020-08-03 23:17:58	2020-08-04 01:00:00	1:42:02	0:06:22	1:35:40	smithi	master	centos	8.1	6	fail

Nodes: smithi035 smithi195 smithi175 smithi158 smithi053 smithi142

Description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}

Log: http://qa-proxy.ceph.com/teuthology/yuriw-2020-08-03_22:42:21-kcephfs-nautilus-distro-basic-smithi/5285179/teuthology.log

Sentry event: http://sentry.ceph.com/sepia/teuthology/?q=5944bca3050c497e8301325173e8278c

Failure Reason:

Command failed on smithi175 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel'

All Details...

log_href: http://qa-proxy.ceph.com/teuthology/yuriw-2020-08-03_22:42:21-kcephfs-nautilus-distro-basic-smithi/5285179/teuthology.log
archive_path: /home/teuthworker/archive/yuriw-2020-08-03_22:42:21-kcephfs-nautilus-distro-basic-smithi/5285179
description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}
duration: 0:06:22
email: ceph-qa@ceph.io
failure_reason: Command failed on smithi175 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel'
flavor: basic
job_id: 5285179
kernel:
- client:
  - branch: testing
- kdb: True
- sha1: distro
last_in_suite: False
machine_type: smithi
name: yuriw-2020-08-03_22:42:21-kcephfs-nautilus-distro-basic-smithi
nuke_on_error: True
os_type: centos
os_version: 8.1
overrides:
- ceph-deploy:
  - fs: xfs
  - conf:
    - client:
      - log file: /var/log/ceph/ceph-$name.$pid.log
    - mon:
      - osd default pool size: 2
    - osd:
      - mon osd full ratio: 0.9
      - mon osd backfillfull_ratio: 0.85
      - bluestore fsck on mount: True
      - mon osd nearfull ratio: 0.8
      - debug bluestore: 20
      - debug bluefs: 20
      - osd objectstore: bluestore
      - bluestore block size: 96636764160
      - debug rocksdb: 10
      - osd failsafe full ratio: 0.95
  - bluestore: True
- selinux:
  - whitelist:
    - scontext=system_u:system_r:logrotate_t:s0
- workunit:
  - sha1: ab2488773626d40fb1f8345a99f4bbd6712c0fd4
  - branch: master
- ceph:
  - log-whitelist:
    - $MDS_ALL_DOWN$
    - $MDS_UP_LESS_THAN_MAX$
    - overall HEALTH_
    - $FS_DEGRADED$
    - $MDS_FAILED$
    - $MDS_DEGRADED$
    - $FS_WITH_FAILED_MDS$
    - $MDS_DAMAGE$
    - $MDS_ALL_DOWN$
    - $MDS_UP_LESS_THAN_MAX$
    - $FS_INLINE_DATA_DEPRECATED$
    - overall HEALTH_
    - $OSD_DOWN$
    - \(OSD_
    - but it is still running
    - is not responding
    - OSD full dropping all updates
    - OSD near full
    - pausewr flag
    - failsafe engaged, dropping updates
    - failsafe disengaged, no longer dropping
    - is full \(reached quota
    - POOL_FULL
    - POOL_BACKFILLFULL
  - sha1: f23ff7620014d0d1324261eb383e8e25c588bdae
  - fs: xfs
  - conf:
    - global:
      - ms die on skipped message: False
    - mgr:
      - debug ms: 1
      - debug mgr: 20
    - client:
      - debug ms: 1
      - debug client: 20
      - client mount timeout: 600
    - mon:
      - mon osd full ratio: 0.7
      - debug mon: 20
      - debug paxos: 20
      - mon op complaint time: 120
      - mon osd nearfull ratio: 0.6
      - debug ms: 1
      - mon osd backfillfull ratio: 0.6
    - mds:
      - mds bal split bits: 3
      - mds bal split size: 100
      - osd op complaint time: 180
      - debug mds: 20
      - mds bal merge size: 5
      - debug ms: 1
      - mds bal frag: True
      - mds verify scatter: True
      - mds bal fragment size max: 10000
      - mds op complaint time: 180
      - mds debug scatterstat: True
      - mds debug frag: True
    - osd:
      - mon osd full ratio: 0.9
      - debug ms: 20
      - debug journal: 20
      - memstore device bytes: 200000000
      - debug osd: 25
      - debug bluestore: 20
      - debug bluefs: 20
      - osd objectstore: memstore
      - osd mon report interval: 5
      - mon osd backfillfull_ratio: 0.85
      - osd op complaint time: 180
      - bluestore block size: 96636764160
      - osd shutdown pgref assert: True
      - debug filestore: 20
      - debug rocksdb: 10
      - mon osd nearfull ratio: 0.8
      - osd failsafe full ratio: 1.0
      - bluestore fsck on mount: True
  - cephfs_ec_profile:
    - m=2
    - k=2
    - crush-failure-domain=osd
    - disabled
- install:
  - ceph:
    - sha1: f23ff7620014d0d1324261eb383e8e25c588bdae
- admin_socket:
  - branch: nautilus
- thrashosds:
  - bdev_inject_crash_probability: 0.5
  - bdev_inject_crash: 2
owner: scheduled_yuriw@teuthology
pid:
roles:
- ['mon.a', 'mgr.y', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'osd.3']
- ['mon.b', 'mon.c', 'mgr.x', 'mds.c', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
- ['client.0']
- ['client.1']
- ['client.2']
- ['client.3']
sentry_event: http://sentry.ceph.com/sepia/teuthology/?q=5944bca3050c497e8301325173e8278c
status: fail
success: False
branch: nautilus
seed:
sha1: f23ff7620014d0d1324261eb383e8e25c588bdae
subset:
suite:
suite_branch: master
suite_path:
suite_relpath:
suite_repo:
suite_sha1: ab2488773626d40fb1f8345a99f4bbd6712c0fd4
targets:
- smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUSX4PgLoDflj2dlZE+po8pOn2Uz1rUta0f9CzVXsE9LkE9VULpm4sx3uIalJVtap8ALw8VXSG4846hCjf5A/XlJEjz23Yfj9Ho//HpZUjZAJjLXCorL9gUCkXRN05znS92AIKGtPpntLSvMelr3fDqJqm9yo8yTTtm2TMFdnuhJc2YRoZ5Y2P6IP4MUfz+cFXtZqkWf3WSMfsis94bun0Wsfn8k4Cw1TCL2pxE53iQDJJg5LuwnABnOTsjeBlfy6Naa23obAFqvNJUTeksh39JEYHGzwGcxjFw4IUavaCoGJcYmU27UFsuonjKuIGDHFutTpSqM946NSBpr1tRv0U+prFMWZmF4dVK60ZCisRrzuRu/V/wVRUzUGs1tl6pxBAW98wF+3lXwDcHvmA1dtzIxbbl1Aw0sb931adDeb1A+OyRxS9eVgRLLOsAr1AHbRtCsJOoI6tPEuGg2Mpb4wA8TLhnCa8oXQAnz+Nlja2wXUjR/9MVIXPrupF7yuy0MU=
- smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/aVP+alxgWegF3q85waKQxIeFSMVxYSe9AGiduhkYkZP0a/szM3pwHaxPbW/XqPSMsaezZYbdIfPX4tCOBzh7uPejYlWaCg6kBNAjuozn7UcHkZSXI/H9k8YCSUS+3jTOQQFAhupeA+M142f8OQiJ0HpnawxtLb34JHu1sR8jx8ry9hLOrvcg6xiL+UED9Oq0OX44fDaULhWD1bgWVhqBdAsuBa9JIVaZ8tVVflVy6qssr8ah1PbRWdoFcWbYrm2mrjI23Z4r6rZ9+11NFRijWNZwRx1bzldx/D5snvZXQdmoNLGOasYRewoZKALGdX2gJAgwlEFrl7uPOgse5rpDJz+s1aR/aKEHxp8xu0Nie6xz4zLzvMgmytL3gSFwCs8I3jM1aQ4nnqNkSk4gPd6Ei4MgTblA74/+1unhIdzj+bY7jugmzkPleZYW5o1Ypn4yUiVvvq4YHdQaypwfmWEg7K9QmA9WvHlt6E6eq0+CPWLuAogngy3ykcjUPzR7XYE=
- smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDivtiquOeff2ubxE9wIMyyXVh9Tz2fDAvf3s/y0pWFquPHcDylamf23Mhqomf3exmg2GcoUn2DNVXHdO/H3xFI2dsJK3ZU2CYUhEA8/6p3JpRG1909sc/IF5hYlIbhrLjKVAtGnAXtuegOEC+oTgG8NOoBzn6BBDfAySzHnEgMNChRTXMH2dAb6JD/ZbS9KUrRUsw/7JT64WeKyvUyo/dQufD+yre/9z5RDzftMiaFfqwsl0MYczkyhrg+pz23MC4YHibxzjUAHCsF4GQgH5jf/7r4/o9q9k4F5Rj29TJk/Fp5Aa1PNWfJ67g8ocS3GRMMMG9q0I1fFDIFcL2x/KcYWXLA0/IWbbytTSrNa35ZD7axDOHJu3tZCmVBS6bB9AtD5UOkvRRgVTzPTudO8gaZofJOCRXLSQdizAtyvupN6ZcE25HsjTs6++o52Kk6+pPSoK+KklH7Py1IS+ajXfaI+jl9l8Ra+b+yj+s+3Q8dYi02JA2gdFF2ZmoDBRQHHzs=
- smithi158.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv8fioE7GAZY7685ITaGWj4xfJJa0zN0LrYSJBH9EcluEHJPIZ1h3I946G1zEEusylQSzmmy4mwixI9Ii5o+OuT9VqOpWUPLgW9UYG2FSGh04VrbytF078dy1gq74fXCqbsYJiMDAGw5rCY7lc0qLJAbBOZHlqMarwtWeIpCcPFDObGN7PA3pbn7iw2l7mBFJ60mpYqWYgyGjUgo06vOHHifg1Mu+wddcn9Uo0lnrtOWogVtQxeJ3ZipNaumT5TFe9Aw2Or0qlY+bfuuMcGFNRJNx/V2avgNoAlAV+PC4B6Uy+gyzTDIsHX8SVXJ83N9GfIpoBhEDaKYIp/3j5Pon55czF0MVzr2FbTlbhinQ5KNzJ43rjJutRqXspA/gzW9T5QRzPAv0wji6Rj/BJcrFmWNf98bg2bnfi8YND4LYfAs1FzoHguBHu9CGlyvTwqnENIJD/v8Ux7P7WztiXQ4KZBZDKKzPkf9MMjhQdpzG3NDyXR7RqVuBSSrLkxchPqUE=
- smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRffIMYNESxFx23pqEkHOATke1OPV1XqMM3oNC0VwLMVFitt2UeL/iXkxgtQ1fnY+Aa46mOVY0GPClA460B3jj7g0dwXuv+X6NbwRAF9987KHYdHDRDt+o19q9Aoqvfqy/osRZ5Cj7mAxIPtpw3/kIZXNHQg+ZnD/LBf1SLKLyHU2d9J9hCOSNkaOHTlICpe+V1Ia0Gwer9sci251tLHLpBF+XiInxmirwRqkZeHQLFd9FUmBgWvFPW8bEBEsgqgvpBAdcucqOgZWF4sE/T2MPKOfAOVV7wJRVv/7yzHOIYFBGS8kzYV3WmjeUucJuj2a8ROQx56hGM2g8VNDpoXMcMUIyMUMeom18MCPJ8GoWbR0LcUZU5R1tWR4RFlKvk0xCBTO5zAk9PfBMs4nK0Kg4wMJEXXZsKcBwlyp1OUE8DDOOQg+rthNCj18Sxi2W96DKcZNdLuBqeew0/HJfT54YPgqPPpsCYN15GRlIP53vx8Q9f3Qof+2orKDyLFvFDE8=
- smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhwTmHTXblbjTKzpmN6TkBeBWFhTCxI9s8MWQmjUMuoG/lLLRiRm1vrupj40RmIQrXnJx76aEpaUyEMOKN1+B54E63MQvPmIpIPpdMFDKJ1dgbXe8t7kBod2l/towr4sA5OULjODoMTYdxUgLtsvcYoejbCATPNWljxcEySuAEce27MEZtsXGH0a9ku+HwS6EFbm0pe9z1WIoAODJXNJATMB5mJbq5nwWu4Mmt9EpfJBehdWjDIQVCEOwAjDguIRLlpqrJwOTjKQ9eA27q84idObb/NCVHVWtl7xYcmzI5Iw49EZBZebzcoSqX4Th6rRViTllRllJbs1AF2xUXxHOmnWpsjDNCEair+br2ESU3IXmUk/s9xNDhCiE9diTyVtFXN3d6AtVjcdT688vBcvkmMwF7zki9QfGBmnWmeWEjB4v3zdays48rmcT/uUuAxt60oWwrJg9V36iAxhzhZsugckDv7AsoKD9ywYdlC+7pD7yTxJD4HbQIQ1u3BQxhmv8=
tasks:
- internal.check_packages:
- internal.buildpackages_prep:
- internal.lock_machines:
  - 6
  - smithi
- internal.save_config:
- internal.check_lock:
- internal.add_remotes:
- console_log:
- internal.connect:
- internal.push_inventory:
- internal.serialize_remote_roles:
- internal.check_conflict:
- internal.check_ceph_data:
- internal.vm_setup:
- kernel:
  - client:
    - branch: testing
  - kdb: True
  - sha1: distro
- internal.base:
- internal.archive_upload:
- internal.archive:
- internal.coredump:
- internal.sudo:
- internal.syslog:
- internal.timer:
- pcp:
- selinux:
- ansible.cephlab:
- clock:
- install:
  - extra_system_packages:
    - deb:
      - bison
      - flex
      - libelf-dev
      - libssl-dev
      - dump
      - indent
    - rpm:
      - bison
      - flex
      - elfutils-libelf-devel
      - openssl-devel
      - libacl-devel
      - libaio-devel
      - libattr-devel
      - libtool
      - libuuid-devel
      - xfsdump
      - xfsprogs
      - xfsprogs-devel
  - extra_packages:
    - deb:
      - python3-cephfs
      - cephfs-shell
    - rpm:
      - python3-cephfs
  - sha1: f23ff7620014d0d1324261eb383e8e25c588bdae
- ceph:
- kclient:
- cephfs_test_runner:
  - modules:
    - tasks.cephfs.test_full
teuthology_branch: master
verbose: True
pcp_grafana_url:
priority:
user:
queue:
posted: 2020-08-03 22:43:38
started: 2020-08-03 23:17:58
updated: 2020-08-04 01:00:00
status_class: danger
runtime: 1:42:02
wait_time: 1:35:40