Description: rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli_mon}

Log: http://qa-proxy.ceph.com/teuthology/khiremat-2022-10-18_04:14:28-rados-wip-khiremat-47893-test-mgr-finisher-thread-1-distro-default-smithi/7072000/teuthology.log

Sentry event: https://sentry.ceph.com/organizations/ceph/?query=322d85346ca94795ba3fb2e33a884d4a

Failure Reason:

{'smithi078.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['device-mapper-multipath'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi107.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['device-mapper-multipath'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi037.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi006.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi045.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}}

  • log_href: http://qa-proxy.ceph.com/teuthology/khiremat-2022-10-18_04:14:28-rados-wip-khiremat-47893-test-mgr-finisher-thread-1-distro-default-smithi/7072000/teuthology.log
  • archive_path: /home/teuthworker/archive/khiremat-2022-10-18_04:14:28-rados-wip-khiremat-47893-test-mgr-finisher-thread-1-distro-default-smithi/7072000
  • description: rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli_mon}
  • duration: 0:07:12
  • email:
  • failure_reason: {'smithi078.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['device-mapper-multipath'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-baseos-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi107.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['device-mapper-multipath'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'absent', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi037.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi006.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}, 'smithi045.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': ['epel'], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['dbench'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'rhel-8-for-x86_64-appstream-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", 'rc': 1, 'results': []}}
  • flavor:
  • job_id: 7072000
  • kernel:
    • kdb: True
    • sha1: distro
  • last_in_suite: False
  • machine_type: smithi
  • name: khiremat-2022-10-18_04:14:28-rados-wip-khiremat-47893-test-mgr-finisher-thread-1-distro-default-smithi
  • nuke_on_error: True
  • os_type: rhel
  • os_version: 8.6
  • overrides:
    • admin_socket:
      • branch: wip-khiremat-47893-test-mgr-finisher-thread-1
    • ceph:
      • conf:
        • global:
          • mon election default strategy: 3
        • mgr:
          • debug mgr: 20
          • debug ms: 1
          • mgr/cephadm/use_agent: True
        • mon:
          • debug mon: 20
          • debug ms: 1
          • debug paxos: 20
        • osd:
          • debug ms: 1
          • debug osd: 20
      • flavor: default
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • sha1: 48f5e6530140cfcced72e63bf173a83a864ce2b8
    • ceph-deploy:
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
    • install:
      • ceph:
        • flavor: default
        • sha1: 48f5e6530140cfcced72e63bf173a83a864ce2b8
    • selinux:
      • whitelist:
        • scontext=system_u:system_r:logrotate_t:s0
    • workunit:
      • branch: wip-khiremat-47893-test-mgr-finisher-thread-1
      • sha1: 48f5e6530140cfcced72e63bf173a83a864ce2b8
  • owner: scheduled_khiremat@teuthology
  • pid:
  • roles:
    • ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0']
    • ['host.b', 'osd.3', 'osd.4', 'osd.5', 'mon.b', 'mgr.b', 'client.1']
    • ['host.c', 'osd.6', 'osd.7', 'osd.8', 'mon.c', 'mgr.c', 'client.2']
    • ['host.d', 'osd.9', 'osd.10', 'osd.11', 'mon.d', 'mgr.d', 'client.3']
    • ['host.e', 'osd.12', 'osd.13', 'osd.14', 'mon.e', 'mgr.e', 'client.4']
  • sentry_event: https://sentry.ceph.com/organizations/ceph/?query=322d85346ca94795ba3fb2e33a884d4a
  • status: dead
  • success: False
  • branch: wip-khiremat-47893-test-mgr-finisher-thread-1
  • seed:
  • sha1: 48f5e6530140cfcced72e63bf173a83a864ce2b8
  • subset:
  • suite:
  • suite_branch: wip-khiremat-47893-test-mgr-finisher-thread-1
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 48f5e6530140cfcced72e63bf173a83a864ce2b8
  • targets:
    • smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHe44UYDdTJFppHYT/retij9+GGfldxFtKfw++xrObRvh5HODApSnFGMspSyJygChJAbcXViWS3ZNSR44mmGbwQ=
    • smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2Au/0V6+TgbmLpwxpbl17HGWXcz9O0yCVrf7eB+7MQl9Q1ZGAEmsSqVtnmKtoCC1AAw8TmZqUOt0ROErNJ43zvCNrYwg7LFazrtTkL8FGFmxriX8upi0yMnGJq5H5w+zp4BUZZSwAluoywscxZ/FoalVkhOSTY+FwzZIzXPYjbu9Y4wvXk9bR2IBPh8JgJKnM6K1yU8OIjX8dB9lIaC7dPuALsxpYXE4I18rwN4MZFLAHFSDiSC3nQRdeRGI+p0+FI3MjI2ESaZV1S+Gc4Vg7+AAyerNlZ/fHFxas9z0d+m/DQ0IcJMsQ/f6Z5X4cVIf8GO91ilApLXQl5wgL/a7yCK67MP8Ndai748LJhoSbac1oS0bwX2S+Q0dXSpB0O83vUx3RMv7qqU5FsUfDPAtIvZYC++NavEHzc0HU8P+HH2YftYIaSbZqpe4J5cCuYa4lAkeRIa/OUcEY3C5iO/vogQV5L6GSTJd8nuMAvnvrtB/NjOj26aV4E43YXOEIPcE=
    • smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4W7ZSjcRQ/nwFn0a0fBoeq2FTzfGv7B3sM0Hj3a7bt6m3sRpqJ+7KfzLuSEERpu6Q1Yz+fz3NM1vuGI9F+j2aCn1ofAo3rTKGgdc4I2NhqqmE2G3/CfxkaWTtrqhDutmrzq7XDEZ+NkXLKHMGsgluDNQFvV46NuWrL6dTSIztbZO34JJ0XWoqJH4ARr59/gvUmQ2Q//qnJq7HGBH2OyfmUFdcK+YaNRdWDHaPz4FINw1Pgpr/qZ/hUqAQVSe+tk+f27o+iobeff/fHyc/d5En1Baph0ozVhUR484LtymqKq3leUgIMlDXh09BAzu0SXEfSUO50wl1gy127pVfXlWALAOSFJkdcF97Amtz1APNf3xPkvXLNln3dyTFFAkxzNHfMlDkctjQVJU8Jh3fLN2tpInaTlMW75kKlXTDLfNu2WCyPhgGusp+9VDBVyl1ptAz4yjU5Vo1Q/7AbM2NU8+7Aqd38VFz2KSTfeTesnt6vG0RSrYUUPCdEXY77AKCZrc=
    • smithi078.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW9wdRfvNNsJ10yDeaDWbN8fm+DgBbRqWXP/0k0ZfOcfaCA4YQ2vv6Fn4U2faNazSh4E4MF2CV9qJDwqkAuK5QCNZzN6T1+Pbo53IrYrW/BSfWs+Mo/Fz0oittkbYQDsb4EdjCwiulPAfxaI2wRquZtfRXKhNNhk+NOPE1MeAUcXmZwZHqsUAb/8PNLCXVNIHElylHSQnECwCk+X4zDiL99JHGWZvigFkvOhJA3u78s9CDAsBCrxQ0FAVmWMXf8B57WB/z12BUuYn2wJ3cGit37sAlgW3zEn+l9DeH05ZbE1pZeLDhHI+4AlR0cfOpB0vZEW5lHaFHjsTUVO2GLwcbCsTFbDSlim/iHnqyQUdNbJvF0p4xzBskfYyhqjefgTgjX4Vd/Sbh4tMzo/uJvIUigNn+C7KSxDYBKvv4twI/pDePh24mHZiPqihbiUcaZLVomkHgHCdAN7aC9DgaT8Midh//oGdr9XRmEXAPBeyCyX8ENiQDEwC+a1r/UTYcADs=
    • smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDac7nePeubRHOkYxyfSryZltUI2u4Oy2kvOYxk/gED1ZUyuzdiu3rxHnYLcctYicvNRWuRe95XdIQZDuJhGEMCMyBfthn9u/dT2Ffl83S7wJUwYIWi8idjPYlxOhFF+r+IGBTEoKaxlTgYBab1e9klI60G8epqEySCOxbWNzopggrsqp8WBXVp9AB1aQDBX0FfXvaBFA35DAnugwWjMBfzx4O8EJ2rf5++SaxjinRe//ut7WZ326fVdun+778gvcelC9areKolutkj7U/F+NHKDOpykRTMU9+W2TiVH/rfzMvGGPZIMpaBWtVac+4BGwrkKcQyIZtnrHd7JARLLCGOs5+tHAojXeRDUL9PcQxyf3F42J9ptwe2bP6WCFPdNEoj2PNTCeBSjlWLsxFsr/PwPE58LcBIHOwrD74sULP2tGBWlmZlbYGqL7b+thghbQOLQZ7DUIe0m54I2qKNNtPXYEGjbwp/KCoZQaSka4AdhwRb2XgJx5E5dbNJmtc3B3M=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • kdb: True
      • sha1: distro
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • pexec:
      • all:
        • sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup
        • sudo dnf -y module reset container-tools
        • sudo dnf -y module install container-tools:3.0
        • sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf
    • install:
    • cephadm:
    • cephadm.shell:
      • host.a:
        • ceph orch apply mds a
    • cephfs_test_runner:
      • modules:
        • tasks.cephadm_cases.test_cli_mon
  • teuthology_branch: main
  • verbose: False
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2022-10-18 04:19:43
  • started: 2022-10-19 05:23:01
  • updated: 2022-10-19 05:41:07
  • status_class: danger
  • runtime: 0:18:06
  • wait_time: 0:10:54