Description: orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}

Log: http://qa-proxy.ceph.com/teuthology/adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304532/teuthology.log

Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b0906b8ea83a4d769675fad973f5ddbd

Failure Reason:

{'smithi134.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.376137', 'end': '2023-06-14 20:11:06.118605', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:10:52.742468', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', '<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi032.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:16.493981', 'end': '2023-06-14 20:11:27.165403', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:11:10.671422', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', '<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}

  • log_href: http://qa-proxy.ceph.com/teuthology/adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304532/teuthology.log
  • archive_path: /home/teuthworker/archive/adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi/7304532
  • description: orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final}
  • duration: 0:06:17
  • email: adking@redhat.com
  • failure_reason: {'smithi134.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:13.376137', 'end': '2023-06-14 20:11:06.118605', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:10:52.742468', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}, 'smithi032.front.sepia.ceph.com': {'changed': True, 'msg': 'All items completed', 'results': [{'_ansible_item_label': 'ceph/python3-asyncssh', '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': True, 'cmd': ['dnf', '-y', 'copr', 'enable', 'ceph/python3-asyncssh'], 'delta': '0:00:16.493981', 'end': '2023-06-14 20:11:27.165403', 'failed': True, 'invocation': {'module_args': {'_raw_params': 'dnf -y copr enable ceph/python3-asyncssh', '_uses_shell': False, 'argv': None, 'chdir': None, 'creates': None, 'executable': None, 'removes': None, 'stdin': None, 'stdin_add_newline': True, 'strip_empty_ends': True, 'warn': True}}, 'item': 'ceph/python3-asyncssh', 'msg': 'non-zero return code', 'rc': 1, 'start': '2023-06-14 20:11:10.671422', 'stderr': 'Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.\nError: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known', 'stderr_lines': ['Enabling a Copr repository. Please note that this repository is not part', 'of the main distribution, and quality may vary.', '', 'The Fedora Project does not exercise any power over the contents of', 'this repository beyond the rules outlined in the Copr FAQ at', ',', 'and packages are not held to any quality or security level.', '', 'Please do not file bug reports about these packages in Fedora', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Name or service not known'], 'stdout': '', 'stdout_lines': []}], 'warnings': ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]}}
  • flavor:
  • job_id: 7304532
  • kernel:
    • kdb: True
    • sha1: distro
  • last_in_suite: False
  • machine_type: smithi
  • name: adking-2023-06-14_18:35:42-orch:cephadm-wip-guits-testing-4-2023-06-14-1428-distro-default-smithi
  • nuke_on_error: True
  • os_type: centos
  • os_version: 8.stream
  • overrides:
    • admin_socket:
      • branch: wip-guits-testing-4-2023-06-14-1428
    • ceph:
      • conf:
        • mgr:
          • debug mgr: 20
          • debug ms: 1
        • mon:
          • debug mon: 20
          • debug ms: 1
          • debug paxos: 20
        • osd:
          • debug ms: 1
          • debug osd: 20
          • osd shutdown pgref assert: True
      • flavor: default
      • log-ignorelist:
        • \(MDS_ALL_DOWN\)
        • \(MDS_UP_LESS_THAN_MAX\)
      • sha1: 2a9f88d69c93a40d0581c0676c06a9c9f9708b5c
    • ceph-deploy:
      • conf:
        • client:
          • log file: /var/log/ceph/ceph-$name.$pid.log
        • mon:
          • osd default pool size: 2
    • install:
      • ceph:
        • flavor: default
        • sha1: 2a9f88d69c93a40d0581c0676c06a9c9f9708b5c
    • selinux:
      • whitelist:
        • scontext=system_u:system_r:logrotate_t:s0
    • workunit:
      • branch: wip-guits-testing-4-2023-06-14-1428
      • sha1: 2a9f88d69c93a40d0581c0676c06a9c9f9708b5c
  • owner: scheduled_adking@teuthology
  • pid:
  • roles:
    • ['host.a', 'client.0']
    • ['host.b', 'client.1']
  • sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b0906b8ea83a4d769675fad973f5ddbd
  • status: dead
  • success: False
  • branch: wip-guits-testing-4-2023-06-14-1428
  • seed:
  • sha1: 2a9f88d69c93a40d0581c0676c06a9c9f9708b5c
  • subset:
  • suite:
  • suite_branch: wip-guits-testing-4-2023-06-14-1428
  • suite_path:
  • suite_relpath:
  • suite_repo:
  • suite_sha1: 2a9f88d69c93a40d0581c0676c06a9c9f9708b5c
  • targets:
    • smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDH5W1+gA1i9lZK4wnCaQGIpAEun0M53fQuOR0h5x7Yr1a82dHzRgwcC9O+J8gCL+B3/I+9EQF4LsfRq+5Jqjx2mHhmtx8mabbDhG0MlOkIoiahnqpeSM/Ndl6lAaP3r0Cj2o88l2LZEFY8RTxNLAypKtwjt1IgYTtDHZpdExNVoC35HTVe6CcenXBWLgihzlKZhTXyEMdf9kenmXs+i2vYYZSJmpGkdUKg1ndiXo5hT84ki/HXpd5GaXaYxWlJ69R+tKex4vf1FiTlceEe6bAJu66U3bNk3lbtjDF4+s4QK6OltKQe/Bynk+gYv9CvBzzJ5oXbFYqOHhbNZRD4tlK3okU6MMuHXOIqAIboVnL4VUuqn+kEpBJYRBCsAZPcXPJVm9oXMuaDn+jvMPJBO4VS3faV0qvFT0FupOibdP8DlVdwT8vL2KVcuNYjUo0A2AcIjMW+DELKQP3GHpbmtTMV7Q1KRaeC1wegUPSzAlf2j5D7oK4ZRC5qBYBbJHlW+ks=
    • smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPOJfbZnU8kqINhwToS1DkxVUxyX2IGYI0sIn13/zfgfXVDA09c9zqmAvJFBNY/G+ziRLjXu2DBVtCJFPZtqcks=
  • tasks:
    • internal.check_packages:
    • internal.buildpackages_prep:
    • internal.save_config:
    • internal.check_lock:
    • internal.add_remotes:
    • console_log:
    • internal.connect:
    • internal.push_inventory:
    • internal.serialize_remote_roles:
    • internal.check_conflict:
    • internal.check_ceph_data:
    • internal.vm_setup:
    • kernel:
      • kdb: True
      • sha1: distro
    • internal.base:
    • internal.archive_upload:
    • internal.archive:
    • internal.coredump:
    • internal.sudo:
    • internal.syslog:
    • internal.timer:
    • pcp:
    • selinux:
    • ansible.cephlab:
    • clock:
    • pexec:
      • all:
        • sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup
        • sudo dnf -y module reset container-tools
        • sudo dnf -y module install container-tools --allowerasing --nobest
        • sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf
    • nvme_loop:
    • cephadm:
      • roleless: True
    • cephadm.shell:
      • host.a:
        • ceph orch status
        • ceph orch ps
        • ceph orch ls
        • ceph orch host ls
        • ceph orch device ls
    • vip:
    • cephadm.shell:
      • host.a:
        • ceph orch device ls --refresh
    • vip.exec:
      • all-hosts:
        • systemctl stop nfs-server
    • cephadm.shell:
      • host.a:
        • ceph orch apply rgw foorgw --port 8800
        • ceph nfs cluster create foo --ingress --virtual-ip {{VIP0}}/{{VIPPREFIXLEN}}
    • vip.exec:
      • host.a:
        • dnf install -y python3-boto3 || apt install -y python3-boto3
        • /home/ubuntu/cephtest/cephadm shell radosgw-admin user create --uid foouser --display-name foo > /tmp/user.json
    • python:
      • host.a: import boto3 import json with open('/tmp/user.json', 'rt') as f: info = json.loads(f.read()) s3 = boto3.resource( 's3', aws_access_key_id=info['keys'][0]['access_key'], aws_secret_access_key=info['keys'][0]['secret_key'], endpoint_url='http://localhost:8800', ) bucket = s3.Bucket('foobucket') bucket.create() bucket.put_object(Key='myobject', Body='thebody')
    • cephadm.shell:
      • host.a:
        • ceph nfs export create rgw --cluster-id foo --pseudo-path /foouser --user-id foouser
    • cephadm.wait_for_service:
      • service: nfs.foo
    • cephadm.wait_for_service:
      • service: ingress.nfs.foo
    • vip.exec:
      • host.a:
        • mkdir /mnt/foo
        • sleep 5
        • mount -t nfs {{VIP0}}:/foouser /mnt/foo
        • test -d /mnt/foo/foobucket
        • find /mnt/foo -ls
        • grep thebody /mnt/foo/foobucket/myobject
        • echo test > /mnt/foo/foobucket/newobject
        • sync
    • python:
      • host.a: import boto3 import json from io import BytesIO with open('/tmp/user.json', 'rt') as f: info = json.loads(f.read()) s3 = boto3.resource( 's3', aws_access_key_id=info['keys'][0]['access_key'], aws_secret_access_key=info['keys'][0]['secret_key'], endpoint_url='http://localhost:8800', ) bucket = s3.Bucket('foobucket') data = BytesIO() bucket.download_fileobj(Fileobj=data, Key='newobject') print(data.getvalue()) assert data.getvalue().decode() == 'test\n'
    • vip.exec:
      • host.a:
        • umount /mnt/foo
    • cephadm.shell:
      • host.a:
        • ceph nfs export rm foo /foouser
        • ceph nfs cluster rm foo
    • cephadm.shell:
      • host.a:
        • stat -c '%u %g' /var/log/ceph | grep '167 167'
        • ceph orch status
        • ceph orch ps
        • ceph orch ls
        • ceph orch host ls
        • ceph orch device ls
        • ceph orch ls | grep '^osd.all-available-devices '
  • teuthology_branch: main
  • verbose: False
  • pcp_grafana_url:
  • priority:
  • user:
  • queue:
  • posted: 2023-06-14 18:36:44
  • started: 2023-06-14 19:55:48
  • updated: 2023-06-14 20:14:21
  • status_class: danger
  • runtime: 0:18:33
  • wait_time: 0:12:16