Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi165.front.sepia.ceph.com smithi True True 2023-01-30 12:22:01.793830 scheduled_vshankar@teuthology centos 8 x86_64 /home/teuthworker/archive/vshankar-2023-01-30_10:51:51-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi/7143825
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7143825 2023-01-30 10:52:28 2023-01-30 12:21:51 2023-01-30 12:54:59 0:33:08 0:21:44 0:11:24 smithi main centos 8.stream fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} 3
Failure Reason:

"2023-01-30T12:44:19.891748+0000 mgr.y (mgr.14100) 1 : cluster [ERR] Failed to load ceph-mgr modules: prometheus" in cluster log

fail 7143802 2023-01-30 10:52:07 2023-01-30 11:22:24 2023-01-30 12:21:58 0:59:34 0:47:17 0:12:17 smithi main rhel 8.6 fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}} 3
Failure Reason:

SELinux denials found on ubuntu@smithi138.front.sepia.ceph.com: ['type=AVC msg=audit(1675078939.178:14788): avc: denied { open } for pid=67597 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079080.488:17099): avc: denied { lock } for pid=78188 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079080.488:17098): avc: denied { open } for pid=78188 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675078938.472:14771): avc: denied { search } for pid=1392 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=524293 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1675079080.488:17098): avc: denied { read } for pid=78188 comm="rpm" name="Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079081.681:17143): avc: denied { read } for pid=78357 comm="rpm" name="Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675078939.178:14790): avc: denied { map } for pid=67597 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262275 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079081.681:17143): avc: denied { open } for pid=78357 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079081.681:17144): avc: denied { lock } for pid=78357 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=524330 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079080.488:17100): avc: denied { map } for pid=78188 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=524340 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675078939.178:14789): avc: denied { lock } for pid=67597 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675078939.178:14788): avc: denied { read } for pid=67597 comm="rpm" name="Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079079.922:17073): avc: denied { read } for pid=705 comm="sssd" name="resolv.conf" dev="sda1" ino=524376 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675079081.681:17145): avc: denied { map } for pid=78357 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=524340 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1']

fail 7143781 2023-01-30 09:56:56 2023-01-30 10:45:19 2023-01-30 11:24:45 0:39:26 0:30:13 0:09:13 smithi main centos 8.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} 2
Failure Reason:

SELinux denials found on ubuntu@smithi165.front.sepia.ceph.com: ['type=AVC msg=audit(1675077482.107:23003): avc: denied { search } for pid=6265 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=393230 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1675076415.005:17022): avc: denied { open } for pid=94186 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076042.878:214): avc: denied { lock } for pid=6379 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077792.241:24769): avc: denied { map } for pid=143066 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=525270 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076042.879:215): avc: denied { map } for pid=6379 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077483.792:23012): avc: denied { read } for pid=131704 comm="rpm" name="Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076289.357:14037): avc: denied { map } for pid=77402 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077483.792:23012): avc: denied { open } for pid=131704 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077483.792:23013): avc: denied { lock } for pid=131704 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077791.858:24734): avc: denied { search } for pid=6265 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=393230 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1675076042.862:213): avc: denied { read } for pid=6379 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076042.862:213): avc: denied { open } for pid=6379 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076415.005:17023): avc: denied { lock } for pid=94186 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077792.241:24768): avc: denied { lock } for pid=143066 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076415.005:17024): avc: denied { map } for pid=94186 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=525270 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077792.241:24767): avc: denied { open } for pid=143066 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675077792.241:24767): avc: denied { read } for pid=143066 comm="rpm" name="Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076289.357:14036): avc: denied { lock } for pid=77402 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076289.357:14035): avc: denied { read } for pid=77402 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076288.997:14034): avc: denied { search } for pid=6265 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=393230 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1675077483.792:23014): avc: denied { map } for pid=131704 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=525270 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076289.357:14035): avc: denied { open } for pid=77402 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076415.005:17022): avc: denied { read } for pid=94186 comm="rpm" name="Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1675076414.654:16991): avc: denied { read } for pid=699 comm="sssd" name="resolv.conf" dev="sda1" ino=525304 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1']

pass 7143750 2023-01-30 09:49:58 2023-01-30 09:51:22 2023-01-30 10:45:32 0:54:10 0:41:18 0:12:52 smithi main ubuntu 20.04 rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} 2
fail 7143726 2023-01-29 17:42:12 2023-01-29 17:50:34 2023-01-29 18:46:52 0:56:18 0:47:53 0:08:25 smithi main rhel 8.6 rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/rados_api_tests} 2
Failure Reason:

Command failed (workunit test rados/test.sh) on smithi131 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=220fca54cc51e0cfb264bdaf9adf6fbd2cc76de6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

fail 7143693 2023-01-29 11:32:45 2023-01-29 12:07:19 2023-01-29 12:43:24 0:36:05 0:28:01 0:08:04 smithi main rhel 8.6 rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} 2
Failure Reason:

SELinux denials found on ubuntu@smithi006.front.sepia.ceph.com: ['type=AVC msg=audit(1674994416.636:418): avc: denied { lock } for pid=2479 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994712.742:15017): avc: denied { read } for pid=68652 comm="rpm" name="Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994416.635:417): avc: denied { read } for pid=2479 comm="rpm" name="Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994814.626:17311): avc: denied { read } for pid=79171 comm="rpm" name="Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994416.635:417): avc: denied { open } for pid=2479 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994416.636:419): avc: denied { map } for pid=2479 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262275 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994712.742:15017): avc: denied { open } for pid=68652 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994814.626:17312): avc: denied { lock } for pid=79171 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994814.626:17313): avc: denied { map } for pid=79171 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262362 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994814.254:17280): avc: denied { read } for pid=793 comm="sssd" name="resolv.conf" dev="sda1" ino=393234 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994712.370:14984): avc: denied { search } for pid=2377 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=262307 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1674994712.742:15018): avc: denied { lock } for pid=68652 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994814.626:17311): avc: denied { open } for pid=79171 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674994712.742:15019): avc: denied { map } for pid=68652 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262275 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1']

pass 7143653 2023-01-29 11:32:04 2023-01-29 11:36:43 2023-01-29 12:08:11 0:31:28 0:24:27 0:07:01 smithi main rhel 8.6 rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps} 2
pass 7143637 2023-01-29 10:45:28 2023-01-29 11:08:17 2023-01-29 11:37:11 0:28:54 0:17:25 0:11:29 smithi main ubuntu 20.04 rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} 2
fail 7143459 2023-01-29 05:22:19 2023-01-30 09:32:29 2023-01-30 09:53:32 0:21:03 0:07:04 0:13:59 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_netns} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ca74598065096e6fcbd8433c8779a2be0c889351

fail 7143443 2023-01-29 05:22:00 2023-01-30 09:18:12 2023-01-30 09:35:54 0:17:42 0:07:01 0:10:41 smithi master krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/secure msgr-failures/few tasks/rbd_workunit_trivial_sync} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ca74598065096e6fcbd8433c8779a2be0c889351

fail 7143426 2023-01-29 05:21:41 2023-01-30 08:54:32 2023-01-30 09:18:19 0:23:47 0:07:00 0:16:47 smithi master krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc msgr-failures/few tasks/krbd_udev_netlink_enobufs} 3
Failure Reason:

Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ca74598065096e6fcbd8433c8779a2be0c889351

fail 7143406 2023-01-29 05:01:18 2023-01-29 05:31:36 2023-01-29 06:05:23 0:33:47 0:23:21 0:10:26 smithi main centos 8.stream smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rgw_s3tests}} 3
Failure Reason:

SELinux denials found on ubuntu@smithi102.front.sepia.ceph.com: ['type=AVC msg=audit(1674971221.217:16994): avc: denied { read } for pid=94020 comm="rpm" name="Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971221.217:16994): avc: denied { open } for pid=94020 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971096.709:14039): avc: denied { lock } for pid=77411 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971220.868:16963): avc: denied { read } for pid=724 comm="sssd" name="resolv.conf" dev="sda1" ino=525302 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674970844.728:216): avc: denied { map } for pid=6369 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971097.486:14072): avc: denied { open } for pid=77563 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971096.708:14038): avc: denied { open } for pid=77411 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971096.709:14040): avc: denied { map } for pid=77411 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971097.486:14074): avc: denied { map } for pid=77563 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971097.486:14073): avc: denied { lock } for pid=77563 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674970844.716:214): avc: denied { open } for pid=6369 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674970844.725:215): avc: denied { lock } for pid=6369 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971095.466:14035): avc: denied { search } for pid=6256 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=393230 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1674971096.708:14038): avc: denied { read } for pid=77411 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971221.217:16996): avc: denied { map } for pid=94020 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=525270 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971221.217:16995): avc: denied { lock } for pid=94020 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525269 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674970844.716:214): avc: denied { read } for pid=6369 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674971097.486:14072): avc: denied { read } for pid=77563 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1']

fail 7143387 2023-01-29 05:00:57 2023-01-29 05:08:27 2023-01-29 05:32:00 0:23:33 0:17:11 0:06:22 smithi main rhel 8.6 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_direct_io}} 3
Failure Reason:

SELinux denials found on ubuntu@smithi102.front.sepia.ceph.com: ['type=AVC msg=audit(1674969549.820:15029): avc: denied { lock } for pid=68742 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969549.464:14997): avc: denied { search } for pid=2429 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=262170 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1674969549.820:15028): avc: denied { read } for pid=68742 comm="rpm" name="Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969650.751:17285): avc: denied { read } for pid=848 comm="sssd" name="resolv.conf" dev="sda1" ino=393234 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969549.820:15030): avc: denied { map } for pid=68742 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262275 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969651.106:17316): avc: denied { open } for pid=79205 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969549.820:15028): avc: denied { open } for pid=68742 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969651.106:17317): avc: denied { lock } for pid=79205 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969651.107:17318): avc: denied { map } for pid=79205 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262362 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674969651.106:17316): avc: denied { read } for pid=79205 comm="rpm" name="Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1']

pass 7143312 2023-01-28 14:24:11 2023-01-29 23:46:11 2023-01-30 03:26:10 3:39:59 2:11:33 1:28:26 smithi main ubuntu 18.04 upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/filestore-xfs ubuntu_18.04} 4
pass 7143303 2023-01-28 14:24:00 2023-01-29 23:32:45 2023-01-30 01:04:17 1:31:32 1:20:25 0:11:07 smithi main ubuntu 18.04 upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} 5
pass 7143281 2023-01-28 08:08:12 2023-01-28 08:21:00 2023-01-28 08:59:20 0:38:20 0:28:11 0:10:09 smithi main ubuntu 20.04 smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_api_tests}} 3
fail 7143259 2023-01-28 07:22:50 2023-01-28 07:51:58 2023-01-28 08:20:49 0:28:51 0:23:28 0:05:23 smithi main rhel 8.6 rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} 2
Failure Reason:

SELinux denials found on ubuntu@smithi165.front.sepia.ceph.com: ['type=AVC msg=audit(1674892907.661:14999): avc: denied { search } for pid=2381 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=262320 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1674893009.572:17334): avc: denied { map } for pid=79321 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262362 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674892908.007:15020): avc: denied { read } for pid=68736 comm="rpm" name="Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674892908.007:15021): avc: denied { lock } for pid=68736 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674893009.572:17332): avc: denied { read } for pid=79321 comm="rpm" name="Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674892908.007:15022): avc: denied { map } for pid=68736 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=262275 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674892908.007:15020): avc: denied { open } for pid=68736 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262274 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674893009.572:17332): avc: denied { open } for pid=79321 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674893009.572:17333): avc: denied { lock } for pid=79321 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=262356 scontext=system_u:system_r:setroubleshootd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674893009.213:17301): avc: denied { read } for pid=855 comm="sssd" name="resolv.conf" dev="sda1" ino=393236 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1']

fail 7143236 2023-01-28 07:22:18 2023-01-28 07:25:07 2023-01-28 07:52:03 0:26:56 0:15:39 0:11:17 smithi main centos 8.stream rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} 2
Failure Reason:

SELinux denials found on ubuntu@smithi165.front.sepia.ceph.com: ['type=AVC msg=audit(1674891571.856:14037): avc: denied { open } for pid=77409 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891321.839:213): avc: denied { lock } for pid=6363 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891321.824:212): avc: denied { read } for pid=6363 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891696.683:16994): avc: denied { map } for pid=94021 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=525267 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891570.404:14034): avc: denied { search } for pid=6250 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=525252 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1', 'type=AVC msg=audit(1674891696.329:16961): avc: denied { read } for pid=704 comm="sssd" name="resolv.conf" dev="sda1" ino=525297 scontext=system_u:system_r:sssd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891321.824:212): avc: denied { open } for pid=6363 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891696.683:16992): avc: denied { read } for pid=94021 comm="rpm" name="Packages" dev="sda1" ino=525266 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891696.683:16992): avc: denied { open } for pid=94021 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525266 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891571.856:14038): avc: denied { lock } for pid=77409 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891571.856:14037): avc: denied { read } for pid=77409 comm="rpm" name="Packages" dev="sda1" ino=98534 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891321.840:214): avc: denied { map } for pid=6363 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891696.683:16993): avc: denied { lock } for pid=94021 comm="rpm" path="/var/lib/rpm/Packages" dev="sda1" ino=525266 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1674891571.856:14039): avc: denied { map } for pid=77409 comm="rpm" path="/var/lib/rpm/Name" dev="sda1" ino=98535 scontext=system_u:system_r:setroubleshootd_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=1']

pass 7143206 2023-01-28 03:58:04 2023-01-30 08:31:24 2023-01-30 08:55:21 0:23:57 0:13:13 0:10:44 smithi main ubuntu 20.04 perf-basic/{ceph objectstore/bluestore settings/optimized ubuntu_latest workloads/client_endpoint_rbd_4K_rand_write} 1
dead 7142506 2023-01-27 19:15:17 2023-01-28 14:20:43 2023-01-29 02:34:18 12:13:35 smithi main centos 8.stream rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-comp-zlib pool/ec-cache-pool supported-random-distro$/{centos_8} workloads/qemu_bonnie} 3
Failure Reason:

hit max job timeout