Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7654114 2024-04-12 15:16:35 2024-04-12 15:21:45 2024-04-12 15:57:46 0:36:01 0:24:51 0:11:10 smithi main ubuntu 22.04 rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-12T15:45:10.434833+0000 mon.a (mon.0) 388 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

dead 7654115 2024-04-12 15:16:36 2024-04-12 15:21:45 2024-04-13 03:32:14 12:10:29 smithi main ubuntu 22.04 rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 3
Failure Reason:

hit max job timeout

fail 7654116 2024-04-12 15:16:37 2024-04-12 15:22:06 2024-04-12 15:46:23 0:24:17 0:14:02 0:10:15 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-12T15:43:32.097274+0000 mon.smithi069 (mon.0) 792 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7654117 2024-04-12 15:16:38 2024-04-12 15:22:48 2024-04-12 15:24:47 0:01:59 0 smithi main centos 9.stream rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7654118 2024-04-12 15:16:39 2024-04-12 15:22:46 2024-04-12 17:18:31 1:55:45 1:44:51 0:10:54 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654119 2024-04-12 15:16:40 2024-04-12 15:23:07 2024-04-12 17:05:59 1:42:52 1:34:30 0:08:22 smithi main centos 9.stream rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} 1
Failure Reason:

Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi047 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh'

fail 7654120 2024-04-12 15:16:41 2024-04-12 15:23:07 2024-04-12 15:46:15 0:23:08 0:11:43 0:11:25 smithi main ubuntu 22.04 rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_ca_signed_key} 2
Failure Reason:

Command failed on smithi007 with status 5: 'sudo systemctl stop ceph-552f93c0-f8e3-11ee-bc8c-c7b262605968@mon.a'

dead 7654121 2024-04-12 15:16:42 2024-04-12 15:24:38 2024-04-13 03:35:29 12:10:51 smithi main ubuntu 22.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} 4
Failure Reason:

hit max job timeout

pass 7654122 2024-04-12 15:16:43 2024-04-12 15:25:38 2024-04-12 16:16:20 0:50:42 0:39:50 0:10:52 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} 4
fail 7654123 2024-04-12 15:16:44 2024-04-12 15:27:09 2024-04-12 15:50:09 0:23:00 0:13:18 0:09:42 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

pass 7654124 2024-04-12 15:16:45 2024-04-12 15:27:09 2024-04-12 16:14:29 0:47:20 0:34:58 0:12:22 smithi main ubuntu 22.04 rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} 1
pass 7654125 2024-04-12 15:16:46 2024-04-12 15:27:10 2024-04-12 16:53:54 1:26:44 1:15:00 0:11:44 smithi main centos 9.stream rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench} 4
fail 7654126 2024-04-12 15:16:47 2024-04-12 15:28:30 2024-04-12 16:06:30 0:38:00 0:28:46 0:09:14 smithi main ubuntu 22.04 rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi191 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7654127 2024-04-12 15:16:48 2024-04-12 15:28:51 2024-04-12 16:06:26 0:37:35 0:14:54 0:22:41 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} 3
Failure Reason:

Command failed on smithi190 with status 1: 'sudo yum -y install ceph-radosgw'

fail 7654128 2024-04-12 15:16:49 2024-04-12 15:33:22 2024-04-12 15:59:40 0:26:18 0:14:12 0:12:06 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-12T15:55:44.955849+0000 mon.smithi169 (mon.0) 788 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7654129 2024-04-12 15:16:50 2024-04-12 15:35:49 2024-04-12 15:58:11 0:22:22 0:12:51 0:09:31 smithi main ubuntu 22.04 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/crash} 2
dead 7654130 2024-04-12 15:16:51 2024-04-12 15:35:50 2024-04-13 03:44:51 12:09:01 smithi main centos 9.stream rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

hit max job timeout

fail 7654131 2024-04-12 15:16:53 2024-04-12 15:35:50 2024-04-12 17:33:02 1:57:12 1:48:24 0:08:48 smithi main centos 9.stream rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-04-12T16:05:30.463879+0000 mon.a (mon.0) 352 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7654133 2024-04-12 15:16:54 2024-04-12 15:35:51 2024-04-12 16:02:00 0:26:09 0:14:30 0:11:39 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-12T15:56:06.800729+0000 mon.a (mon.0) 662 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7654134 2024-04-12 15:16:55 2024-04-12 15:36:51 2024-04-12 16:23:39 0:46:48 0:33:56 0:12:52 smithi main centos 9.stream rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/one workloads/snaps-few-objects} 2
pass 7654136 2024-04-12 15:16:56 2024-04-12 15:38:34 2024-04-12 15:59:27 0:20:53 0:11:58 0:08:55 smithi main centos 9.stream rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} 1
fail 7654138 2024-04-12 15:16:57 2024-04-12 15:38:34 2024-04-12 17:35:54 1:57:20 1:38:10 0:19:10 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/radosbench} 3
Failure Reason:

"2024-04-12T16:30:00.004676+0000 mon.a (mon.0) 1134 : cluster [WRN] Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set" in cluster log

fail 7654140 2024-04-12 15:16:58 2024-04-12 15:40:05 2024-04-12 16:39:36 0:59:31 0:48:14 0:11:17 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654141 2024-04-12 15:16:59 2024-04-12 15:40:15 2024-04-12 16:10:55 0:30:40 0:19:01 0:11:39 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-12T16:06:36.638559+0000 mon.smithi082 (mon.0) 776 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7654143 2024-04-12 15:17:00 2024-04-12 15:41:16 2024-04-13 03:54:06 12:12:50 smithi main centos 9.stream rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} 4
Failure Reason:

hit max job timeout

fail 7654145 2024-04-12 15:17:01 2024-04-12 15:42:36 2024-04-12 17:12:06 1:29:30 1:20:30 0:09:00 smithi main ubuntu 22.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} 1
Failure Reason:

Command failed (workunit test erasure-code/test-erasure-eio.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh'

fail 7654146 2024-04-12 15:17:02 2024-04-12 15:42:37 2024-04-12 16:20:16 0:37:39 0:28:45 0:08:54 smithi main ubuntu 22.04 rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi187 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7654147 2024-04-12 15:17:03 2024-04-12 15:42:37 2024-04-12 16:07:41 0:25:04 0:14:56 0:10:08 smithi main centos 9.stream rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} tasks/e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi073 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

pass 7654148 2024-04-12 15:17:04 2024-04-12 15:42:47 2024-04-12 16:21:11 0:38:24 0:25:58 0:12:26 smithi main centos 9.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} 3
fail 7654149 2024-04-12 15:17:05 2024-04-12 15:43:48 2024-04-12 16:12:12 0:28:24 0:17:15 0:11:09 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-12T16:06:49.704773+0000 mon.a (mon.0) 379 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7654150 2024-04-12 15:17:06 2024-04-12 15:45:08 2024-04-12 16:10:08 0:25:00 0:10:09 0:14:51 smithi main ubuntu 22.04 rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats'

fail 7654151 2024-04-12 15:17:08 2024-04-12 15:50:30 2024-04-12 16:21:39 0:31:09 0:15:18 0:15:51 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-12T16:14:03.449649+0000 mon.a (mon.0) 411 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7654152 2024-04-12 15:17:09 2024-04-12 15:57:25 2024-04-12 15:59:25 0:02:00 0 smithi main centos 9.stream rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7654153 2024-04-12 15:17:10 2024-04-12 15:57:24 2024-04-12 16:26:28 0:29:04 0:18:49 0:10:15 smithi main centos 9.stream rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654154 2024-04-12 15:17:11 2024-04-12 15:57:34 2024-04-12 16:46:52 0:49:18 0:39:21 0:09:57 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654155 2024-04-12 15:17:12 2024-04-12 15:58:04 2024-04-12 16:20:49 0:22:45 0:14:08 0:08:37 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-04-12T16:17:26.963947+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7654156 2024-04-12 15:17:13 2024-04-12 15:58:05 2024-04-12 16:23:56 0:25:51 0:15:08 0:10:43 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps} 4
fail 7654157 2024-04-12 15:17:14 2024-04-12 15:58:05 2024-04-12 16:33:48 0:35:43 0:25:39 0:10:04 smithi main ubuntu 22.04 rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-12T16:22:57.447061+0000 mon.a (mon.0) 542 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

pass 7654158 2024-04-12 15:17:15 2024-04-12 15:58:06 2024-04-12 16:38:08 0:40:02 0:29:07 0:10:55 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} 4
fail 7654159 2024-04-12 15:17:16 2024-04-12 15:58:16 2024-04-12 16:33:09 0:34:53 0:26:36 0:08:17 smithi main centos 9.stream rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi069 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7654160 2024-04-12 15:17:17 2024-04-12 15:58:16 2024-04-12 16:30:57 0:32:41 0:20:14 0:12:27 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-12T16:26:15.181968+0000 mon.smithi072 (mon.0) 794 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7654161 2024-04-12 15:17:18 2024-04-12 15:58:17 2024-04-13 04:08:55 12:10:38 smithi main centos 9.stream rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} 3
Failure Reason:

hit max job timeout

dead 7654162 2024-04-12 15:17:19 2024-04-12 15:58:17 2024-04-13 04:09:11 12:10:54 smithi main ubuntu 22.04 rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

hit max job timeout

fail 7654163 2024-04-12 15:17:20 2024-04-12 15:58:17 2024-04-12 17:58:20 2:00:03 1:48:40 0:11:23 smithi main centos 9.stream rados/upgrade/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-04-12T16:20:00.000140+0000 mon.a (mon.0) 518 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded" in cluster log

fail 7654164 2024-04-12 15:17:21 2024-04-12 15:59:28 2024-04-12 16:41:46 0:42:18 0:17:08 0:25:10 smithi main ubuntu 22.04 rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 4
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0'

fail 7654165 2024-04-12 15:17:22 2024-04-12 16:13:42 2024-04-12 16:15:41 0:01:59 0:00:01 0:01:58 smithi main ubuntu 22.04 rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
pass 7654166 2024-04-12 15:17:24 2024-04-12 16:13:40 2024-04-12 16:37:49 0:24:09 0:14:10 0:09:59 smithi main ubuntu 22.04 rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} 4
fail 7654167 2024-04-12 15:17:25 2024-04-12 16:13:41 2024-04-12 18:06:45 1:53:04 1:44:11 0:08:53 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654168 2024-04-12 15:17:26 2024-04-12 16:13:41 2024-04-12 16:49:23 0:35:42 0:25:22 0:10:20 smithi main ubuntu 22.04 rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-12T16:35:59.913984+0000 mon.a (mon.0) 392 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7654169 2024-04-12 15:17:27 2024-04-12 16:13:41 2024-04-12 16:38:23 0:24:42 0:15:34 0:09:08 smithi main centos 9.stream rados/standalone/{supported-random-distro$/{centos_latest} workloads/mon} 1
Failure Reason:

Command failed (workunit test mon/mkfs.sh) on smithi190 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mkfs.sh'

pass 7654170 2024-04-12 15:17:28 2024-04-12 16:13:42 2024-04-12 16:36:30 0:22:48 0:12:40 0:10:08 smithi main centos 9.stream rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream} 1-start 2-services/rgw 3-final} 1
fail 7654171 2024-04-12 15:17:29 2024-04-12 16:13:42 2024-04-12 17:16:03 1:02:21 0:43:01 0:19:20 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} 3
Failure Reason:

"2024-04-12T17:00:00.000126+0000 mon.a (mon.0) 1010 : cluster [WRN] Health detail: HEALTH_WARN noscrub flag(s) set" in cluster log

pass 7654172 2024-04-12 15:17:30 2024-04-12 16:13:52 2024-04-12 16:50:03 0:36:11 0:22:51 0:13:20 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} 4
pass 7654173 2024-04-12 15:17:31 2024-04-12 16:14:33 2024-04-12 16:43:32 0:28:59 0:16:50 0:12:09 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} 3
fail 7654174 2024-04-12 15:17:32 2024-04-12 16:16:24 2024-04-12 16:46:33 0:30:09 0:14:38 0:15:31 smithi main centos 9.stream rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} 4
Failure Reason:

Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0'

fail 7654175 2024-04-12 15:17:33 2024-04-12 16:29:00 2024-04-12 16:51:56 0:22:56 0:14:14 0:08:42 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-12T16:49:43.981374+0000 mon.smithi170 (mon.0) 795 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7654176 2024-04-12 15:17:34 2024-04-12 16:29:00 2024-04-12 17:04:09 0:35:09 0:26:09 0:09:00 smithi main centos 9.stream rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi130 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7654177 2024-04-12 15:17:35 2024-04-12 16:29:11 2024-04-12 18:24:28 1:55:17 1:37:49 0:17:28 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} 3
Failure Reason:

"2024-04-12T17:20:00.000095+0000 mon.a (mon.0) 1268 : cluster [WRN] Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; Degraded data redundancy: 98/118215 objects degraded (0.083%), 2 pgs degraded, 2 pgs undersized" in cluster log

fail 7654178 2024-04-12 15:17:36 2024-04-12 16:29:11 2024-04-12 16:54:26 0:25:15 0:15:39 0:09:36 smithi main centos 9.stream rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi151 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c5618cc018927fa77780d25b6deb1f5dc254d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

dead 7654179 2024-04-12 15:17:37 2024-04-12 16:29:11 2024-04-13 04:47:16 12:18:05 smithi main ubuntu 22.04 rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} 4
Failure Reason:

hit max job timeout

fail 7654180 2024-04-12 15:17:38 2024-04-12 16:37:55 2024-04-12 16:39:54 0:01:59 0 smithi main ubuntu 22.04 rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7654181 2024-04-12 15:17:40 2024-04-12 16:37:53 2024-04-12 17:01:00 0:23:07 0:13:33 0:09:34 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-12T16:57:58.892966+0000 mon.smithi084 (mon.0) 778 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7654182 2024-04-12 15:17:41 2024-04-12 16:37:54 2024-04-12 17:43:22 1:05:28 0:55:53 0:09:35 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7654183 2024-04-12 15:17:42 2024-04-12 16:38:14 2024-04-12 16:55:26 0:17:12 0:07:47 0:09:25 smithi main centos 9.stream rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats'

fail 7654184 2024-04-12 15:17:43 2024-04-12 16:38:14 2024-04-12 17:03:11 0:24:57 0:15:20 0:09:37 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-12T16:55:10.923464+0000 mon.a (mon.0) 420 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log