Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7652453 2024-04-11 17:05:09 2024-04-11 23:57:41 2024-04-12 00:20:26 0:22:45 0:13:27 0:09:18 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

pass 7652455 2024-04-11 17:05:10 2024-04-11 23:57:41 2024-04-12 01:21:36 1:23:55 1:12:52 0:11:03 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} 4
fail 7652457 2024-04-11 17:05:11 2024-04-12 00:00:42 2024-04-12 00:35:46 0:35:04 0:25:37 0:09:27 smithi main centos 9.stream rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7652459 2024-04-11 17:05:13 2024-04-12 00:04:33 2024-04-12 00:26:13 0:21:40 0:07:20 0:14:20 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} 3
Failure Reason:

Command failed on smithi131 with status 1: 'sudo yum -y install ceph-radosgw'

fail 7652461 2024-04-11 17:05:14 2024-04-12 00:13:25 2024-04-12 00:36:10 0:22:45 0:14:03 0:08:42 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-12T00:33:43.950154+0000 mon.smithi008 (mon.0) 791 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7652463 2024-04-11 17:05:15 2024-04-12 00:17:27 2024-04-12 12:27:57 12:10:30 smithi main ubuntu 22.04 rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

hit max job timeout

fail 7652465 2024-04-11 17:05:16 2024-04-12 00:17:27 2024-04-12 02:13:42 1:56:15 1:44:41 0:11:34 smithi main centos 9.stream rados/upgrade/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-04-12T00:46:31.636012+0000 mon.a (mon.0) 319 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7652467 2024-04-11 17:05:17 2024-04-12 00:18:28 2024-04-12 00:51:38 0:33:10 0:14:54 0:18:16 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-12T00:47:07.554580+0000 mon.a (mon.0) 692 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

pass 7652469 2024-04-11 17:05:18 2024-04-12 00:28:40 2024-04-12 01:00:07 0:31:27 0:21:36 0:09:51 smithi main ubuntu 22.04 rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} 2
fail 7652472 2024-04-11 17:05:20 2024-04-12 00:28:53 2024-04-12 00:30:53 0:02:00 0:00:01 0:01:59 smithi main centos 9.stream rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7652474 2024-04-11 17:05:21 2024-04-12 00:28:52 2024-04-12 01:58:03 1:29:11 1:14:26 0:14:45 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/radosbench} 3
Failure Reason:

"2024-04-12T01:10:00.000098+0000 mon.a (mon.0) 1265 : cluster [WRN] Health detail: HEALTH_WARN noscrub flag(s) set" in cluster log

fail 7652476 2024-04-11 17:05:22 2024-04-12 00:32:54 2024-04-12 01:36:55 1:04:01 0:53:00 0:11:01 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7652477 2024-04-11 17:05:23 2024-04-12 00:33:35 2024-04-12 01:05:54 0:32:19 0:19:21 0:12:58 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-12T00:58:17.734661+0000 mon.smithi083 (mon.0) 563 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

dead 7652478 2024-04-11 17:05:24 2024-04-12 00:36:15 2024-04-12 12:52:29 12:16:14 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-redelete} 4
Failure Reason:

hit max job timeout

fail 7652479 2024-04-11 17:05:25 2024-04-12 00:42:37 2024-04-12 01:20:38 0:38:01 0:27:59 0:10:02 smithi main ubuntu 22.04 rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi113 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7652480 2024-04-11 17:05:26 2024-04-12 00:42:37 2024-04-12 01:07:34 0:24:57 0:14:45 0:10:12 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-12T01:03:41.739559+0000 mon.smithi115 (mon.0) 792 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7652481 2024-04-11 17:05:27 2024-04-12 00:42:38 2024-04-12 01:23:20 0:40:42 0:27:58 0:12:44 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} 3
fail 7652482 2024-04-11 17:05:28 2024-04-12 00:44:18 2024-04-12 01:07:05 0:22:47 0:14:22 0:08:25 smithi main centos 9.stream rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

fail 7652483 2024-04-11 17:05:29 2024-04-12 00:44:18 2024-04-12 01:11:31 0:27:13 0:16:00 0:11:13 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} 3
Failure Reason:

"2024-04-12T01:04:49.009522+0000 mon.a (mon.0) 381 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7652484 2024-04-11 17:05:30 2024-04-12 00:44:19 2024-04-12 01:14:22 0:30:03 0:16:15 0:13:48 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-12T01:05:13.158113+0000 mon.a (mon.0) 447 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7652485 2024-04-11 17:05:31 2024-04-12 00:50:10 2024-04-12 00:52:09 0:01:59 0:00:01 0:01:58 smithi main centos 9.stream rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7652486 2024-04-11 17:05:32 2024-04-12 00:50:08 2024-04-12 01:28:24 0:38:16 0:18:34 0:19:42 smithi main centos 9.stream rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} 1
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7652487 2024-04-11 17:05:33 2024-04-12 00:59:30 2024-04-12 01:43:21 0:43:51 0:33:06 0:10:45 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7652488 2024-04-11 17:05:34 2024-04-12 00:59:30 2024-04-12 01:22:33 0:23:03 0:14:01 0:09:02 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} 1
Failure Reason:

"2024-04-12T01:19:31.142711+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

dead 7652489 2024-04-11 17:05:35 2024-04-12 00:59:41 2024-04-12 01:00:45 0:01:04 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} 3
Failure Reason:

Error reimaging machines: Failed to power on smithi071

fail 7652490 2024-04-11 17:05:36 2024-04-12 00:59:41 2024-04-12 01:23:56 0:24:15 0:13:31 0:10:44 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} 2
Failure Reason:

"2024-04-12T01:21:49.218263+0000 mon.smithi071 (mon.0) 801 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7652491 2024-04-11 17:05:37 2024-04-12 01:00:52 2024-04-12 13:25:45 12:24:53 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects-redelete} 4
Failure Reason:

hit max job timeout

fail 7652492 2024-04-11 17:05:38 2024-04-12 01:15:55 2024-04-12 01:51:43 0:35:48 0:25:33 0:10:15 smithi main ubuntu 22.04 rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} 3
Failure Reason:

"2024-04-12T01:39:58.206716+0000 mon.a (mon.0) 425 : cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log

fail 7652493 2024-04-11 17:05:39 2024-04-12 01:15:56 2024-04-12 01:56:43 0:40:47 0:28:55 0:11:52 smithi main ubuntu 22.04 rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

fail 7652494 2024-04-11 17:05:40 2024-04-12 01:15:56 2024-04-12 01:45:23 0:29:27 0:19:38 0:09:49 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} 2
Failure Reason:

"2024-04-12T01:41:37.526431+0000 mon.smithi017 (mon.0) 757 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

dead 7652495 2024-04-11 17:05:41 2024-04-12 01:15:57 2024-04-12 13:25:50 12:09:53 smithi main centos 9.stream rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

hit max job timeout

fail 7652496 2024-04-11 17:05:42 2024-04-12 01:15:57 2024-04-12 03:13:11 1:57:14 1:47:25 0:09:49 smithi main centos 9.stream rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} 2
Failure Reason:

"2024-04-12T01:44:23.314876+0000 mon.a (mon.0) 254 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

pass 7652497 2024-04-11 17:05:43 2024-04-12 01:15:57 2024-04-12 01:39:17 0:23:20 0:13:59 0:09:21 smithi main centos 9.stream rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} 3
fail 7652498 2024-04-11 17:05:44 2024-04-12 01:16:00 2024-04-12 01:17:59 0:01:59 0:00:01 0:01:58 smithi main ubuntu 22.04 rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7652499 2024-04-11 17:05:45 2024-04-12 01:15:58 2024-04-12 03:15:15 1:59:17 1:48:58 0:10:19 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7652500 2024-04-11 17:05:46 2024-04-12 01:15:58 2024-04-12 01:57:28 0:41:30 0:25:04 0:16:26 smithi main ubuntu 22.04 rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} 2
Failure Reason:

"2024-04-12T01:44:50.361434+0000 mon.a (mon.0) 389 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log

fail 7652501 2024-04-11 17:05:47 2024-04-12 01:21:40 2024-04-12 01:47:27 0:25:47 0:15:44 0:10:03 smithi main ubuntu 22.04 rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} 1
Failure Reason:

Command failed (workunit test mon/mkfs.sh) on smithi107 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mkfs.sh'

fail 7652502 2024-04-11 17:05:48 2024-04-12 01:21:41 2024-04-12 02:18:33 0:56:52 0:43:51 0:13:01 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} 3
Failure Reason:

"2024-04-12T02:00:00.000074+0000 mon.a (mon.0) 1431 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 139/1684 objects degraded (8.254%), 1 pg degraded" in cluster log

fail 7652503 2024-04-11 17:05:49 2024-04-12 01:23:21 2024-04-12 01:48:27 0:25:06 0:13:55 0:11:11 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} 2
Failure Reason:

"2024-04-12T01:44:28.152290+0000 mon.smithi106 (mon.0) 787 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7652504 2024-04-11 17:05:50 2024-04-12 01:23:22 2024-04-12 01:59:57 0:36:35 0:26:28 0:10:07 smithi main centos 9.stream rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} 1
Failure Reason:

Command failed (workunit test cephtool/test.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

dead 7652505 2024-04-11 17:05:51 2024-04-12 01:25:27 2024-04-12 13:37:06 12:11:39 smithi main ubuntu 22.04 rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-redelete} 4
Failure Reason:

hit max job timeout

fail 7652506 2024-04-11 17:05:52 2024-04-12 01:28:18 2024-04-12 03:18:31 1:50:13 1:24:32 0:25:41 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} 3
Failure Reason:

"2024-04-12T02:20:00.000114+0000 mon.a (mon.0) 1098 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull; Degraded data redundancy: 3592/84495 objects degraded (4.251%), 2 pgs degraded" in cluster log

fail 7652507 2024-04-11 17:05:53 2024-04-12 01:30:59 2024-04-12 01:56:05 0:25:06 0:15:48 0:09:18 smithi main centos 9.stream rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/e2e} 2
Failure Reason:

Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

fail 7652508 2024-04-11 17:05:54 2024-04-12 01:31:01 2024-04-12 01:33:00 0:01:59 0:00:01 0:01:58 smithi main ubuntu 22.04 rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}
fail 7652509 2024-04-11 17:05:55 2024-04-12 01:30:59 2024-04-12 01:54:01 0:23:02 0:13:41 0:09:21 smithi main centos 9.stream rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} 2
Failure Reason:

"2024-04-12T01:51:33.218319+0000 mon.smithi092 (mon.0) 796 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

fail 7652510 2024-04-11 17:05:56 2024-04-12 01:31:00 2024-04-12 02:29:58 0:58:58 0:48:36 0:10:22 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} 2
Failure Reason:

valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun

fail 7652511 2024-04-11 17:05:57 2024-04-12 01:31:00 2024-04-12 01:58:16 0:27:16 0:15:57 0:11:19 smithi main centos 9.stream rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} 2
Failure Reason:

"2024-04-12T01:51:12.026656+0000 mon.a (mon.0) 620 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log

fail 7652512 2024-04-11 17:05:57 2024-04-12 01:31:01 2024-04-12 02:02:31 0:31:30 0:16:58 0:14:32 smithi main ubuntu 22.04 rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_cephadm} 1
Failure Reason:

Command failed (workunit test cephadm/test_cephadm.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'

fail 7652513 2024-04-11 17:05:58 2024-04-12 01:36:52 2024-04-12 02:04:46 0:27:54 0:08:27 0:19:27 smithi main centos 8.stream rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/reef backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} 3
Failure Reason:

Command failed on smithi110 with status 1: 'sudo yum -y install ceph-radosgw'

fail 7652514 2024-04-11 17:05:59 2024-04-12 01:39:23 2024-04-12 03:29:08 1:49:45 1:32:26 0:17:19 smithi main centos 9.stream rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} 1
Failure Reason:

Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi017 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a5074d4516d566e9d8b6aec912f26afd099de101 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh'

fail 7652515 2024-04-11 17:06:00 2024-04-12 01:46:25 2024-04-12 02:16:21 0:29:56 0:20:38 0:09:18 smithi main ubuntu 22.04 rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rmdir-reactivate} 2
Failure Reason:

"2024-04-12T02:12:13.753456+0000 mon.smithi123 (mon.0) 771 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log