Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi060.front.sepia.ceph.com smithi True True 2024-03-28 14:01:13.952235 scheduled_pdonnell@teuthology centos 8 x86_64 /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627930
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 7628028 2024-03-28 09:07:11 2024-03-28 09:24:09 2024-03-28 11:58:13 2:34:04 0:33:46 2:00:18 smithi fix-cephadmunit-start centos 9.stream rbd:nvmeof/{base/install centos_latest conf/{disable-pool-app} workloads/nvmeof_thrash} 4
Failure Reason:

"2024-03-28T11:32:54.643875+0000 mon.a (mon.0) 221 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

running 7627930 2024-03-28 07:16:53 2024-03-28 14:01:13 2024-03-28 14:18:06 0:18:26 smithi main centos 8.stream fs/upgrade/nofs/{bluestore-bitmap centos_8.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}} 1
pass 7627896 2024-03-28 07:16:20 2024-03-28 13:30:44 2024-03-28 13:58:25 0:27:41 0:16:45 0:10:56 smithi main centos 9.stream fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} 2
fail 7627797 2024-03-28 07:14:43 2024-03-28 12:28:43 2024-03-28 13:15:39 0:46:56 0:37:45 0:09:11 smithi main centos 9.stream fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/centos_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} 2
Failure Reason:

Test failure: test_reading_conf (tasks.cephfs.test_cephfs_shell.TestShellOpts)

pass 7627775 2024-03-28 06:19:47 2024-03-28 06:40:26 2024-03-28 09:30:49 2:50:23 2:24:16 0:26:07 smithi main centos 9.stream fs:fscrypt:/fscrypt-basic/{begin/{0-install 1-ceph 2-logrotate 3-modules} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{0-client 1-tests/fscrypt-ffsb}} 3
fail 7627763 2024-03-28 05:47:33 2024-03-28 11:58:05 2024-03-28 12:15:17 0:17:12 0:07:47 0:09:25 smithi main centos 9.stream crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/readwrite} 2
Failure Reason:

No module named 'tasks'

fail 7627736 2024-03-28 05:14:02 2024-03-28 05:40:58 2024-03-28 06:29:05 0:48:07 0:23:59 0:24:08 smithi main centos 9.stream rgw:notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/others/{0-install supported-distros/{centos_latest} test_others}} 2
Failure Reason:

Command failed (bucket notification tests against different endpoints) on smithi060 with status 1: "BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a '!kafka_test,!amqp_test,!amqp_ssl_test,!kafka_security_test,!modification_required,!manual_test'"

pass 7627734 2024-03-28 05:13:59 2024-03-28 05:18:45 2024-03-28 05:51:50 0:33:05 0:16:31 0:16:34 smithi main centos 9.stream rgw:notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/amqp/{0-install centos_latest test_amqp}} 2
pass 7627698 2024-03-28 02:28:36 2024-03-28 04:22:58 2024-03-28 05:03:39 0:40:41 0:27:05 0:13:36 smithi main centos 9.stream rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} 4
pass 7627693 2024-03-28 02:28:33 2024-03-28 03:47:32 2024-03-28 04:26:32 0:39:00 0:27:04 0:11:56 smithi main centos 9.stream rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} 4
pass 7627688 2024-03-28 02:28:29 2024-03-28 03:10:05 2024-03-28 03:49:26 0:39:21 0:17:18 0:22:03 smithi main centos 9.stream rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-small} 4
pass 7627685 2024-03-28 02:28:27 2024-03-28 02:41:27 2024-03-28 03:10:02 0:28:35 0:14:10 0:14:25 smithi main centos 9.stream rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} 4
pass 7627662 2024-03-27 22:56:19 2024-03-27 23:13:27 2024-03-28 02:08:24 2:54:57 2:44:51 0:10:06 smithi main centos 9.stream rgw/tools/{centos_latest cluster ignore-pg-availability tasks} 1
pass 7626641 2024-03-27 15:03:26 2024-03-27 22:14:47 2024-03-27 22:49:53 0:35:06 0:24:36 0:10:30 smithi main centos 9.stream rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} 2
pass 7626586 2024-03-27 15:02:43 2024-03-27 21:46:02 2024-03-27 22:14:57 0:28:55 0:23:37 0:05:18 smithi main rhel 8.6 rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/prometheus} 2
fail 7626549 2024-03-27 15:02:13 2024-03-27 21:22:54 2024-03-27 21:44:21 0:21:27 0:10:45 0:10:42 smithi main centos 9.stream rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} 2
Failure Reason:

Command failed on smithi060 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7c37d80f7a88f4eb561c4fe02558660a650b10ac'

dead 7625964 2024-03-27 08:49:23 2024-03-27 09:12:46 2024-03-27 21:24:51 12:12:05 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} 2
Failure Reason:

hit max job timeout

pass 7625916 2024-03-27 08:06:38 2024-03-27 08:38:26 2024-03-27 09:16:43 0:38:17 0:27:08 0:11:09 smithi main centos 9.stream crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} 2
pass 7625702 2024-03-27 05:30:47 2024-03-27 07:58:59 2024-03-27 08:39:52 0:40:53 0:28:39 0:12:14 smithi main centos 8.stream fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} 3
fail 7625660 2024-03-27 05:29:54 2024-03-27 07:17:22 2024-03-27 07:59:16 0:41:54 0:29:45 0:12:09 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} 3
Failure Reason:

Command failed (workunit test suites/fsx.sh) on smithi047 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=155268c4e432a12433aa833f174f9fe3b1016ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh'