Name Machine Type Up Locked Locked Since Locked By OS Type OS Version Arch Description
smithi137.front.sepia.ceph.com smithi True True 2024-04-24 21:53:59.419231 scheduled_cbodley@teuthology centos 9 x86_64 /home/teuthworker/archive/cbodley-2024-04-24_20:45:10-rgw-wip-cbodley-testing-distro-default-smithi/7672123
Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
running 7672123 2024-04-24 20:45:16 2024-04-24 21:28:35 2024-04-24 22:19:10 0:51:56 smithi main centos 9.stream rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/kafka/{0-install supported-distros/{centos_latest} test_kafka}} 2
fail 7671847 2024-04-24 15:50:47 2024-04-24 17:08:04 2024-04-24 17:55:39 0:47:35 0:37:11 0:10:24 smithi main ubuntu 22.04 orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_22.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} 2
Failure Reason:

"2024-04-24T17:38:50.032070+0000 mon.a (mon.0) 874 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED)" in cluster log

fail 7671812 2024-04-24 15:50:12 2024-04-24 16:45:07 2024-04-24 20:16:09 3:31:02 0:24:48 3:06:14 smithi main centos 9.stream orch:cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-services/client-keyring 3-final} 2
Failure Reason:

Command failed on smithi008 with status 125: "sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5904d29475f5be602879d9fb26280e89b808d5cc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88659ff6-0274-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi204:172.21.15.204=smithi204;smithi008:172.21.15.8=smithi008'"

pass 7671738 2024-04-24 15:14:36 2024-04-24 19:12:21 2024-04-24 19:41:51 0:29:30 0:15:28 0:14:02 smithi main centos 8.stream krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf ms_mode/crc$/{crc-rxbounce} msgr-failures/many tasks/rbd_workunit_suites_fsstress} 3
pass 7671693 2024-04-24 14:46:12 2024-04-24 14:47:37 2024-04-24 15:08:50 0:21:13 0:14:33 0:06:40 smithi main centos 9.stream rgw:notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/amqp/{0-install centos_latest test_amqp}} 2
fail 7671690 2024-04-24 14:20:17 2024-04-24 14:21:31 2024-04-24 14:49:00 0:27:29 0:19:46 0:07:43 smithi main centos 9.stream fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} 3
Failure Reason:

Command failed on smithi131 with status 2: "sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c37a646-0247-11ef-bc93-c7b262605968 -- bash -c 'ls /home/ubuntu'"

pass 7671652 2024-04-24 14:13:06 2024-04-24 17:54:28 2024-04-24 19:14:04 1:19:36 1:13:06 0:06:30 smithi main rhel 8.6 rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-zstd 4-supported-random-distro$/{rhel_8} 5-pool/replicated-data-pool 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup conf/{disable-pool-app}} 3
pass 7671563 2024-04-24 14:11:29 2024-04-24 15:08:18 2024-04-24 16:46:10 1:37:52 1:27:30 0:10:22 smithi main centos 8.stream rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/filestore-xfs 4-supported-random-distro$/{centos_8} 5-pool/ec-data-pool 6-prepare/qcow2-http 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup conf/{disable-pool-app}} 3
pass 7671486 2024-04-24 13:01:09 2024-04-24 13:57:34 2024-04-24 14:21:48 0:24:14 0:14:56 0:09:18 smithi main ubuntu 22.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest} 2
pass 7671462 2024-04-24 13:00:46 2024-04-24 13:31:44 2024-04-24 13:57:58 0:26:14 0:15:19 0:10:55 smithi main ubuntu 22.04 rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest} 2
pass 7671424 2024-04-24 13:00:13 2024-04-24 13:03:46 2024-04-24 13:32:05 0:28:19 0:18:44 0:09:35 smithi main ubuntu 22.04 rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default ignore-pg-availability overrides s3a-hadoop supported-random-distro$/{ubuntu_latest}} 2
fail 7671259 2024-04-24 09:26:34 2024-04-24 09:52:29 2024-04-24 10:33:46 0:41:17 0:33:24 0:07:53 smithi main centos 9.stream rbd/iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} 3
Failure Reason:

"2024-04-24T10:07:42.412256+0000 mon.a (mon.0) 207 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log

fail 7671223 2024-04-24 07:33:20 2024-04-24 08:40:52 2024-04-24 09:40:31 0:59:39 0:46:19 0:13:20 smithi main centos 9.stream fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} 2
Failure Reason:

reached maximum tries (51) after waiting for 300 seconds

dead 7671188 2024-04-24 06:59:44 2024-04-24 07:02:30 2024-04-24 08:43:24 1:40:54 smithi main centos 9.stream rbd:nvmeof/{base/install centos_latest conf/{disable-pool-app} workloads/nvmeof_thrash} 4
fail 7671165 2024-04-24 05:23:00 2024-04-24 06:09:56 2024-04-24 07:00:06 0:50:10 0:35:18 0:14:52 smithi main centos 9.stream fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} 3
Failure Reason:

Command failed (workunit test suites/fsx.sh) on smithi005 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a3c8a22bd5efe18ab230a8e2d072db8f381a0411 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh'

pass 7671074 2024-04-24 01:17:33 2024-04-24 19:41:55 2024-04-24 21:53:57 2:12:02 2:05:27 0:06:35 smithi main rhel 8.6 upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
pass 7671040 2024-04-24 01:16:57 2024-04-24 10:39:14 2024-04-24 13:04:20 2:25:06 2:13:23 0:11:43 smithi main centos 8.stream upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
pass 7670929 2024-04-24 01:09:40 2024-04-24 03:42:22 2024-04-24 06:13:49 2:31:27 2:17:59 0:13:28 smithi main ubuntu 20.04 upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} 2
pass 7670650 2024-04-23 21:41:29 2024-04-24 02:07:28 2024-04-24 03:45:41 1:38:13 1:28:33 0:09:40 smithi main ubuntu 22.04 rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} 2
pass 7670602 2024-04-23 21:40:40 2024-04-24 01:24:29 2024-04-24 02:08:09 0:43:40 0:36:08 0:07:32 smithi main centos 9.stream rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} 2