pdonnell-2021-02-24_05:54:31-fs-wip-pdonnell-testing-20210224.040859-distro-basic-gibba

5911348

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba011

fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073}

Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_ino_release_cb'

5911367

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba022 gibba024 gibba042

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc} wsync/{yes}}

"2021-02-24T07:56:09.293387+0000 mds.f (mds.2) 1168 : cluster [WRN] Scrub error on inode 0x10000000265 (/client.0/tmp/testdir/dir1/dir2/dir3) see mds.f log and `damage ls` output for details" in cluster log

5911384

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba023 gibba029

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair}

Test failure: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair)

5911386

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba020 gibba031

fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}

"2021-02-24T09:25:21.813987+0000 mds.c (mds.0) 1 : cluster [WRN] evicting unresponsive client gibba031:1 (4433), after 1066.65 seconds" in cluster log

5911389

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba033 gibba038

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace}

Test failure: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace)

5911392

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba007 gibba017 gibba018

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}}

"2021-02-24T09:29:39.281034+0000 mon.a (mon.0) 346 : cluster [WRN] Health check failed: Degraded data redundancy: 28/200 objects degraded (14.000%), 2 pgs degraded (PG_DEGRADED)" in cluster log

5911401

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba004 gibba008 gibba021

fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}

Command failed on gibba004 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

5911409

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba014 gibba016

fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}}

Test failure: test_subvolume_snapshot_clone_with_upgrade (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)

5911414

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba029 gibba039

fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}}

Command failed (workunit test suites/pjd.sh) on gibba029 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2426538adff197d2896080787a966758aaf9b31d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh'

5911438

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba033 gibba039

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}

Test failure: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub)

5911444

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba015 gibba037 gibba045

fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}}

Command failed on gibba015 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

5911447

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba036 gibba039 gibba043

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc} wsync/{no}}

SSH connection to gibba036 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911455

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba006 gibba028 gibba032

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}}

"2021-02-24T12:04:51.864128+0000 mds.e (mds.0) 19 : cluster [WRN] Scrub error on inode 0x100000001fe (/client.0/tmp/blogbench-1.0/src) see mds.e log and `damage ls` output for details" in cluster log

5911463

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

centos 8.2

gibba003 gibba024

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush}

Test failure: test_flush (tasks.cephfs.test_flush.TestFlush)

5911466

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba007 gibba031 gibba033

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}}

"2021-02-24T12:43:08.017833+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000001e8b (/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel) see mds.f log and `damage ls` output for details" in cluster log

5911470

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba015 gibba037 gibba045

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}}

SSH connection to gibba015 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911479

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba007 gibba031 gibba033

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd} wsync/{no}}

SSH connection to gibba007 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911485

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba016 gibba020

fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth}

Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth)

5911487

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba002 gibba030 gibba042

fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}}

Command failed on gibba002 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

5911491

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba036 gibba039 gibba043

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc} wsync/{no}}

"2021-02-24T13:39:05.814714+0000 mds.c (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000000262 (/client.0/tmp/testdir/dir1) see mds.c log and `damage ls` output for details" in cluster log

5911495

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba016 gibba041 gibba044

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}}

SSH connection to gibba016 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911503

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba003 gibba017 gibba024

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}}

"2021-02-24T14:03:13.951075+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000348 (/client.0/tmp/tmp) see mds.f log and `damage ls` output for details" in cluster log

5911507

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba018 gibba026 gibba041

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}}

"2021-02-24T14:19:21.330851+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x100000012e0 (/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite) see mds.f log and `damage ls` output for details" in cluster log

5911510

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba013 gibba014

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub}

Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks)

5911518

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba036 gibba039 gibba043

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen} wsync/{no}}

SSH connection to gibba036 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911523

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba006 gibba009 gibba038

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}}

"2021-02-24T14:57:33.757266+0000 mds.f (mds.0) 29 : cluster [WRN] Scrub error on inode 0x1000000045f (/client.0/tmp/tmp/fstest_042da8672029728d2311eef8b66b0561/fstest_a14ca18ecbc078ab4c71ecc57cf55e6b) see mds.f log and `damage ls` output for details" in cluster log

5911529

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba016 gibba024 gibba044

fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}

Command failed on gibba016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

5911532

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 20.04

gibba015 gibba045

fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots}

Test failure: test_snapclient_cache (tasks.cephfs.test_snapshots.TestSnapshots), test_snapclient_cache (tasks.cephfs.test_snapshots.TestSnapshots)

5911535

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba003 gibba017 gibba026

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}}

SSH connection to gibba003 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911538

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

centos 8.2

gibba036 gibba039 gibba043

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}}

"2021-02-24T15:34:00.128808+0000 mds.c (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000002159 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-43) see mds.c log and `damage ls` output for details" in cluster log

5911542

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 18.04

gibba007 gibba031 gibba033

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}}

"2021-02-24T15:57:47.646017+0000 mds.d (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000000348 (/client.0/tmp/tmp) see mds.d log and `damage ls` output for details" in cluster log

5911553

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba002 gibba030

fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}}

Command failed (workunit test suites/pjd.sh) on gibba002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2426538adff197d2896080787a966758aaf9b31d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh'

5911558

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba003 gibba017 gibba019

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}}

SSH connection to gibba003 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911570

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

 

gibba012 gibba022 gibba041

fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}}

Command failed on gibba012 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json'

5911572

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba006 gibba023 gibba037

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io} wsync/{no}}

SSH connection to gibba006 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'

5911574

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

centos 8.2

gibba015 gibba018 gibba028

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc} wsync/{yes}}

"2021-02-24T17:07:44.110284+0000 mds.d (mds.2) 4 : cluster [WRN] Scrub error on inode 0x10000000266 (/client.0/tmp/testdir/dir1/dir2/dir3/dir4) see mds.d log and `damage ls` output for details" in cluster log

5911576

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

centos 8.2

gibba017 gibba019

fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}}

"2021-02-24T17:13:48.675965+0000 mon.a (mon.0) 431 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log

5911578

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

ubuntu 18.04

gibba002 gibba003 gibba030

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench} wsync/{yes}}

"2021-02-24T17:14:50.318574+0000 mds.f (mds.0) 24 : cluster [WRN] Scrub error on inode 0x100000001fa (/client.0/tmp/blogbench-1.0/man) see mds.f log and `damage ls` output for details" in cluster log

5911592

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

centos 8.2

gibba007 gibba031 gibba033

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress} wsync/{no}}

"2021-02-24T17:44:09.655273+0000 mds.c (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000001409 (/client.0/tmp/fsstress/ltp-full-20091231/testcases/open_posix_testsuite/conformance/interfaces) see mds.c log and `damage ls` output for details" in cluster log

5911604

FAIL

wip-pdonnell-testing-20210224.040859

wip-pdonnell-testing-20210224.040859

master

gibba

rhel 8.3

gibba002 gibba003 gibba030

fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone} wsync/{no}}

SSH connection to gibba002 was lost: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes,nowsync'