sage-2019-04-17_18:12:05-rados-wip-sage-testing-2019-04-16-1726-distro-basic-smithi

3858628

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi101

rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

"2019-04-17 22:32:19.360818 mon.a (mon.0) 190 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858630

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi181

rados/singleton/{all/lost-unfound-delete.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

"2019-04-17 22:34:00.569547 mon.a (mon.0) 130 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858631

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi001 smithi203

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi203 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858632

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi036 smithi055

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml}

Found coredumps on ubuntu@smithi055.front.sepia.ceph.com

3858633

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi135 smithi182

rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/repair_test.yaml}

"2019-04-17 22:39:29.538309 mon.b (mon.0) 225 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858635

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi171 smithi202

rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/sync-many.yaml workloads/pool-create-delete.yaml}

"2019-04-17 22:25:44.377323 mon.b (mon.0) 174 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858639

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi064 smithi075 smithi166 smithi170

rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/hammer.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml}

Command failed on smithi170 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'

3858642

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi173 smithi186

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/ssh_orchestrator.yaml}

Test failure: test_host_ls (tasks.mgr.test_ssh_orchestrator.TestOrchestratorCli)

3858644

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi073 smithi087 smithi124

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml}

Command failed on smithi073 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858645

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi104 smithi133

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/set-chunks-read.yaml}

"2019-04-17 22:42:17.296268 mon.a (mon.0) 330 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858647

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

 

smithi049 smithi070

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml}

Command failed (workunit test rados/test.sh) on smithi049 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

3858648

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi041 smithi201

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml}

Command failed on smithi041 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858649

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi031

rados/singleton/{all/max-pg-per-osd.from-primary.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

Command failed on smithi031 with status 6: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats'

3858651

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi160 smithi184

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/small-objects.yaml}

Command failed on smithi184 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858653

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi099

rados/singleton/{all/max-pg-per-osd.from-replica.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}}

"2019-04-17 22:43:50.707676 mon.a (mon.0) 397 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858655

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi090 smithi146

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml}

Command failed on smithi146 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858656

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi032 smithi067 smithi136 smithi163

rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}

Found coredumps on ubuntu@smithi136.front.sepia.ceph.com

3858657

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi125 smithi169

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml}

Command failed on smithi125 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858658

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi006 smithi105 smithi158 smithi194

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/jewel-v1only.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/few.yaml rados.yaml thrashers/none.yaml thrashosds-health.yaml workloads/rbd_cls.yaml}

Command failed on smithi105 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10'

3858659

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi076 smithi106

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi076 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858662

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi086 smithi185

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml}

"2019-04-17 22:56:33.186124 mon.a (mon.0) 382 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858667

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi204

rados/singleton/{all/mon-config-key-caps.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}}

"2019-04-17 22:48:23.133243 mon.a (mon.0) 94 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY)" in cluster log

3858669

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi066 smithi084

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml}

timed out waiting for mon to be updated with osd.0: 55834574901 < 55834574902

3858673

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi071 smithi134

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read.yaml}

Command failed on smithi071 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858674

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi001 smithi096 smithi114 smithi164

rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_latest.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml}

Command failed on smithi164 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858676

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi068 smithi190

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/crash.yaml}

Test failure: test_info (tasks.mgr.test_crash.TestCrash)

3858679

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi188

rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}

Command failed (workunit test cephtool/test.sh) on smithi188 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

3858680

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi060 smithi079 smithi089 smithi154

rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/nautilus.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-octopus.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} supported-random-distro$/{rhel_latest.yaml} thrashosds-health.yaml}

Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858683

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 

smithi098 smithi183

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml}

Command failed on smithi098 with status 1: '\n sudo yum -y install ceph-debuginfo\n '

3858684

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi112 smithi137

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-agent-small.yaml}

"2019-04-17 23:01:05.778578 mon.a (mon.0) 294 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858685

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi132

rados/singleton/{all/osd-backfill.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}

"2019-04-17 22:57:46.067355 mon.a (mon.0) 168 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858686

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi114 smithi164

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi164 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858689

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi065 smithi085

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml}

Command failed on smithi085 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858690

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi036

rados/singleton/{all/osd-recovery-incomplete.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}}

timed out waiting for mon to be updated with osd.0: 38654705676 < 38654705677

3858692

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi037 smithi142

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml}

Command failed on smithi037 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858693

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi104 smithi110 smithi112 smithi130

rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}

Found coredumps on ubuntu@smithi110.front.sepia.ceph.com

3858694

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi180 smithi204

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/dashboard.yaml}

Test failure: test_standby (tasks.mgr.test_dashboard.TestDashboard)

3858695

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi036 smithi113 smithi174 smithi188

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/luminous-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml}

Scrubbing terminated -- not all pgs were active and clean.

3858698

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi134 smithi172

rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/rados_cls_all.yaml}

"2019-04-20 14:25:47.810752 mon.b (mon.0) 196 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858699

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi162 smithi166

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml}

Command failed on smithi162 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858701

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi091 smithi190

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-many-deletes.yaml}

Command failed on smithi190 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858703

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi167

rados/singleton/{all/peer.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}}

"2019-04-20 14:41:12.920407 mon.a (mon.0) 261 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858704

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi114 smithi193

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}

Command failed on smithi193 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6'

3858705

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi019 smithi115 smithi121

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml}

Command failed on smithi121 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858711

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi072 smithi087 smithi095 smithi165

rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/few.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}

Command failed on smithi095 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858715

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi184 smithi191

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi184 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858717

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi002 smithi031 smithi122

rados/multimon/{clusters/21.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_clock_no_skews.yaml}

"2019-04-20 14:37:02.771870 mon.a (mon.0) 53 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY)" in cluster log

3858718

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

 

smithi071 smithi201

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml tasks/mon_recovery.yaml validater/lockdep.yaml}

Command failed on smithi071 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'

3858720

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi040

rados/singleton/{all/radostool.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}

Command failed on smithi040 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'

3858721

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi060 smithi085

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/rados_api_tests.yaml}

"2019-04-20 14:44:22.935513 mon.a (mon.0) 364 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858722

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi173

rados/standalone/{supported-random-distro$/{rhel_latest.yaml} workloads/osd.yaml}

Command failed (workunit test osd/osd-backfill-prio.sh) on smithi173 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-backfill-prio.sh'

3858724

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi012 smithi035

rados/monthrash/{ceph.yaml clusters/9-mons.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/many.yaml workloads/rados_mon_osdmap_prune.yaml}

"2019-04-20 14:40:20.336670 mon.a (mon.0) 114 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858726

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi041 smithi161

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-small-objects.yaml}

Command failed on smithi041 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858727

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi089 smithi097

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-snaps-few-objects-overwrites.yaml}

Command failed on smithi097 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858728

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi068 smithi090 smithi106 smithi150

rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}

Found coredumps on ubuntu@smithi106.front.sepia.ceph.com

3858730

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi099 smithi136

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/insights.yaml}

Test failure: test_health_history (tasks.mgr.test_insights.TestInsights)

3858731

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi005 smithi023

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/radosbench.yaml}

reached maximum tries (500) after waiting for 3000 seconds

3858732

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi012 smithi146 smithi171 smithi192

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/mimic-v1only.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/radosbench.yaml}

"2019-04-20 15:16:15.429750 mon.a (mon.0) 531 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858734

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi118

rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

Command failed (workunit test cephtool/test.sh) on smithi118 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'

3858735

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi027 smithi049 smithi143

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}

Command failed on smithi049 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858736

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi124 smithi174

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/redirect.yaml}

Found coredumps on ubuntu@smithi174.front.sepia.ceph.com

3858737

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi131 smithi204

rados/singleton-nomsgr/{all/multi-backfill-reject.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml}}

"2019-04-20 14:43:07.291219 mon.a (mon.0) 220 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858740

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi105

rados/singleton/{all/recovery-preemption.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}}

"2019-04-20 14:47:45.187796 mon.a (mon.0) 125 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858741

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi170 smithi193

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/redirect_promote_tests.yaml}

Command failed on smithi170 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3'

3858742

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi130 smithi137

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi137 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858743

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi003 smithi132

rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/rados_striper.yaml}

"2019-04-20 14:40:16.140911 mon.a (mon.0) 143 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858745

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi134 smithi172

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/module_selftest.yaml}

Test failure: test_selftest_command_spam (tasks.mgr.test_module_selftest.TestModuleSelftest)

3858746

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi014 smithi176

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/redirect_set_object.yaml}

Command failed on smithi014 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858749

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi088

rados/singleton/{all/test-crash.yaml msgr-failures/many.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

Command failed (workunit test rados/test_crash.sh) on smithi088 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_crash.sh'

3858751

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 

smithi074 smithi087

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml tasks/rados_api_tests.yaml validater/valgrind.yaml}

Command failed on smithi074 with status 1: '\n sudo yum -y install ceph-debuginfo\n '

3858752

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi077 smithi187

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=clay-k=4-m=2.yaml}

Command failed on smithi187 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858757

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi168

rados/singleton/{all/test_envlibrados_for_rocksdb.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

"2019-04-20 14:44:59.839900 mon.a (mon.0) 83 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858759

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi100 smithi164

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml}

Command failed on smithi164 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'

3858761

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi044 smithi070

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/orchestrator_cli.yaml}

Test failure: test_mds_rm (tasks.mgr.test_orchestrator_cli.TestOrchestratorCli)

3858763

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi026 smithi162 smithi166

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml}

Command failed on smithi162 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858764

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi129 smithi152

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/fastclose.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-pool-snaps-few-objects-overwrites.yaml}

Command failed on smithi129 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858765

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi032 smithi125 smithi142 smithi181

rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}

Found coredumps on ubuntu@smithi142.front.sepia.ceph.com

3858766

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi076 smithi159

rados/singleton/{all/thrash-backfill-full.yaml msgr-failures/many.yaml msgr/async-v2only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}}

Command failed on smithi159 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858767

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi081 smithi138 smithi157 smithi205

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus-v1only.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/few.yaml rados.yaml thrashers/none.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml}

Command failed on smithi157 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3'

3858768

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi149 smithi169

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml}

Command failed on smithi169 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858769

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi061 smithi202

rados/singleton/{all/thrash-eio.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}}

Command failed on smithi202 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858772

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi080 smithi193

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/write_fadvise_dontneed.yaml}

Command failed on smithi080 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858774

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi170 smithi183

rados/singleton/{all/thrash-rados/{thrash-rados.yaml thrashosds-health.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{centos_latest.yaml}}

"2019-04-20 15:09:38.661779 mon.a (mon.0) 317 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858776

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi051 smithi158

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_16.04.yaml} tasks/progress.yaml}

Test failure: test_osd_came_back (tasks.mgr.test_progress.TestProgress)

3858777

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi042 smithi110

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml}

Command failed on smithi110 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858778

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi157 smithi205

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/admin_socket_objecter_requests.yaml}

Command failed on smithi157 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3'

3858783

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi096

rados/singleton-bluestore/{all/cephtool.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

"2019-04-20 15:03:40.309363 mon.a (mon.0) 106 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858784

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 

smithi001

rados/singleton-flat/valgrind-leaks.yaml

Command failed on smithi001 with status 1: '\n sudo yum -y install ceph-debuginfo\n '

3858787

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi066 smithi105 smithi148 smithi194

rados/upgrade/mimic-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/mimic.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-nautilus.yaml 8-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} supported-random-distro$/{centos_latest.yaml} thrashosds-health.yaml}

failed to become clean before timeout expired

3858788

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi133 smithi145

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/cache-agent-big.yaml}

Command failed on smithi145 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858790

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

 

smithi167 smithi180

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/default/{default.yaml thrashosds-health.yaml} msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml tasks/rados_cls_all.yaml validater/lockdep.yaml}

Command failed on smithi167 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858794

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi083 smithi156

rados/basic/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_workunit_loadgen_mostlyread.yaml}

"2019-04-20 15:02:11.232537 mon.b (mon.0) 143 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858795

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi057 smithi100 smithi163

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}

Command failed on smithi057 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858797

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi190

rados/singleton/{all/admin-socket.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml}}

"2019-04-20 14:59:38.310283 mon.a (mon.0) 73 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858799

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi006 smithi198

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/few.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-small-objects-fast-read-overwrites.yaml}

Command failed on smithi198 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858802

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi060 smithi085

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/cache-pool-snaps-readproxy.yaml}

Command failed on smithi060 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0'

3858804

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi139 smithi182

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi139 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858805

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi051 smithi083 smithi112 smithi156

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/careful.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}

Command failed on smithi051 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858808

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi031 smithi074

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/cache-pool-snaps.yaml}

Command failed on smithi074 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858809

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi081 smithi178

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml msgr-failures/fastclose.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml}

Command failed on smithi178 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858813

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi091

rados/singleton/{all/divergent_priors2.yaml msgr-failures/many.yaml msgr/async-v1only.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml}}

"2019-04-20 15:14:18.680725 mon.a (mon.0) 252 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858814

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi055 smithi093

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{default.yaml} backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/crush-compat.yaml msgr-failures/osd-delay.yaml msgr/async.yaml objectstore/bluestore-comp.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/cache-snaps.yaml}

Command failed on smithi055 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0'

3858815

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi014 smithi176

rados/mgr/{clusters/{2-node-mgr.yaml openstack.yaml} debug/mgr.yaml objectstore/bluestore-comp.yaml supported-random-distro$/{centos_latest.yaml} tasks/ssh_orchestrator.yaml}

Test failure: test_host_ls (tasks.mgr.test_ssh_orchestrator.TestOrchestratorCli)

3858816

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi040

rados/standalone/{supported-random-distro$/{centos_latest.yaml} workloads/erasure-code.yaml}

Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a53b006cf29f3be585ea5e0f09899714d7563095 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh'

3858817

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi184 smithi191

rados/monthrash/{ceph.yaml clusters/3-mons.yaml msgr-failures/mon-delay.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} thrashers/sync.yaml workloads/pool-create-delete.yaml}

"2019-04-20 15:23:44.989550 mon.a (mon.0) 126 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log

3858819

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 18.04

smithi017 smithi051

rados/thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-low-osd-mem-target.yaml rados.yaml supported-random-distro$/{ubuntu_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/cache.yaml}

"2019-04-20 15:23:31.735935 mon.b (mon.0) 200 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)" in cluster log

3858821

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi070 smithi129 smithi146 smithi183

rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/hammer.yaml backoff/normal.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{ubuntu_16.04.yaml} msgr-failures/few.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/radosbench.yaml}

Command failed on smithi183 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1'

3858822

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi001 smithi037 smithi067

rados/thrash-erasure-code-big/{ceph.yaml cluster/{12-osds.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-rados-plugin=jerasure-k=4-m=2.yaml}

Command failed on smithi037 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858823

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi003 smithi131 smithi132

rados/multimon/{clusters/9.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{centos_latest.yaml} tasks/mon_clock_no_skews.yaml}

"2019-04-20 15:17:10.041426 mon.b (mon.0) 54 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY)" in cluster log

3858824

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 

smithi044 smithi070

rados/verify/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-thrash/none.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/bluestore-bitmap.yaml rados.yaml tasks/mon_recovery.yaml validater/valgrind.yaml}

Command failed on smithi070 with status 1: 'sudo yum install -y kernel'

3858828

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

rhel 7.5

smithi158 smithi199

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/peering.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/few.yaml msgr/async-v2only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/pool-snaps-few-objects.yaml}

Command failed on smithi199 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858829

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi157 smithi205

rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/careful.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml}

Command failed on smithi157 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858833

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi089 smithi097

rados/thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/few.yaml objectstore/bluestore-comp.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_16.04.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml}

Command failed on smithi097 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

3858835

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.5

smithi044 smithi070

rados/thrash-erasure-code-overwrites/{bluestore-bitmap.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/normal.yaml msgr-failures/osd-delay.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/fastread.yaml thrashosds-health.yaml workloads/ec-small-objects-overwrites.yaml}

Command failed on smithi044 with status 1: 'sudo yum install -y kernel'

3858840

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

centos 7.6

smithi014 smithi060 smithi085 smithi199

rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/jewel-v1only.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_latest.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/mapgap.yaml thrashosds-health.yaml workloads/rbd_cls.yaml}

Scrubbing terminated -- not all pgs were active and clean.

3858841

FAIL

wip-sage-testing-2019-04-16-1726

wip-sage-testing-2019-04-16-1726

master

smithi

ubuntu 16.04

smithi002 smithi038

rados/thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml 2-recovery-overrides/{more-active-recovery.yaml} backoff/normal.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/off.yaml msgr-failures/fastclose.yaml msgr/async-v1only.yaml objectstore/bluestore-bitmap.yaml rados.yaml supported-random-distro$/{ubuntu_16.04.yaml} thrashers/pggrow.yaml thrashosds-health.yaml workloads/radosbench.yaml}

Command failed on smithi002 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2'