| User | Scheduled | Started | Updated | Runtime | Suite | Branch | Machine Type | Revision | Pass | Fail |
|---|---|---|---|---|---|---|---|---|---|---|
| nojha | 2019-02-21 07:02:24 | 2019-02-21 07:03:25 | 2019-02-21 10:05:39 | 3:02:14 | rados:thrash-erasure-code | wip-2-36739 | smithi | 21939af | 2 | 98 |
| Status | Job ID | Links | Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Description | Nodes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| fail | 3621486 |
|
2019-02-21 07:02:29 | 2019-02-21 07:03:25 | 2019-02-21 08:09:26 | 1:06:01 | 0:57:12 | 0:08:49 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:27:47.309343 osd.0 (osd.0) 124 : cluster [ERR] osd.0 pg 2.5s1 found obsolete rollback obj 1#2:a001dbad:::benchmark_data_smithi071_9070_object20982:head#1044 generation < trimmed_to 183'4363...repaired" in cluster log |
||||||||||||||
| fail | 3621487 |
|
2019-02-21 07:02:29 | 2019-02-21 07:03:30 | 2019-02-21 08:23:31 | 1:20:01 | 1:08:48 | 0:11:13 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:29:08.641340 osd.7 (osd.7) 70 : cluster [ERR] osd.7 pg 2.8s0 found obsolete rollback obj 0#2:100e0816:::benchmark_data_smithi148_9375_object9408:head#f17 generation < trimmed_to 103'3922...repaired" in cluster log |
||||||||||||||
| fail | 3621488 |
|
2019-02-21 07:02:29 | 2019-02-21 07:05:23 | 2019-02-21 08:21:23 | 1:16:00 | 1:07:50 | 0:08:10 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:29:16.088512 osd.2 (osd.2) 147 : cluster [ERR] osd.2 pg 2.cs0 found obsolete rollback obj 0#2:3001a7be:::benchmark_data_smithi170_40833_object52807:head#b36 generation < trimmed_to 170'2968...repaired" in cluster log |
||||||||||||||
| fail | 3621489 |
|
2019-02-21 07:02:29 | 2019-02-21 07:07:16 | 2019-02-21 08:19:17 | 1:12:01 | 1:05:19 | 0:06:42 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:31:45.340074 osd.6 (osd.6) 114 : cluster [ERR] osd.6 pg 2.22s0 found obsolete rollback obj 0#2:44064398:::benchmark_data_smithi039_41338_object7553:head#64b generation < trimmed_to 151'2107...repaired" in cluster log |
||||||||||||||
| fail | 3621490 |
|
2019-02-21 07:02:29 | 2019-02-21 07:07:16 | 2019-02-21 08:23:17 | 1:16:01 | 1:07:49 | 0:08:12 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:31:56.108690 osd.6 (osd.6) 136 : cluster [ERR] osd.6 pg 2.cs0 found obsolete rollback obj 0#2:30000ec4:::benchmark_data_smithi099_41279_object40321:head#1e07 generation < trimmed_to 181'9438...repaired" in cluster log |
||||||||||||||
| fail | 3621491 |
|
2019-02-21 07:02:29 | 2019-02-21 07:07:20 | 2019-02-21 08:17:21 | 1:10:01 | 1:02:15 | 0:07:46 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:37:01.128263 osd.6 (osd.6) 91 : cluster [ERR] osd.6 pg 3.5s0 found obsolete rollback obj 0#3:a000ef69:::benchmark_data_smithi082_60182_object62920:head#24dd generation < trimmed_to 369'10139...repaired" in cluster log |
||||||||||||||
| fail | 3621492 |
|
2019-02-21 07:02:29 | 2019-02-21 07:09:23 | 2019-02-21 08:23:23 | 1:14:00 | 1:05:48 | 0:08:12 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:42:37.517188 osd.5 (osd.5) 331 : cluster [ERR] osd.5 pg 3.21s1 found obsolete rollback obj 1#3:8401af2e:::benchmark_data_smithi173_98217_object35146:head#c91 generation < trimmed_to 381'3767...repaired" in cluster log |
||||||||||||||
| fail | 3621493 |
|
2019-02-21 07:02:29 | 2019-02-21 07:09:29 | 2019-02-21 08:09:29 | 1:00:00 | 0:52:19 | 0:07:41 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:31:41.278404 osd.7 (osd.7) 90 : cluster [ERR] osd.7 pg 2.1s0 found obsolete rollback obj 0#2:800514b4:::benchmark_data_smithi199_9707_object1366:head#a84 generation < trimmed_to 112'3648...repaired" in cluster log |
||||||||||||||
| fail | 3621494 |
|
2019-02-21 07:02:29 | 2019-02-21 07:09:37 | 2019-02-21 08:09:37 | 1:00:00 | 0:52:23 | 0:07:37 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:31:08.204532 osd.5 (osd.5) 92 : cluster [ERR] osd.5 pg 2.0s2 found obsolete rollback obj 2#2:00018a63:::benchmark_data_smithi149_9081_object20132:head#e22 generation < trimmed_to 111'3841...repaired" in cluster log |
||||||||||||||
| fail | 3621495 |
|
2019-02-21 07:02:29 | 2019-02-21 07:09:37 | 2019-02-21 08:25:43 | 1:16:06 | 1:05:38 | 0:10:28 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:35:48.162220 osd.1 (osd.1) 79 : cluster [ERR] osd.1 pg 2.26s2 found obsolete rollback obj 2#2:6401c0b6:::benchmark_data_smithi023_40678_object34732:head#8c1 generation < trimmed_to 132'2278...repaired" in cluster log |
||||||||||||||
| fail | 3621496 |
|
2019-02-21 07:02:29 | 2019-02-21 07:11:11 | 2019-02-21 08:19:11 | 1:08:00 | 1:01:12 | 0:06:48 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:38:13.952494 osd.3 (osd.3) 181 : cluster [ERR] osd.3 pg 2.es1 found obsolete rollback obj 1#2:70003b80:::benchmark_data_smithi117_40987_object35932:head#104f generation < trimmed_to 216'4623...repaired" in cluster log |
||||||||||||||
| fail | 3621497 |
|
2019-02-21 07:02:29 | 2019-02-21 07:11:17 | 2019-02-21 08:07:17 | 0:56:00 | 0:49:12 | 0:06:48 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:32:30.782060 osd.6 (osd.6) 69 : cluster [ERR] osd.6 pg 2.17s0 found obsolete rollback obj 0#2:e805fdf0:::benchmark_data_smithi172_41072_object1635:head#ab4 generation < trimmed_to 86'3223...repaired" in cluster log |
||||||||||||||
| fail | 3621498 |
|
2019-02-21 07:02:29 | 2019-02-21 07:13:21 | 2019-02-21 08:25:21 | 1:12:00 | 1:05:14 | 0:06:46 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:33:46.154820 osd.6 (osd.6) 59 : cluster [ERR] osd.6 pg 2.7s0 found obsolete rollback obj 0#2:e005809d:::benchmark_data_smithi050_41022_object6339:head#b13 generation < trimmed_to 87'3002...repaired" in cluster log |
||||||||||||||
| fail | 3621499 |
|
2019-02-21 07:02:29 | 2019-02-21 07:13:34 | 2019-02-21 08:27:34 | 1:14:00 | 1:05:50 | 0:08:10 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:35:12.444435 osd.0 (osd.0) 80 : cluster [ERR] osd.0 pg 2.11s0 found obsolete rollback obj 0#2:88001100:::benchmark_data_smithi200_41738_object10455:head#de9 generation < trimmed_to 111'3914...repaired" in cluster log |
||||||||||||||
| fail | 3621500 |
|
2019-02-21 07:02:29 | 2019-02-21 07:13:39 | 2019-02-21 08:25:40 | 1:12:01 | 1:04:28 | 0:07:33 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:34:11.919445 osd.5 (osd.5) 54 : cluster [ERR] osd.5 pg 2.2s0 found obsolete rollback obj 0#2:40047989:::benchmark_data_smithi158_9236_object18120:head#1bb0 generation < trimmed_to 94'7107...repaired" in cluster log |
||||||||||||||
| fail | 3621501 |
|
2019-02-21 07:02:29 | 2019-02-21 07:15:26 | 2019-02-21 08:17:25 | 1:01:59 | 0:55:17 | 0:06:42 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:35:54.615796 osd.7 (osd.7) 64 : cluster [ERR] osd.7 pg 2.1es1 found obsolete rollback obj 1#2:7806a9f2:::benchmark_data_smithi165_8989_object2441:head#969 generation < trimmed_to 102'2981...repaired" in cluster log |
||||||||||||||
| fail | 3621502 |
|
2019-02-21 07:02:29 | 2019-02-21 07:15:26 | 2019-02-21 08:27:26 | 1:12:00 | 1:05:59 | 0:06:01 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:39:59.332899 osd.2 (osd.2) 201 : cluster [ERR] osd.2 pg 2.cs0 found obsolete rollback obj 0#2:300045bf:::benchmark_data_smithi144_41023_object15363:head#132e generation < trimmed_to 230'5878...repaired" in cluster log |
||||||||||||||
| fail | 3621503 |
|
2019-02-21 07:02:29 | 2019-02-21 07:15:27 | 2019-02-21 08:25:28 | 1:10:01 | 1:01:49 | 0:08:12 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:37:48.802351 osd.7 (osd.7) 105 : cluster [ERR] osd.7 pg 2.8s2 found obsolete rollback obj 2#2:100430f5:::benchmark_data_smithi085_9371_object12327:head#cb7 generation < trimmed_to 121'4036...repaired" in cluster log |
||||||||||||||
| fail | 3621504 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:37:13 | 1:14:00 | 1:06:59 | 0:07:01 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:49.939679 osd.2 (osd.2) 69 : cluster [ERR] osd.2 pg 2.1s0 found obsolete rollback obj 0#2:80026ff7:::benchmark_data_smithi167_9296_object15155:head#cbd generation < trimmed_to 101'3651...repaired" in cluster log |
||||||||||||||
| fail | 3621505 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:12 | 2019-02-21 08:45:13 | 1:22:01 | 1:14:50 | 0:07:11 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:00.601396 osd.1 (osd.1) 64 : cluster [ERR] osd.1 pg 2.as0 found obsolete rollback obj 0#2:50171d0f:::benchmark_data_smithi002_40706_object741:head#107f generation < trimmed_to 85'4226...repaired" in cluster log |
||||||||||||||
| fail | 3621506 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:31:13 | 1:08:00 | 1:00:25 | 0:07:35 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:12.051942 osd.2 (osd.2) 89 : cluster [ERR] osd.2 pg 2.10s0 found obsolete rollback obj 0#2:08001118:::benchmark_data_smithi183_41386_object29369:head#d0b generation < trimmed_to 124'3554...repaired" in cluster log |
||||||||||||||
| fail | 3621507 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:31:13 | 1:08:00 | 1:01:00 | 0:07:00 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:50:02.602794 osd.2 (osd.2) 68 : cluster [ERR] osd.2 pg 2.12s2 found obsolete rollback obj 2#2:480204e1:::benchmark_data_smithi038_40743_object18740:head#745 generation < trimmed_to 128'2001...repaired" in cluster log |
||||||||||||||
| fail | 3621508 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:37:13 | 1:14:00 | 1:06:36 | 0:07:24 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:04.816907 osd.5 (osd.5) 71 : cluster [ERR] osd.5 pg 2.4s0 found obsolete rollback obj 0#2:200056b1:::benchmark_data_smithi143_41125_object17098:head#b16 generation < trimmed_to 101'2994...repaired" in cluster log |
||||||||||||||
| fail | 3621509 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:35:13 | 1:12:00 | 1:04:25 | 0:07:35 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:49:48.382480 osd.6 (osd.6) 147 : cluster [ERR] osd.6 pg 2.cs0 found obsolete rollback obj 0#2:3000210f:::benchmark_data_smithi201_9666_object15132:head#12d4 generation < trimmed_to 206'5601...repaired" in cluster log |
||||||||||||||
| fail | 3621510 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:12 | 2019-02-21 08:27:12 | 1:04:00 | 0:57:38 | 0:06:22 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:01:32.492601 osd.4 (osd.4) 355 : cluster [ERR] osd.4 pg 4.1s0 found obsolete rollback obj 0#4:8004d797:::benchmark_data_smithi191_129312_object5565:head#b98 generation < trimmed_to 393'3623...repaired" in cluster log |
||||||||||||||
| fail | 3621511 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:12 | 2019-02-21 08:39:13 | 1:16:01 | 1:08:47 | 0:07:14 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:50:05.269034 osd.3 (osd.3) 88 : cluster [ERR] osd.3 pg 2.2bs2 found obsolete rollback obj 2#2:d402a97b:::benchmark_data_smithi103_41485_object3130:head#82d generation < trimmed_to 116'2668...repaired" in cluster log |
||||||||||||||
| fail | 3621512 |
|
2019-02-21 07:02:29 | 2019-02-21 07:23:13 | 2019-02-21 08:43:13 | 1:20:00 | 1:13:23 | 0:06:37 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:50:32.097808 osd.2 (osd.2) 174 : cluster [ERR] osd.2 pg 2.21s0 found obsolete rollback obj 0#2:84004145:::benchmark_data_smithi041_40936_object14398:head#a5f generation < trimmed_to 213'3563...repaired" in cluster log |
||||||||||||||
| fail | 3621513 |
|
2019-02-21 07:02:30 | 2019-02-21 07:23:13 | 2019-02-21 08:39:13 | 1:16:00 | 1:09:58 | 0:06:02 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:56:36.108763 osd.4 (osd.4) 68 : cluster [ERR] osd.4 pg 3.3s0 found obsolete rollback obj 0#3:c0006908:::benchmark_data_smithi129_64849_object16696:head#53d generation < trimmed_to 320'1430...repaired" in cluster log |
||||||||||||||
| fail | 3621514 |
|
2019-02-21 07:02:30 | 2019-02-21 07:23:13 | 2019-02-21 08:39:14 | 1:16:01 | 1:08:35 | 0:07:26 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:51:22.547713 osd.0 (osd.0) 136 : cluster [ERR] osd.0 pg 2.0s0 found obsolete rollback obj 0#2:00020b20:::benchmark_data_smithi062_9725_object928:head#aaf generation < trimmed_to 138'4811...repaired" in cluster log |
||||||||||||||
| fail | 3621515 |
|
2019-02-21 07:02:30 | 2019-02-21 07:23:18 | 2019-02-21 08:39:18 | 1:16:00 | 1:08:58 | 0:07:02 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:12.923804 osd.1 (osd.1) 66 : cluster [ERR] osd.1 pg 2.9s0 found obsolete rollback obj 0#2:903ed9b1:::benchmark_data_smithi134_9273_object246:head#d31 generation < trimmed_to 85'3399...repaired" in cluster log |
||||||||||||||
| fail | 3621516 |
|
2019-02-21 07:02:30 | 2019-02-21 07:23:21 | 2019-02-21 08:43:21 | 1:20:00 | 1:13:07 | 0:06:53 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:48:54.250550 osd.3 (osd.3) 75 : cluster [ERR] osd.3 pg 2.bs0 found obsolete rollback obj 0#2:d002bc13:::benchmark_data_smithi192_41137_object4805:head#10cd generation < trimmed_to 86'5090...repaired" in cluster log |
||||||||||||||
| fail | 3621517 |
|
2019-02-21 07:02:30 | 2019-02-21 07:23:23 | 2019-02-21 08:45:23 | 1:22:00 | 1:14:06 | 0:07:54 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:49:45.009315 osd.3 (osd.3) 84 : cluster [ERR] osd.3 pg 2.2ds0 found obsolete rollback obj 0#2:b400f282:::benchmark_data_smithi006_41002_object12356:head#7df generation < trimmed_to 123'2289...repaired" in cluster log |
||||||||||||||
| fail | 3621518 |
|
2019-02-21 07:02:30 | 2019-02-21 07:25:36 | 2019-02-21 08:31:36 | 1:06:00 | 0:58:36 | 0:07:24 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:52:06.495581 osd.2 (osd.2) 131 : cluster [ERR] osd.2 pg 2.cs0 found obsolete rollback obj 0#2:3001064e:::benchmark_data_smithi001_8833_object6794:head#105c generation < trimmed_to 149'5351...repaired" in cluster log |
||||||||||||||
| fail | 3621519 |
|
2019-02-21 07:02:30 | 2019-02-21 07:25:36 | 2019-02-21 08:39:42 | 1:14:06 | 1:06:49 | 0:07:17 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:53:41.425971 osd.6 (osd.6) 157 : cluster [ERR] osd.6 pg 2.22s0 found obsolete rollback obj 0#2:440344a3:::benchmark_data_smithi095_9684_object34176:head#c0d generation < trimmed_to 183'3299...repaired" in cluster log |
||||||||||||||
| fail | 3621520 |
|
2019-02-21 07:02:30 | 2019-02-21 07:25:36 | 2019-02-21 08:37:37 | 1:12:01 | 1:04:37 | 0:07:24 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:51:51.377374 osd.5 (osd.5) 127 : cluster [ERR] osd.5 pg 2.9s1 found obsolete rollback obj 1#2:90028bd7:::benchmark_data_smithi163_40860_object66633:head#1bed generation < trimmed_to 181'7204...repaired" in cluster log |
||||||||||||||
| fail | 3621521 |
|
2019-02-21 07:02:30 | 2019-02-21 07:25:36 | 2019-02-21 08:39:37 | 1:14:01 | 1:06:27 | 0:07:34 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:05:32.666807 osd.2 (osd.2) 434 : cluster [ERR] osd.2 pg 4.19s1 found obsolete rollback obj 1#4:98072cc9:::benchmark_data_smithi096_156402_object14806:head#7d8 generation < trimmed_to 484'2242...repaired" in cluster log |
||||||||||||||
| fail | 3621522 |
|
2019-02-21 07:02:30 | 2019-02-21 07:25:46 | 2019-02-21 08:37:46 | 1:12:00 | 1:04:22 | 0:07:38 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:54:30.862963 osd.7 (osd.7) 184 : cluster [ERR] osd.7 pg 3.9s2 found obsolete rollback obj 2#3:90043a7c:::benchmark_data_smithi182_73028_object796:head#bd5 generation < trimmed_to 220'3935...repaired" in cluster log |
||||||||||||||
| fail | 3621523 |
|
2019-02-21 07:02:30 | 2019-02-21 07:27:55 | 2019-02-21 08:39:56 | 1:12:01 | 1:04:43 | 0:07:18 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:00:24.958381 osd.2 (osd.2) 282 : cluster [ERR] osd.2 pg 3.21s0 found obsolete rollback obj 0#3:8400cd05:::benchmark_data_smithi053_93911_object6638:head#b7d generation < trimmed_to 347'3887...repaired" in cluster log |
||||||||||||||
| fail | 3621524 |
|
2019-02-21 07:02:30 | 2019-02-21 07:29:52 | 2019-02-21 08:31:52 | 1:02:00 | 0:55:05 | 0:06:55 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:51:57.805139 osd.4 (osd.4) 82 : cluster [ERR] osd.4 pg 2.9s0 found obsolete rollback obj 0#2:9000bf64:::benchmark_data_smithi132_9320_object23833:head#a1c generation < trimmed_to 109'2634...repaired" in cluster log |
||||||||||||||
| fail | 3621525 |
|
2019-02-21 07:02:30 | 2019-02-21 07:31:45 | 2019-02-21 08:59:47 | 1:28:02 | 1:21:58 | 0:06:04 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:55:26.252600 osd.1 (osd.1) 175 : cluster [ERR] osd.1 pg 2.24s0 found obsolete rollback obj 0#2:240040d1:::benchmark_data_smithi164_41260_object9152:head#e93 generation < trimmed_to 216'4065...repaired" in cluster log |
||||||||||||||
| fail | 3621526 |
|
2019-02-21 07:02:30 | 2019-02-21 07:31:45 | 2019-02-21 08:49:45 | 1:18:00 | 1:11:07 | 0:06:53 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:55:35.347711 osd.2 (osd.2) 129 : cluster [ERR] osd.2 pg 2.7s2 found obsolete rollback obj 2#2:e00287ee:::benchmark_data_smithi136_9298_object45547:head#15d1 generation < trimmed_to 161'5873...repaired" in cluster log |
||||||||||||||
| fail | 3621527 |
|
2019-02-21 07:02:30 | 2019-02-21 07:31:59 | 2019-02-21 08:47:59 | 1:16:00 | 1:09:32 | 0:06:28 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:56:14.128852 osd.4 (osd.4) 112 : cluster [ERR] osd.4 pg 2.19s2 found obsolete rollback obj 2#2:9800f8c8:::benchmark_data_smithi188_40857_object20111:head#d03 generation < trimmed_to 143'4036...repaired" in cluster log |
||||||||||||||
| fail | 3621528 |
|
2019-02-21 07:02:30 | 2019-02-21 07:37:22 | 2019-02-21 08:35:22 | 0:58:00 | 0:51:03 | 0:06:57 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:58:23.311570 osd.5 (osd.5) 84 : cluster [ERR] osd.5 pg 2.3s0 found obsolete rollback obj 0#2:c001a697:::benchmark_data_smithi119_40918_object10642:head#94d generation < trimmed_to 102'2631...repaired" in cluster log |
||||||||||||||
| fail | 3621529 |
|
2019-02-21 07:02:30 | 2019-02-21 07:37:45 | 2019-02-21 08:51:47 | 1:14:02 | 1:07:33 | 0:06:29 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 07:57:55.043474 osd.0 (osd.0) 61 : cluster [ERR] osd.0 pg 2.9s2 found obsolete rollback obj 2#2:915fd131:::benchmark_data_smithi046_40716_object59:head#1ab2 generation < trimmed_to 89'6873...repaired" in cluster log |
||||||||||||||
| fail | 3621530 |
|
2019-02-21 07:02:30 | 2019-02-21 07:39:41 | 2019-02-21 08:41:41 | 1:02:00 | 0:55:17 | 0:06:43 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:00:40.383706 osd.3 (osd.3) 84 : cluster [ERR] osd.3 pg 2.19s0 found obsolete rollback obj 0#2:9801fc68:::benchmark_data_smithi005_40993_object30679:head#b10 generation < trimmed_to 126'2895...repaired" in cluster log |
||||||||||||||
| fail | 3621531 |
|
2019-02-21 07:02:30 | 2019-02-21 07:41:38 | 2019-02-21 08:59:38 | 1:18:00 | 1:10:39 | 0:07:21 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:01:17.355492 osd.3 (osd.3) 63 : cluster [ERR] osd.3 pg 2.6s1 found obsolete rollback obj 1#2:6002d22f:::benchmark_data_smithi033_9002_object4023:head#a4d generation < trimmed_to 93'2749...repaired" in cluster log |
||||||||||||||
| fail | 3621532 |
|
2019-02-21 07:02:30 | 2019-02-21 07:41:46 | 2019-02-21 08:57:47 | 1:16:01 | 1:09:18 | 0:06:43 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:09:18.160366 osd.3 (osd.3) 264 : cluster [ERR] osd.3 pg 3.5s1 found obsolete rollback obj 1#3:a0000fc3:::benchmark_data_smithi012_49107_object60370:head#26ac generation < trimmed_to 318'10942...repaired" in cluster log |
||||||||||||||
| fail | 3621533 |
|
2019-02-21 07:02:30 | 2019-02-21 07:47:24 | 2019-02-21 09:09:25 | 1:22:01 | 1:15:07 | 0:06:54 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:09:27.391590 osd.0 (osd.0) 105 : cluster [ERR] osd.0 pg 2.1s2 found obsolete rollback obj 2#2:80033f24:::benchmark_data_smithi104_41281_object49968:head#127d generation < trimmed_to 180'4803...repaired" in cluster log |
||||||||||||||
| fail | 3621534 |
|
2019-02-21 07:02:30 | 2019-02-21 07:47:25 | 2019-02-21 08:47:25 | 1:00:00 | 0:53:35 | 0:06:25 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:21:16.186550 osd.5 (osd.5) 235 : cluster [ERR] osd.5 pg 4.7s1 found obsolete rollback obj 1#4:e0000aed:::benchmark_data_smithi017_109827_object19548:head#fca generation < trimmed_to 460'5317...repaired" in cluster log |
||||||||||||||
| fail | 3621535 |
|
2019-02-21 07:02:30 | 2019-02-21 07:49:42 | 2019-02-21 08:59:42 | 1:10:00 | 1:03:48 | 0:06:12 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:10:27.066607 osd.5 (osd.5) 87 : cluster [ERR] osd.5 pg 2.1s1 found obsolete rollback obj 1#2:8000cb22:::benchmark_data_smithi131_41413_object12886:head#cb8 generation < trimmed_to 99'3712...repaired" in cluster log |
||||||||||||||
| fail | 3621536 |
|
2019-02-21 07:02:30 | 2019-02-21 07:51:43 | 2019-02-21 08:57:43 | 1:06:00 | 1:00:24 | 0:05:36 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:13:16.090476 osd.4 (osd.4) 107 : cluster [ERR] osd.4 pg 2.6s2 found obsolete rollback obj 2#2:60006f99:::benchmark_data_smithi044_55634_object19823:head#c00 generation < trimmed_to 133'3763...repaired" in cluster log |
||||||||||||||
| fail | 3621537 |
|
2019-02-21 07:02:30 | 2019-02-21 07:51:43 | 2019-02-21 09:09:43 | 1:18:00 | 1:11:36 | 0:06:24 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:17:47.793631 osd.0 (osd.0) 19 : cluster [ERR] osd.0 pg 2.3cs0 found obsolete rollback obj 0#2:3d87e336:::benchmark_data_smithi118_41336_object0:head#916 generation < trimmed_to 270'2328...repaired" in cluster log |
||||||||||||||
| fail | 3621538 |
|
2019-02-21 07:02:30 | 2019-02-21 07:51:43 | 2019-02-21 08:59:43 | 1:08:00 | 1:01:42 | 0:06:18 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:14:02.433243 osd.3 (osd.3) 106 : cluster [ERR] osd.3 pg 2.19s0 found obsolete rollback obj 0#2:9800b22d:::benchmark_data_smithi008_9384_object2723:head#ba2 generation < trimmed_to 127'4243...repaired" in cluster log |
||||||||||||||
| fail | 3621539 |
|
2019-02-21 07:02:30 | 2019-02-21 07:53:38 | 2019-02-21 08:49:38 | 0:56:00 | 0:50:21 | 0:05:39 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:13:31.033671 osd.5 (osd.5) 78 : cluster [ERR] osd.5 pg 2.1es0 found obsolete rollback obj 0#2:780931d3:::benchmark_data_smithi069_8756_object6590:head#6a7 generation < trimmed_to 108'2131...repaired" in cluster log |
||||||||||||||
| fail | 3621540 |
|
2019-02-21 07:02:30 | 2019-02-21 07:55:22 | 2019-02-21 09:11:23 | 1:16:01 | 1:09:08 | 0:06:53 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:20:31.730869 osd.4 (osd.4) 170 : cluster [ERR] osd.4 pg 2.fs0 found obsolete rollback obj 0#2:f0015f4f:::benchmark_data_smithi154_40947_object22182:head#ee2 generation < trimmed_to 175'5393...repaired" in cluster log |
||||||||||||||
| fail | 3621541 |
|
2019-02-21 07:02:30 | 2019-02-21 07:57:31 | 2019-02-21 09:07:32 | 1:10:01 | 1:03:09 | 0:06:52 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:18:05.966594 osd.3 (osd.3) 77 : cluster [ERR] osd.3 pg 2.19s0 found obsolete rollback obj 0#2:98000eb6:::benchmark_data_smithi101_40901_object23768:head#e31 generation < trimmed_to 116'4002...repaired" in cluster log |
||||||||||||||
| fail | 3621542 |
|
2019-02-21 07:02:30 | 2019-02-21 07:59:25 | 2019-02-21 08:59:25 | 1:00:00 | 0:54:13 | 0:05:47 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:20:45.081196 osd.0 (osd.0) 88 : cluster [ERR] osd.0 pg 2.15s1 found obsolete rollback obj 1#2:a80263be:::benchmark_data_smithi031_8806_object32835:head#fb0 generation < trimmed_to 107'4265...repaired" in cluster log |
||||||||||||||
| fail | 3621543 |
|
2019-02-21 07:02:30 | 2019-02-21 08:05:35 | 2019-02-21 09:09:35 | 1:04:00 | 0:58:07 | 0:05:53 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:27:05.374394 osd.3 (osd.3) 94 : cluster [ERR] osd.3 pg 2.19s0 found obsolete rollback obj 0#2:98003dee:::benchmark_data_smithi074_40724_object22186:head#eba generation < trimmed_to 112'4364...repaired" in cluster log |
||||||||||||||
| fail | 3621544 |
|
2019-02-21 07:02:30 | 2019-02-21 08:07:29 | 2019-02-21 09:13:30 | 1:06:01 | 1:00:07 | 0:05:54 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:28:09.666242 osd.1 (osd.1) 84 : cluster [ERR] osd.1 pg 2.9s0 found obsolete rollback obj 0#2:9001a8aa:::benchmark_data_smithi139_41113_object19552:head#a3d generation < trimmed_to 116'2918...repaired" in cluster log |
||||||||||||||
| fail | 3621545 |
|
2019-02-21 07:02:30 | 2019-02-21 08:09:34 | 2019-02-21 09:13:33 | 1:03:59 | 0:57:02 | 0:06:57 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:29:19.981074 osd.5 (osd.5) 67 : cluster [ERR] osd.5 pg 2.1s1 found obsolete rollback obj 1#2:80bb3be4:::benchmark_data_smithi172_9192_object376:head#1876 generation < trimmed_to 88'6273...repaired" in cluster log |
||||||||||||||
| fail | 3621546 |
|
2019-02-21 07:02:30 | 2019-02-21 08:09:34 | 2019-02-21 09:25:34 | 1:16:00 | 1:08:28 | 0:07:32 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:32:36.540196 osd.6 (osd.6) 134 : cluster [ERR] osd.6 pg 2.2s1 found obsolete rollback obj 1#2:4002aefc:::benchmark_data_smithi071_9791_object22303:head#103b generation < trimmed_to 147'4853...repaired" in cluster log |
||||||||||||||
| fail | 3621547 |
|
2019-02-21 07:02:30 | 2019-02-21 08:09:38 | 2019-02-21 09:05:39 | 0:56:01 | 0:50:34 | 0:05:27 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:41:41.792653 osd.3 (osd.3) 358 : cluster [ERR] osd.3 pg 4.8s0 found obsolete rollback obj 0#4:100229b5:::benchmark_data_smithi199_91677_object13218:head#990 generation < trimmed_to 389'2681...repaired" in cluster log |
||||||||||||||
| fail | 3621548 |
|
2019-02-21 07:02:30 | 2019-02-21 08:15:41 | 2019-02-21 09:43:41 | 1:28:00 | 1:21:13 | 0:06:47 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:38:54.214659 osd.3 (osd.3) 107 : cluster [ERR] osd.3 pg 2.7s0 found obsolete rollback obj 0#2:e000897e:::benchmark_data_smithi166_9276_object21412:head#b88 generation < trimmed_to 140'3616...repaired" in cluster log |
||||||||||||||
| fail | 3621549 |
|
2019-02-21 07:02:30 | 2019-02-21 08:15:41 | 2019-02-21 09:23:41 | 1:08:00 | 1:01:11 | 0:06:49 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:36:53.986257 osd.1 (osd.1) 99 : cluster [ERR] osd.1 pg 2.2s1 found obsolete rollback obj 1#2:40010598:::benchmark_data_smithi067_41283_object16484:head#1465 generation < trimmed_to 128'5464...repaired" in cluster log |
||||||||||||||
| fail | 3621550 |
|
2019-02-21 07:02:30 | 2019-02-21 08:17:44 | 2019-02-21 09:23:43 | 1:05:59 | 1:00:02 | 0:05:57 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:40:56.119209 osd.6 (osd.6) 113 : cluster [ERR] osd.6 pg 2.6s1 found obsolete rollback obj 1#2:60017dcb:::benchmark_data_smithi165_9258_object3685:head#919 generation < trimmed_to 152'3297...repaired" in cluster log |
||||||||||||||
| fail | 3621551 |
|
2019-02-21 07:02:30 | 2019-02-21 08:17:45 | 2019-02-21 09:13:44 | 0:55:59 | 0:49:01 | 0:06:58 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:39:01.281934 osd.3 (osd.3) 87 : cluster [ERR] osd.3 pg 2.5s0 found obsolete rollback obj 0#2:a02ad28a:::benchmark_data_smithi082_41433_object2163:head#7d2 generation < trimmed_to 120'2025...repaired" in cluster log |
||||||||||||||
| fail | 3621552 |
|
2019-02-21 07:02:30 | 2019-02-21 08:19:44 | 2019-02-21 09:33:43 | 1:13:59 | 1:07:36 | 0:06:23 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:43:09.127224 osd.1 (osd.1) 109 : cluster [ERR] osd.1 pg 2.ds0 found obsolete rollback obj 0#2:b00252da:::benchmark_data_smithi117_40839_object31186:head#1bf5 generation < trimmed_to 159'8633...repaired" in cluster log |
||||||||||||||
| fail | 3621553 |
|
2019-02-21 07:02:30 | 2019-02-21 08:19:43 | 2019-02-21 09:45:44 | 1:26:01 | 1:18:57 | 0:07:04 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:42:04.387084 osd.0 (osd.0) 82 : cluster [ERR] osd.0 pg 2.1s2 found obsolete rollback obj 2#2:800106fa:::benchmark_data_smithi039_40956_object10543:head#1a63 generation < trimmed_to 107'7330...repaired" in cluster log |
||||||||||||||
| fail | 3621554 |
|
2019-02-21 07:02:30 | 2019-02-21 08:21:31 | 2019-02-21 09:23:31 | 1:02:00 | 0:54:33 | 0:07:27 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:48:08.145166 osd.4 (osd.4) 189 : cluster [ERR] osd.4 pg 3.2s0 found obsolete rollback obj 0#3:40037673:::benchmark_data_smithi105_76477_object798:head#d2e generation < trimmed_to 241'3386...repaired" in cluster log |
||||||||||||||
| fail | 3621555 |
|
2019-02-21 07:02:30 | 2019-02-21 08:21:31 | 2019-02-21 09:33:31 | 1:12:00 | 1:05:20 | 0:06:40 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:42:49.256555 osd.7 (osd.7) 68 : cluster [ERR] osd.7 pg 2.1s0 found obsolete rollback obj 0#2:800a7425:::benchmark_data_smithi181_8946_object7309:head#c4b generation < trimmed_to 90'3429...repaired" in cluster log |
||||||||||||||
| fail | 3621556 |
|
2019-02-21 07:02:31 | 2019-02-21 08:21:36 | 2019-02-21 09:31:35 | 1:09:59 | 1:03:22 | 0:06:37 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:47:35.131888 osd.7 (osd.7) 147 : cluster [ERR] osd.7 pg 2.10s1 found obsolete rollback obj 1#2:08020e03:::benchmark_data_smithi110_41149_object9182:head#d5d generation < trimmed_to 188'5167...repaired" in cluster log |
||||||||||||||
| fail | 3621557 |
|
2019-02-21 07:02:31 | 2019-02-21 08:23:25 | 2019-02-21 09:31:25 | 1:08:00 | 1:01:38 | 0:06:22 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:48:49.535357 osd.4 (osd.4) 112 : cluster [ERR] osd.4 pg 2.6s0 found obsolete rollback obj 0#2:6000afe4:::benchmark_data_smithi099_8956_object15052:head#8f2 generation < trimmed_to 127'2651...repaired" in cluster log |
||||||||||||||
| pass | 3621558 |
|
2019-02-21 07:02:31 | 2019-02-21 08:23:25 | 2019-02-21 09:29:25 | 1:06:00 | 0:58:31 | 0:07:29 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
| fail | 3621559 |
|
2019-02-21 07:02:31 | 2019-02-21 08:23:25 | 2019-02-21 09:29:25 | 1:06:00 | 0:59:39 | 0:06:21 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:46:23.411347 osd.2 (osd.2) 70 : cluster [ERR] osd.2 pg 2.ds1 found obsolete rollback obj 1#2:b001341b:::benchmark_data_smithi049_40759_object4065:head#971 generation < trimmed_to 99'2840...repaired" in cluster log |
||||||||||||||
| fail | 3621560 |
|
2019-02-21 07:02:31 | 2019-02-21 08:23:31 | 2019-02-21 09:29:32 | 1:06:01 | 0:58:28 | 0:07:33 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:47:52.602268 osd.1 (osd.1) 123 : cluster [ERR] osd.1 pg 2.ds0 found obsolete rollback obj 0#2:b0008bc8:::benchmark_data_smithi085_41320_object67527:head#29f8 generation < trimmed_to 153'11484...repaired" in cluster log |
||||||||||||||
| fail | 3621561 |
|
2019-02-21 07:02:31 | 2019-02-21 08:23:32 | 2019-02-21 09:21:33 | 0:58:01 | 0:52:17 | 0:05:44 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:46:15.466760 osd.0 (osd.0) 102 : cluster [ERR] osd.0 pg 2.fs0 found obsolete rollback obj 0#2:f001769a:::benchmark_data_smithi170_9108_object981:head#76a generation < trimmed_to 127'2623...repaired" in cluster log |
||||||||||||||
| fail | 3621562 |
|
2019-02-21 07:02:31 | 2019-02-21 08:25:36 | 2019-02-21 09:31:36 | 1:06:00 | 0:58:25 | 0:07:35 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:54:38.609411 osd.0 (osd.0) 190 : cluster [ERR] osd.0 pg 2.8s2 found obsolete rollback obj 2#2:1001c0f2:::benchmark_data_smithi158_40970_object19231:head#1070 generation < trimmed_to 186'5748...repaired" in cluster log |
||||||||||||||
| fail | 3621563 |
|
2019-02-21 07:02:31 | 2019-02-21 08:25:37 | 2019-02-21 09:37:37 | 1:12:00 | 1:05:08 | 0:06:52 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:51:10.370271 osd.2 (osd.2) 160 : cluster [ERR] osd.2 pg 2.16s2 found obsolete rollback obj 2#2:6813d1c3:::benchmark_data_smithi125_9401_object5958:head#1050 generation < trimmed_to 116'4206...repaired" in cluster log |
||||||||||||||
| fail | 3621564 |
|
2019-02-21 07:02:31 | 2019-02-21 08:25:37 | 2019-02-21 09:45:37 | 1:20:00 | 1:12:46 | 0:07:14 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:53:54.660142 osd.1 (osd.1) 159 : cluster [ERR] osd.1 pg 2.22s2 found obsolete rollback obj 2#2:44016af9:::benchmark_data_smithi023_9400_object25295:head#e08 generation < trimmed_to 199'3966...repaired" in cluster log |
||||||||||||||
| fail | 3621565 |
|
2019-02-21 07:02:31 | 2019-02-21 08:25:41 | 2019-02-21 09:45:42 | 1:20:01 | 1:13:14 | 0:06:47 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:06:53.827268 osd.6 (osd.6) 414 : cluster [ERR] osd.6 pg 3.1s0 found obsolete rollback obj 0#3:80028256:::benchmark_data_smithi068_93241_object68059:head#14d5 generation < trimmed_to 438'5575...repaired" in cluster log |
||||||||||||||
| fail | 3621566 |
|
2019-02-21 07:02:31 | 2019-02-21 08:25:44 | 2019-02-21 09:35:45 | 1:10:01 | 1:02:31 | 0:07:30 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:48:25.948024 osd.6 (osd.6) 61 : cluster [ERR] osd.6 pg 2.cs0 found obsolete rollback obj 0#2:3c336ecb:::benchmark_data_smithi187_41502_object353:head#131f generation < trimmed_to 89'4924...repaired" in cluster log |
||||||||||||||
| fail | 3621567 |
|
2019-02-21 07:02:31 | 2019-02-21 08:27:26 | 2019-02-21 09:47:27 | 1:20:01 | 1:12:42 | 0:07:19 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:50:58.369755 osd.3 (osd.3) 95 : cluster [ERR] osd.3 pg 2.1as0 found obsolete rollback obj 0#2:59690bdf:::benchmark_data_smithi191_41109_object86:head#96b generation < trimmed_to 109'2414...repaired" in cluster log |
||||||||||||||
| fail | 3621568 |
|
2019-02-21 07:02:31 | 2019-02-21 08:27:27 | 2019-02-21 09:37:27 | 1:10:00 | 1:03:38 | 0:06:22 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:59:55.723080 osd.5 (osd.5) 263 : cluster [ERR] osd.5 pg 3.es2 found obsolete rollback obj 2#3:7005fe55:::benchmark_data_smithi075_63717_object3825:head#8fb generation < trimmed_to 322'3111...repaired" in cluster log |
||||||||||||||
| fail | 3621569 |
|
2019-02-21 07:02:31 | 2019-02-21 08:27:27 | 2019-02-21 09:49:28 | 1:22:01 | 1:14:23 | 0:07:38 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:52:07.220967 osd.2 (osd.2) 83 : cluster [ERR] osd.2 pg 2.es0 found obsolete rollback obj 0#2:7001ce0d:::benchmark_data_smithi190_41180_object23365:head#156f generation < trimmed_to 104'5680...repaired" in cluster log |
||||||||||||||
| fail | 3621570 |
|
2019-02-21 07:02:31 | 2019-02-21 08:27:36 | 2019-02-21 09:39:36 | 1:12:00 | 1:05:50 | 0:06:10 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:14:24.419240 osd.4 (osd.4) 626 : cluster [ERR] osd.4 pg 5.es0 found obsolete rollback obj 0#5:70006950:::benchmark_data_smithi144_180360_object21334:head#1615 generation < trimmed_to 650'8545...repaired" in cluster log |
||||||||||||||
| fail | 3621571 |
|
2019-02-21 07:02:31 | 2019-02-21 08:29:44 | 2019-02-21 09:35:43 | 1:05:59 | 0:59:27 | 0:06:32 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:53:14.700519 osd.1 (osd.1) 110 : cluster [ERR] osd.1 pg 2.ds0 found obsolete rollback obj 0#2:b0007f2b:::benchmark_data_smithi183_40829_object10036:head#a55 generation < trimmed_to 148'3214...repaired" in cluster log |
||||||||||||||
| fail | 3621572 |
|
2019-02-21 07:02:31 | 2019-02-21 08:31:38 | 2019-02-21 09:51:38 | 1:20:00 | 1:13:23 | 0:06:37 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:53:38.494786 osd.6 (osd.6) 30 : cluster [ERR] osd.6 pg 2.19s1 found obsolete rollback obj 1#2:9fec46f3:::benchmark_data_smithi001_9386_object96:head#7c7 generation < trimmed_to 89'1993...repaired" in cluster log |
||||||||||||||
| fail | 3621573 |
|
2019-02-21 07:02:31 | 2019-02-21 08:31:38 | 2019-02-21 09:55:38 | 1:24:00 | 1:17:13 | 0:06:47 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:54:37.668140 osd.0 (osd.0) 70 : cluster [ERR] osd.0 pg 2.8s0 found obsolete rollback obj 0#2:100017d2:::benchmark_data_smithi038_41180_object29889:head#1a02 generation < trimmed_to 93'6859...repaired" in cluster log |
||||||||||||||
| fail | 3621574 |
|
2019-02-21 07:02:31 | 2019-02-21 08:31:40 | 2019-02-21 09:33:39 | 1:01:59 | 0:54:40 | 0:07:19 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:52:58.190486 osd.3 (osd.3) 71 : cluster [ERR] osd.3 pg 2.bs0 found obsolete rollback obj 0#2:d001fea2:::benchmark_data_smithi089_8930_object4804:head#1159 generation < trimmed_to 99'4768...repaired" in cluster log |
||||||||||||||
| fail | 3621575 |
|
2019-02-21 07:02:31 | 2019-02-21 08:31:54 | 2019-02-21 09:41:54 | 1:10:00 | 1:04:08 | 0:05:52 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:55:44.592799 osd.7 (osd.7) 163 : cluster [ERR] osd.7 pg 2.8s2 found obsolete rollback obj 2#2:1000d2c0:::benchmark_data_smithi148_8874_object47348:head#1647 generation < trimmed_to 198'6492...repaired" in cluster log |
||||||||||||||
| fail | 3621576 |
|
2019-02-21 07:02:31 | 2019-02-21 08:35:31 | 2019-02-21 09:35:30 | 0:59:59 | 0:52:54 | 0:07:05 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:57:06.577266 osd.7 (osd.7) 71 : cluster [ERR] osd.7 pg 2.9s2 found obsolete rollback obj 2#2:900217cb:::benchmark_data_smithi161_40940_object11199:head#11b5 generation < trimmed_to 96'4642...repaired" in cluster log |
||||||||||||||
| fail | 3621577 |
|
2019-02-21 07:02:31 | 2019-02-21 08:35:31 | 2019-02-21 09:43:31 | 1:08:00 | 1:00:13 | 0:07:47 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:57:06.657117 osd.0 (osd.0) 67 : cluster [ERR] osd.0 pg 2.1cs0 found obsolete rollback obj 0#2:38015c41:::benchmark_data_smithi119_41287_object15960:head#a84 generation < trimmed_to 97'2841...repaired" in cluster log |
||||||||||||||
| fail | 3621578 |
|
2019-02-21 07:02:31 | 2019-02-21 08:37:29 | 2019-02-21 09:39:29 | 1:02:00 | 0:55:53 | 0:06:07 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:01:49.614976 osd.0 (osd.0) 130 : cluster [ERR] osd.0 pg 2.12s0 found obsolete rollback obj 0#2:4800974a:::benchmark_data_smithi163_9086_object47678:head#f22 generation < trimmed_to 154'4055...repaired" in cluster log |
||||||||||||||
| fail | 3621579 |
|
2019-02-21 07:02:31 | 2019-02-21 08:37:29 | 2019-02-21 09:47:29 | 1:10:00 | 1:03:51 | 0:06:09 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 08:59:28.619204 osd.3 (osd.3) 62 : cluster [ERR] osd.3 pg 2.bs0 found obsolete rollback obj 0#2:d006266a:::benchmark_data_smithi182_9190_object4212:head#d09 generation < trimmed_to 126'3412...repaired" in cluster log |
||||||||||||||
| pass | 3621580 |
|
2019-02-21 07:02:31 | 2019-02-21 08:37:38 | 2019-02-21 10:05:39 | 1:28:01 | 1:20:35 | 0:07:26 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
| fail | 3621581 |
|
2019-02-21 07:02:31 | 2019-02-21 08:37:48 | 2019-02-21 09:47:48 | 1:10:00 | 1:02:23 | 0:07:37 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:00:46.704435 osd.3 (osd.3) 102 : cluster [ERR] osd.3 pg 2.es1 found obsolete rollback obj 1#2:70024ef2:::benchmark_data_smithi062_40926_object6787:head#cdb generation < trimmed_to 115'3540...repaired" in cluster log |
||||||||||||||
| fail | 3621582 |
|
2019-02-21 07:02:31 | 2019-02-21 08:39:27 | 2019-02-21 09:37:28 | 0:58:01 | 0:50:15 | 0:07:46 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:00:21.814255 osd.7 (osd.7) 14 : cluster [ERR] osd.7 pg 2.16s2 found obsolete rollback obj 2#2:6a29e547:::benchmark_data_smithi129_41355_object201:head#860 generation < trimmed_to 109'2148...repaired" in cluster log |
||||||||||||||
| fail | 3621583 |
|
2019-02-21 07:02:31 | 2019-02-21 08:39:27 | 2019-02-21 09:55:28 | 1:16:01 | 1:08:43 | 0:07:18 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:03:01.753674 osd.3 (osd.3) 9 : cluster [ERR] osd.3 pg 2.10s0 found obsolete rollback obj 0#2:080a587e:::benchmark_data_smithi103_41395_object958:head#1605 generation < trimmed_to 109'5709...repaired" in cluster log |
||||||||||||||
| fail | 3621584 |
|
2019-02-21 07:02:31 | 2019-02-21 08:39:28 | 2019-02-21 09:53:33 | 1:14:05 | 1:07:06 | 0:06:59 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:09:07.205139 osd.2 (osd.2) 268 : cluster [ERR] osd.2 pg 3.8s0 found obsolete rollback obj 0#3:1001dfea:::benchmark_data_smithi095_98792_object32229:head#fd0 generation < trimmed_to 296'4160...repaired" in cluster log |
||||||||||||||
| fail | 3621585 |
|
2019-02-21 07:02:31 | 2019-02-21 08:39:28 | 2019-02-21 09:39:28 | 1:00:00 | 0:52:18 | 0:07:42 | smithi | master | rhel | 7.5 | rados:thrash-erasure-code/{ceph.yaml clusters/{fixed-2.yaml openstack.yaml} fast/fast.yaml leveldb.yaml msgr-failures/osd-delay.yaml objectstore/filestore-xfs.yaml rados.yaml recovery-overrides/{default.yaml} supported-random-distro$/{rhel_latest.yaml} thrashers/morepggrow.yaml thrashosds-health.yaml workloads/ec-radosbench.yaml} | 2 |
Failure Reason:
"2019-02-21 09:00:49.373720 osd.7 (osd.7) 4 : cluster [ERR] osd.7 pg 2.1es1 found obsolete rollback obj 1#2:780a6a8e:::benchmark_data_smithi081_41383_object2666:head#48a generation < trimmed_to 119'1173...repaired" in cluster log |
||||||||||||||
Pulpito