Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
fail 726273 2015-01-28 15:07:31 2015-01-28 15:08:56 2015-01-28 15:40:52 0:31:56 0:26:38 0:05:18 plana,mira,burnupi wip-10057 ubuntu rados/multimon/{clusters/6.yaml msgr-failures/few.yaml tasks/mon_clock_no_skews.yaml} 2
Failure Reason:

'wait_until_healthy'reached maximum tries (150) after waiting for 900 seconds

fail 726274 2015-01-28 15:07:32 2015-01-28 15:08:56 2015-01-28 16:46:58 1:38:02 0:29:24 1:08:38 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:18:50.455962 mon.0 10.214.131.9:6789/0 21 : cluster [WRN] mon.1 10.214.136.6:6789/0 clock skew 3.72749s > max 0.5s" in cluster log

pass 726275 2015-01-28 15:07:32 2015-01-28 15:09:00 2015-01-28 16:17:05 1:08:05 0:39:58 0:28:07 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
fail 726276 2015-01-28 15:07:33 2015-01-28 15:08:56 2015-01-28 16:32:58 1:24:02 0:28:24 0:55:38 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 08:04:30.278152 mon.0 10.214.132.17:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.988812s in the future, clocks not synchronized" in cluster log

fail 726277 2015-01-28 15:07:34 2015-01-28 15:08:58 2015-01-28 16:09:02 1:00:04 0:25:14 0:34:50 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2015-01-28 07:43:31.968608 mon.0 10.214.132.26:6789/0 9 : cluster [WRN] mon.1 10.214.133.33:6789/0 clock skew 1.91895s > max 0.5s" in cluster log

fail 726278 2015-01-28 15:07:34 2015-01-28 15:08:56 2015-01-28 15:46:59 0:38:03 0:25:12 0:12:51 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2015-01-28 07:23:07.567642 mon.0 10.214.135.34:6789/0 10 : cluster [WRN] mon.1 10.214.136.14:6789/0 clock skew 0.698553s > max 0.5s" in cluster log

fail 726279 2015-01-28 15:07:35 2015-01-28 15:08:56 2015-01-28 15:40:54 0:31:58 0:27:17 0:04:41 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/force-sync-many.yaml workloads/pool-create-delete.yaml} 2
Failure Reason:

"2015-01-28 07:12:25.373281 mon.0 10.214.131.23:6789/0 9 : cluster [WRN] mon.5 10.214.133.37:6791/0 clock skew 0.599072s > max 0.5s" in cluster log

fail 726280 2015-01-28 15:07:36 2015-01-28 15:08:57 2015-01-28 17:17:07 2:08:10 0:27:19 1:40:51 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2015-01-28 08:49:29.615147 mon.0 10.214.132.8:6789/0 3 : cluster [WRN] message from mon.2 was stamped 3.498789s in the future, clocks not synchronized" in cluster log

fail 726281 2015-01-28 15:07:36 2015-01-28 15:09:01 2015-01-28 16:19:06 1:10:05 0:25:49 0:44:16 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2015-01-28 07:53:57.625472 mon.0 10.214.131.13:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.559457s in the future, clocks not synchronized" in cluster log

fail 726282 2015-01-28 15:07:37 2015-01-28 15:09:29 2015-01-28 15:51:32 0:42:03 0:29:28 0:12:35 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2015-01-28 07:25:52.697064 mon.1 10.214.136.28:6789/0 2 : cluster [WRN] message from mon.0 was stamped 1.153465s in the future, clocks not synchronized" in cluster log

fail 726283 2015-01-28 15:07:37 2015-01-28 15:08:56 2015-01-28 16:40:58 1:32:02 0:24:18 1:07:44 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2015-01-28 08:13:10.009790 mon.0 10.214.131.4:6789/0 7 : cluster [WRN] mon.2 10.214.131.32:6790/0 clock skew 2.22799s > max 0.5s" in cluster log

pass 726284 2015-01-28 15:07:38 2015-01-28 15:09:02 2015-01-28 15:19:02 0:10:00 0:08:15 0:01:45 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/dump-stuck.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
pass 726285 2015-01-28 15:07:39 2015-01-28 15:09:34 2015-01-28 16:07:38 0:58:04 0:47:51 0:10:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
fail 726286 2015-01-28 15:07:39 2015-01-28 15:09:31 2015-01-28 16:13:36 1:04:05 0:25:36 0:38:29 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2015-01-28 07:47:48.897866 mon.0 10.214.131.9:6789/0 3 : cluster [WRN] message from mon.1 was stamped 3.246322s in the future, clocks not synchronized" in cluster log

pass 726287 2015-01-28 15:07:40 2015-01-28 15:10:05 2015-01-28 15:36:07 0:26:02 0:06:16 0:19:46 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml workloads/rados_5925.yaml} 2
pass 726288 2015-01-28 15:07:41 2015-01-28 15:09:53 2015-01-28 15:37:55 0:28:02 0:20:03 0:07:59 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-small-objects.yaml} 2
fail 726289 2015-01-28 15:07:41 2015-01-28 15:09:21 2015-01-28 16:11:25 1:02:04 0:25:51 0:36:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2015-01-28 07:43:08.270046 mon.0 10.214.131.4:6789/0 6 : cluster [WRN] mon.1 10.214.133.37:6789/0 clock skew 2.12426s > max 0.5s" in cluster log

fail 726290 2015-01-28 15:07:42 2015-01-28 15:08:56 2015-01-28 16:00:59 0:52:03 0:25:20 0:26:43 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2015-01-28 07:35:52.159528 mon.0 10.214.131.11:6789/0 3 : cluster [WRN] message from mon.2 was stamped 1.469218s in the future, clocks not synchronized" in cluster log

fail 726291 2015-01-28 15:07:42 2015-01-28 15:10:12 2015-01-28 16:14:17 1:04:05 0:24:45 0:39:20 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2015-01-28 07:50:44.958434 mon.0 10.214.134.104:6789/0 10 : cluster [WRN] mon.1 10.214.134.108:6789/0 clock skew 1.87731s > max 0.5s" in cluster log

pass 726292 2015-01-28 15:07:43 2015-01-28 15:09:09 2015-01-28 16:31:15 1:22:06 1:06:16 0:15:50 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
fail 726293 2015-01-28 15:07:44 2015-01-28 15:09:02 2015-01-28 15:45:05 0:36:03 0:24:47 0:11:16 plana,mira,burnupi wip-10057 ubuntu rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml} 2
Failure Reason:

"2015-01-28 07:20:05.403927 mon.0 10.214.136.6:6789/0 7 : cluster [WRN] mon.1 10.214.137.128:6789/0 clock skew 0.640142s > max 0.5s" in cluster log

fail 726294 2015-01-28 15:07:44 2015-01-28 15:09:05 2015-01-28 16:01:09 0:52:04 0:27:42 0:24:22 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2015-01-28 07:31:23.521385 mon.0 10.214.131.32:6789/0 3 : cluster [WRN] message from mon.1 was stamped 2.215374s in the future, clocks not synchronized" in cluster log

pass 726295 2015-01-28 15:07:45 2015-01-28 15:08:57 2015-01-28 15:16:57 0:08:00 0:07:05 0:00:55 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/ec-lost-unfound.yaml fs/btrfs.yaml msgr-failures/few.yaml} 1
pass 726296 2015-01-28 15:07:46 2015-01-28 15:08:56 2015-01-28 17:15:03 2:06:07 0:56:49 1:09:18 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
dead 726297 2015-01-28 15:07:46 2015-01-28 15:08:59 2015-01-28 17:57:12 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml workloads/rados_api_tests.yaml} 2
fail 726298 2015-01-28 15:07:47 2015-01-28 15:09:00 2015-01-28 17:47:12 2:38:12 0:27:55 2:10:17 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2015-01-28 09:19:11.503425 mon.0 10.214.131.15:6789/0 9 : cluster [WRN] mon.1 10.214.132.4:6789/0 clock skew 1.85347s > max 0.5s" in cluster log

fail 726299 2015-01-28 15:07:48 2015-01-28 15:08:56 2015-01-28 16:27:02 1:18:06 0:25:56 0:52:10 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
Failure Reason:

"2015-01-28 08:02:11.647995 mon.0 10.214.131.2:6789/0 3 : cluster [WRN] message from mon.2 was stamped 1.369657s in the future, clocks not synchronized" in cluster log

pass 726300 2015-01-28 15:07:48 2015-01-28 15:08:56 2015-01-28 17:19:02 2:10:06 1:08:17 1:01:49 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
dead 726301 2015-01-28 15:07:49 2015-01-28 15:08:59 2015-01-28 17:57:12 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/objectstore/ceph_objectstore_tool.yaml 1
fail 726302 2015-01-28 15:07:49 2015-01-28 15:09:38 2015-01-28 16:19:43 1:10:05 0:25:29 0:44:36 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 07:54:59.702803 mon.0 10.214.131.35:6789/0 6 : cluster [WRN] mon.1 10.214.134.8:6789/0 clock skew 1.15306s > max 0.5s" in cluster log

fail 726303 2015-01-28 15:07:50 2015-01-28 15:09:58 2015-01-28 17:38:09 2:28:11 0:25:30 2:02:41 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2015-01-28 09:13:41.924424 mon.0 10.214.132.20:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.652427s in the future, clocks not synchronized" in cluster log

pass 726304 2015-01-28 15:07:51 2015-01-28 15:09:40 2015-01-28 17:33:51 2:24:11 0:44:31 1:39:40 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
fail 726305 2015-01-28 15:07:51 2015-01-28 15:09:15 2015-01-28 15:45:18 0:36:03 0:24:11 0:11:52 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml workloads/rados_mon_workunits.yaml} 2
Failure Reason:

"2015-01-28 07:20:26.889847 mon.0 10.214.135.110:6789/0 7 : cluster [WRN] mon.1 10.214.136.30:6789/0 clock skew 0.914992s > max 0.5s" in cluster log

fail 726306 2015-01-28 15:07:52 2015-01-28 15:09:23 2015-01-28 15:47:25 0:38:02 0:24:35 0:13:27 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2015-01-28 07:22:05.749184 mon.0 10.214.131.2:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.308349s in the future, clocks not synchronized" in cluster log

pass 726307 2015-01-28 15:07:53 2015-01-28 15:08:56 2015-01-28 15:18:53 0:09:57 0:08:56 0:01:01 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/lost-unfound-delete.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
fail 726308 2015-01-28 15:07:53 2015-01-28 15:08:56 2015-01-28 16:29:01 1:20:05 0:27:17 0:52:48 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2015-01-28 08:01:21.602336 mon.0 10.214.131.24:6789/0 3 : cluster [WRN] message from mon.2 was stamped 1.480931s in the future, clocks not synchronized" in cluster log

dead 726309 2015-01-28 15:07:54 2015-01-28 15:09:01 2015-01-28 17:57:14 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/small-objects.yaml}
dead 726310 2015-01-28 15:07:54 2015-01-28 15:09:13 2015-01-28 17:57:27 2:48:14 plana,mira,burnupi wip-10057 ubuntu rados/multimon/{clusters/9.yaml msgr-failures/many.yaml tasks/mon_clock_with_skews.yaml}
pass 726311 2015-01-28 15:07:55 2015-01-28 15:08:56 2015-01-28 16:22:59 1:14:03 0:47:26 0:26:37 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 726312 2015-01-28 15:07:56 2015-01-28 15:08:56 2015-01-28 15:28:55 0:19:59 0:19:04 0:00:55 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/ec-small-objects.yaml} 2
fail 726313 2015-01-28 15:07:56 2015-01-28 15:09:19 2015-01-28 16:59:27 1:50:08 0:24:02 1:26:06 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2015-01-28 08:34:33.191702 mon.0 10.214.131.18:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.272506s in the future, clocks not synchronized" in cluster log

fail 726314 2015-01-28 15:07:57 2015-01-28 15:09:11 2015-01-28 16:47:19 1:38:08 0:28:57 1:09:11 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 08:18:53.832950 mon.0 10.214.134.108:6789/0 3 : cluster [WRN] message from mon.5 was stamped 0.728713s in the future, clocks not synchronized" in cluster log

fail 726315 2015-01-28 15:07:58 2015-01-28 15:09:07 2015-01-28 17:11:17 2:02:10 0:28:05 1:34:05 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2015-01-28 08:43:00.791395 mon.0 10.214.131.6:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.078594s in the future, clocks not synchronized" in cluster log

dead 726316 2015-01-28 15:07:58 2015-01-28 15:10:02 2015-01-28 17:58:15 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml}
fail 726317 2015-01-28 15:07:59 2015-01-28 15:10:09 2015-01-28 16:26:14 1:16:05 0:24:07 0:51:58 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:02:07.476766 mon.0 10.214.132.37:6789/0 3 : cluster [WRN] message from mon.2 was stamped 0.935845s in the future, clocks not synchronized" in cluster log

fail 726318 2015-01-28 15:07:59 2015-01-28 15:10:10 2015-01-28 16:42:17 1:32:07 0:24:10 1:07:57 plana,mira,burnupi wip-10057 ubuntu rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/many.yaml tasks/rados_python.yaml} 2
Failure Reason:

"2015-01-28 08:17:44.816955 mon.0 10.214.134.26:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.480736s in the future, clocks not synchronized" in cluster log

pass 726319 2015-01-28 15:08:00 2015-01-28 15:09:45 2015-01-28 15:19:45 0:10:00 0:08:25 0:01:35 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/lost-unfound.yaml fs/btrfs.yaml msgr-failures/few.yaml} 1
pass 726320 2015-01-28 15:08:01 2015-01-28 15:09:43 2015-01-28 16:47:50 1:38:07 1:09:31 0:28:36 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 726321 2015-01-28 15:08:01 2015-01-28 15:09:55 2015-01-28 15:59:59 0:50:04 0:26:12 0:23:52 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2015-01-28 07:34:12.748478 mon.0 10.214.131.30:6789/0 6 : cluster [WRN] mon.1 10.214.132.5:6789/0 clock skew 2.13221s > max 0.5s" in cluster log

fail 726322 2015-01-28 15:08:02 2015-01-28 15:09:36 2015-01-28 16:09:40 1:00:04 0:43:33 0:16:31 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.0 restart

fail 726323 2015-01-28 15:08:03 2015-01-28 15:09:27 2015-01-28 17:01:35 1:52:08 0:25:39 1:26:29 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2015-01-28 08:35:30.399031 mon.0 10.214.131.34:6789/0 3 : cluster [WRN] message from mon.2 was stamped 0.758883s in the future, clocks not synchronized" in cluster log

fail 726324 2015-01-28 15:08:03 2015-01-28 15:09:49 2015-01-28 16:53:57 1:44:08 0:30:33 1:13:35 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml workloads/pool-create-delete.yaml} 2
Failure Reason:

"2015-01-28 08:22:48.104954 mon.0 10.214.132.29:6789/0 5 : cluster [WRN] message from mon.1 was stamped 5.376032s in the future, clocks not synchronized" in cluster log

fail 726325 2015-01-28 15:08:04 2015-01-28 15:10:07 2015-01-28 15:46:09 0:36:02 0:30:04 0:05:58 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2015-01-28 07:15:06.237332 mon.0 10.214.132.29:6789/0 5 : cluster [WRN] message from mon.1 was stamped 5.340375s in the future, clocks not synchronized" in cluster log

fail 726326 2015-01-28 15:08:05 2015-01-28 15:08:56 2015-01-28 16:37:01 1:28:05 0:25:52 1:02:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
Failure Reason:

"2015-01-28 08:10:26.061834 mon.0 10.214.133.33:6789/0 3 : cluster [WRN] message from mon.1 was stamped 2.206292s in the future, clocks not synchronized" in cluster log

dead 726327 2015-01-28 15:08:05 2015-01-28 15:09:59 2015-01-28 17:58:12 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 726328 2015-01-28 15:08:06 2015-01-28 15:09:25 2015-01-28 15:47:28 0:38:03 0:26:03 0:12:00 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2015-01-28 07:19:22.760450 mon.0 10.214.131.31:6789/0 3 : cluster [WRN] message from mon.2 was stamped 1.679495s in the future, clocks not synchronized" in cluster log

fail 726329 2015-01-28 15:08:06 2015-01-28 15:09:51 2015-01-28 17:44:03 2:34:12 0:27:54 2:06:18 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2015-01-28 09:15:28.802976 mon.0 10.214.131.24:6789/0 3 : cluster [WRN] message from mon.1 was stamped 3.156854s in the future, clocks not synchronized" in cluster log

pass 726330 2015-01-28 15:08:07 2015-01-28 15:10:04 2015-01-28 15:22:04 0:12:00 0:10:49 0:01:11 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/mon-config-keys.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
dead 726331 2015-01-28 15:08:08 2015-01-28 15:08:58 2015-01-28 17:57:11 2:48:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
pass 726332 2015-01-28 15:08:08 2015-01-28 15:09:17 2015-01-28 15:17:17 0:08:00 0:06:32 0:01:28 plana,mira,burnupi wip-10057 ubuntu rados/objectstore/filejournal.yaml 1
fail 726333 2015-01-28 15:08:09 2015-01-28 15:09:47 2015-01-28 15:43:49 0:34:02 0:28:33 0:05:29 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml workloads/rados_5925.yaml} 2
Failure Reason:

"2015-01-28 07:15:40.261281 mon.0 10.214.131.9:6789/0 3 : cluster [WRN] message from mon.7 was stamped 2.938791s in the future, clocks not synchronized" in cluster log

fail 726334 2015-01-28 15:08:10 2015-01-28 15:10:45 2015-01-28 16:04:49 0:54:04 0:35:36 0:18:28 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2015-01-28 07:32:48.115312 mon.0 10.214.132.34:6789/0 3 : cluster [WRN] message from mon.1 was stamped 5.192653s in the future, clocks not synchronized" in cluster log

fail 726335 2015-01-28 15:08:10 2015-01-28 15:11:45 2015-01-28 15:51:48 0:40:03 0:25:34 0:14:29 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2015-01-28 07:28:21.111570 mon.0 10.214.134.22:6789/0 10 : cluster [WRN] mon.1 10.214.136.20:6789/0 clock skew 0.784915s > max 0.5s" in cluster log

dead 726336 2015-01-28 15:08:11 2015-01-28 15:12:09 2015-01-28 17:58:22 2:46:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/ec-small-objects.yaml} 2
dead 726337 2015-01-28 15:08:11 2015-01-28 15:17:00 2015-01-28 17:57:13 2:40:13 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
dead 726338 2015-01-28 15:08:12 2015-01-28 15:17:20 2015-01-28 17:57:32 2:40:12 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
dead 726339 2015-01-28 15:08:13 2015-01-28 15:18:01 2015-01-28 17:58:13 2:40:12 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 726340 2015-01-28 15:08:13 2015-01-28 15:18:16 2015-01-28 15:58:19 0:40:03 0:28:43 0:11:20 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2015-01-28 07:33:00.972569 mon.0 10.214.134.136:6789/0 7 : cluster [WRN] mon.1 10.214.135.32:6789/0 clock skew 0.78795s > max 0.5s" in cluster log

pass 726341 2015-01-28 15:08:14 2015-01-28 15:18:56 2015-01-28 15:36:57 0:18:01 0:16:06 0:01:55 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/mon-thrasher.yaml fs/btrfs.yaml msgr-failures/few.yaml} 1
fail 726342 2015-01-28 15:08:15 2015-01-28 15:19:05 2015-01-28 15:59:08 0:40:03 0:29:44 0:10:19 plana,mira,burnupi wip-10057 ubuntu rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_stress_watch.yaml} 2
Failure Reason:

"2015-01-28 07:34:12.920012 mon.0 10.214.131.18:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.253666s in the future, clocks not synchronized" in cluster log

dead 726343 2015-01-28 15:08:15 2015-01-28 15:19:20 2015-01-28 17:57:33 2:38:13 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
fail 726344 2015-01-28 15:08:16 2015-01-28 15:19:33 2015-01-28 16:15:38 0:56:05 0:24:10 0:31:55 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 07:50:09.397355 mon.0 10.214.134.26:6789/0 3 : cluster [WRN] message from mon.2 was stamped 1.253594s in the future, clocks not synchronized" in cluster log

fail 726345 2015-01-28 15:08:16 2015-01-28 15:19:48 2015-01-28 16:15:53 0:56:05 0:27:11 0:28:54 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

"2015-01-28 07:47:23.187491 mon.0 10.214.131.23:6789/0 3 : cluster [WRN] message from mon.2 was stamped 3.473557s in the future, clocks not synchronized" in cluster log

fail 726346 2015-01-28 15:08:17 2015-01-28 15:21:49 2015-01-28 16:45:55 1:24:06 0:26:15 0:57:51 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2015-01-28 08:18:54.193797 mon.0 10.214.131.31:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.960948s in the future, clocks not synchronized" in cluster log

fail 726347 2015-01-28 15:08:18 2015-01-28 15:22:07 2015-01-28 16:00:09 0:38:02 0:26:44 0:11:18 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2015-01-28 07:34:40.459325 mon.0 10.214.132.17:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.868665s in the future, clocks not synchronized" in cluster log

fail 726348 2015-01-28 15:08:18 2015-01-28 15:22:38 2015-01-28 16:16:41 0:54:03 0:25:33 0:28:30 plana,mira,burnupi wip-10057 ubuntu rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
Failure Reason:

"2015-01-28 07:50:27.679436 mon.0 10.214.133.30:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.006864s in the future, clocks not synchronized" in cluster log

dead 726349 2015-01-28 15:08:19 2015-01-28 15:24:04 2015-01-28 17:58:16 2:34:12 plana,mira,burnupi wip-10057 ubuntu rados/multimon/{clusters/21.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml} 3
fail 726350 2015-01-28 15:08:20 2015-01-28 15:24:52 2015-01-28 17:19:01 1:54:09 0:27:50 1:26:19 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:50:40.576038 mon.0 10.214.131.9:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.773033s in the future, clocks not synchronized" in cluster log

fail 726351 2015-01-28 15:08:20 2015-01-28 15:27:10 2015-01-28 17:05:17 1:38:07 0:23:45 1:14:22 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2015-01-28 08:41:03.744755 mon.0 10.214.132.13:6789/0 7 : cluster [WRN] mon.1 10.214.132.20:6789/0 clock skew 1.23825s > max 0.5s" in cluster log

fail 726352 2015-01-28 15:08:21 2015-01-28 15:28:58 2015-01-28 16:03:00 0:34:02 0:28:19 0:05:43 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml workloads/rados_mon_workunits.yaml} 2
Failure Reason:

"2015-01-28 07:35:08.262885 mon.0 10.214.131.10:6789/0 8 : cluster [WRN] mon.1 10.214.135.20:6789/0 clock skew 0.669534s > max 0.5s" in cluster log

dead 726353 2015-01-28 15:08:21 2015-01-28 15:30:22 2015-01-28 17:58:33 2:28:11 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 726354 2015-01-28 15:08:22 2015-01-28 15:30:43 2015-01-28 17:38:53 2:08:10 0:50:45 1:17:25 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 726355 2015-01-28 15:08:23 2015-01-28 15:36:11 2015-01-28 15:44:11 0:08:00 0:07:18 0:00:42 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/osd-backfill.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
pass 726356 2015-01-28 15:08:23 2015-01-28 15:37:00 2015-01-28 16:49:05 1:12:05 1:01:45 0:10:20 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/ec-radosbench.yaml} 2
fail 726357 2015-01-28 15:08:24 2015-01-28 15:37:58 2015-01-28 17:34:07 1:56:09 0:44:13 1:11:56 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2015-01-28 08:48:06.967161 mon.0 10.214.131.31:6789/0 3 : cluster [WRN] message from mon.2 was stamped 5.368597s in the future, clocks not synchronized" in cluster log

fail 726358 2015-01-28 15:08:25 2015-01-28 15:39:00 2015-01-28 16:13:02 0:34:02 0:25:39 0:08:23 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2015-01-28 07:48:45.417982 mon.0 10.214.136.6:6789/0 7 : cluster [WRN] mon.1 10.214.137.128:6789/0 clock skew 0.805032s > max 0.5s" in cluster log

pass 726359 2015-01-28 15:08:25 2015-01-28 15:40:55 2015-01-28 16:41:00 1:00:05 0:55:37 0:04:28 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
fail 726360 2015-01-28 15:08:26 2015-01-28 15:40:57 2015-01-28 16:27:00 0:46:03 0:24:10 0:21:53 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 08:01:38.375296 mon.0 10.214.133.10:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.344647s in the future, clocks not synchronized" in cluster log

pass 726361 2015-01-28 15:08:26 2015-01-28 15:44:12 2015-01-28 16:46:13 1:02:01 0:19:20 0:42:41 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/ec-small-objects.yaml} 2
fail 726362 2015-01-28 15:08:27 2015-01-28 15:44:14 2015-01-28 16:18:16 0:34:02 0:29:53 0:04:09 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2015-01-28 07:48:51.325107 mon.0 10.214.132.29:6789/0 5 : cluster [WRN] message from mon.1 was stamped 5.468733s in the future, clocks not synchronized" in cluster log

fail 726363 2015-01-28 15:08:28 2015-01-28 15:45:08 2015-01-28 17:01:14 1:16:06 1:05:58 0:10:08 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 07:59:44.764323 mon.1 10.214.136.28:6789/0 10 : cluster [WRN] message from mon.0 was stamped 0.522503s in the future, clocks not synchronized" in cluster log

fail 726364 2015-01-28 15:08:28 2015-01-28 15:45:21 2015-01-28 16:17:23 0:32:02 0:26:06 0:05:56 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
Failure Reason:

"2015-01-28 07:49:09.021760 mon.0 10.214.131.31:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.834461s in the future, clocks not synchronized" in cluster log

pass 726365 2015-01-28 15:08:29 2015-01-28 15:46:13 2015-01-28 15:58:13 0:12:00 0:10:52 0:01:08 plana,mira,burnupi wip-10057 ubuntu rados/objectstore/filestore-idempotent-aio-journal.yaml 1
pass 726366 2015-01-28 15:08:30 2015-01-28 15:46:52 2015-01-28 15:56:52 0:10:00 0:09:53 0:00:07 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/osd-recovery-incomplete.yaml fs/btrfs.yaml msgr-failures/few.yaml} 1
fail 726367 2015-01-28 15:08:30 2015-01-28 15:47:02 2015-01-28 16:57:07 1:10:05 0:23:56 0:46:09 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/xfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:30:47.987495 mon.0 10.214.133.10:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.090095s in the future, clocks not synchronized" in cluster log

fail 726368 2015-01-28 15:08:31 2015-01-28 15:47:28 2015-01-28 16:43:32 0:56:04 0:32:14 0:23:50 plana,mira,burnupi wip-10057 ubuntu rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/many.yaml tasks/rados_workunit_loadgen_big.yaml} 2
Failure Reason:

"2015-01-28 08:10:42.656071 mon.0 10.214.132.34:6789/0 5 : cluster [WRN] message from mon.1 was stamped 5.186798s in the future, clocks not synchronized" in cluster log

fail 726369 2015-01-28 15:08:32 2015-01-28 15:47:31 2015-01-28 16:45:35 0:58:04 0:26:01 0:32:03 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:20:11.720212 mon.0 10.214.133.30:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.369583s in the future, clocks not synchronized" in cluster log

fail 726370 2015-01-28 15:08:32 2015-01-28 15:48:04 2015-01-28 16:58:07 1:10:03 0:07:38 1:02:25 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

No JSON object could be decoded

fail 726371 2015-01-28 15:08:33 2015-01-28 15:49:50 2015-01-28 16:31:53 0:42:03 0:29:16 0:12:47 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml workloads/pool-create-delete.yaml} 2
Failure Reason:

"2015-01-28 08:03:09.385230 mon.5 10.214.133.25:6791/0 2 : cluster [WRN] message from mon.0 was stamped 0.627095s in the future, clocks not synchronized" in cluster log

pass 726372 2015-01-28 15:08:33 2015-01-28 15:51:35 2015-01-28 16:49:39 0:58:04 0:31:14 0:26:50 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
fail 726373 2015-01-28 15:08:34 2015-01-28 15:51:52 2015-01-28 16:31:54 0:40:02 0:25:07 0:14:55 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

"2015-01-28 08:06:12.423157 mon.0 10.214.131.11:6789/0 3 : cluster [WRN] message from mon.2 was stamped 0.967828s in the future, clocks not synchronized" in cluster log

dead 726374 2015-01-28 15:08:35 2015-01-28 15:57:09 2015-01-28 17:57:16 2:00:07 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml}
dead 726375 2015-01-28 15:08:36 2015-01-28 15:58:16 2015-01-28 17:58:25 2:00:09 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps.yaml} 2
fail 726376 2015-01-28 15:08:36 2015-01-28 15:58:21 2015-01-28 16:28:23 0:30:02 0:24:23 0:05:39 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

"2015-01-28 08:02:21.484752 mon.1 10.214.132.5:6789/0 2 : cluster [WRN] message from mon.0 was stamped 0.988257s in the future, clocks not synchronized" in cluster log

dead 726377 2015-01-28 15:08:37 2015-01-28 15:59:08 2015-01-28 17:57:17 1:58:09 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
pass 726378 2015-01-28 15:08:37 2015-01-28 15:59:10 2015-01-28 16:11:11 0:12:01 0:09:03 0:02:58 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/osd-recovery.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
pass 726379 2015-01-28 15:08:38 2015-01-28 16:00:02 2015-01-28 16:48:06 0:48:04 0:07:15 0:40:49 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml workloads/rados_5925.yaml} 2
fail 726380 2015-01-28 15:08:39 2015-01-28 16:00:13 2015-01-28 17:18:18 1:18:05 0:30:41 0:47:24 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2015-01-28 08:47:09.056196 mon.0 10.214.132.34:6789/0 5 : cluster [WRN] message from mon.1 was stamped 5.633727s in the future, clocks not synchronized" in cluster log

fail 726381 2015-01-28 15:08:39 2015-01-28 16:01:02 2015-01-28 16:45:05 0:44:03 0:27:09 0:16:54 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 08:18:22.811472 mon.0 10.214.131.23:6789/0 3 : cluster [WRN] message from mon.2 was stamped 3.530303s in the future, clocks not synchronized" in cluster log

pass 726382 2015-01-28 15:08:40 2015-01-28 16:01:12 2015-01-28 16:47:15 0:46:03 0:43:16 0:02:47 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
fail 726383 2015-01-28 15:08:40 2015-01-28 16:03:03 2015-01-28 17:15:08 1:12:05 0:28:25 0:43:40 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2015-01-28 08:44:32.059925 mon.0 10.214.131.32:6789/0 3 : cluster [WRN] message from mon.1 was stamped 2.023872s in the future, clocks not synchronized" in cluster log

fail 726384 2015-01-28 15:08:41 2015-01-28 16:04:51 2015-01-28 16:42:54 0:38:03 0:26:30 0:11:33 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/ec-small-objects.yaml} 2
Failure Reason:

"2015-01-28 08:15:00.192308 mon.0 10.214.133.37:6789/0 5 : cluster [WRN] message from mon.1 was stamped 3.406520s in the future, clocks not synchronized" in cluster log

fail 726385 2015-01-28 15:08:42 2015-01-28 16:07:41 2015-01-28 17:53:49 1:46:08 0:26:04 1:20:04 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2015-01-28 09:30:23.257289 mon.0 10.214.134.14:6789/0 6 : cluster [WRN] mon.1 10.214.135.24:6789/0 clock skew 0.948528s > max 0.5s" in cluster log

pass 726386 2015-01-28 15:08:42 2015-01-28 16:09:05 2015-01-28 16:19:05 0:10:00 0:08:41 0:01:19 plana,mira,burnupi wip-10057 ubuntu rados/multimon/{clusters/3.yaml msgr-failures/many.yaml tasks/mon_clock_no_skews.yaml} 1
pass 726387 2015-01-28 15:08:43 2015-01-28 16:09:43 2015-01-28 17:03:47 0:54:04 0:09:11 0:44:53 plana,mira,burnupi wip-10057 ubuntu rados/singleton-nomsgr/all/multi-backfill-reject.yaml 2
pass 726388 2015-01-28 15:08:44 2015-01-28 16:11:13 2015-01-28 17:51:21 1:40:08 0:51:20 0:48:48 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml workloads/rados_api_tests.yaml} 2
fail 726389 2015-01-28 15:08:44 2015-01-28 16:11:28 2015-01-28 16:53:31 0:42:03 0:24:23 0:17:40 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
Failure Reason:

"2015-01-28 08:29:10.859644 mon.0 10.214.131.33:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.368862s in the future, clocks not synchronized" in cluster log

pass 726390 2015-01-28 15:08:45 2015-01-28 16:13:05 2015-01-28 16:19:05 0:06:00 0:05:49 0:00:11 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/peer.yaml fs/btrfs.yaml msgr-failures/few.yaml} 1
fail 726391 2015-01-28 15:08:46 2015-01-28 16:13:39 2015-01-28 17:39:46 1:26:07 0:50:11 0:35:56 plana,mira,burnupi wip-10057 ubuntu 14.04 rados/thrash-erasure-code-isa/{clusters/fixed-2.yaml distros/ubuntu_14.04.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.4 restart

fail 726392 2015-01-28 15:08:46 2015-01-28 16:14:20 2015-01-28 17:04:23 0:50:03 0:26:09 0:23:54 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
Failure Reason:

"2015-01-28 08:39:04.047011 mon.0 10.214.132.17:6789/0 5 : cluster [WRN] message from mon.1 was stamped 0.752465s in the future, clocks not synchronized" in cluster log

pass 726393 2015-01-28 15:08:47 2015-01-28 16:15:41 2015-01-28 17:47:48 1:32:07 0:25:04 1:07:03 plana,mira,burnupi wip-10057 ubuntu rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
fail 726394 2015-01-28 15:08:48 2015-01-28 16:15:56 2015-01-28 17:08:00 0:52:04 0:27:38 0:24:26 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
Failure Reason:

"2015-01-28 08:41:28.167424 mon.0 10.214.133.33:6789/0 3 : cluster [WRN] message from mon.1 was stamped 1.718400s in the future, clocks not synchronized" in cluster log

dead 726395 2015-01-28 15:08:48 2015-01-28 16:16:44 2015-01-28 17:56:51 1:40:07 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml}
fail 726396 2015-01-28 15:08:49 2015-01-28 16:17:08 2015-01-28 16:49:10 0:32:02 0:25:18 0:06:44 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
Failure Reason:

"2015-01-28 08:24:08.928264 mon.1 10.214.137.128:6789/0 2 : cluster [WRN] message from mon.0 was stamped 0.667152s in the future, clocks not synchronized" in cluster log

pass 726397 2015-01-28 15:08:50 2015-01-28 16:17:26 2015-01-28 17:37:32 1:20:06 0:42:22 0:37:44 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
dead 726398 2015-01-28 15:08:50 2015-01-28 16:18:20 2015-01-28 17:58:27 1:40:07 plana,mira,burnupi wip-10057 ubuntu rados/objectstore/filestore-idempotent.yaml 1
fail 726399 2015-01-28 15:08:51 2015-01-28 16:19:08 2015-01-28 16:53:11 0:34:03 0:23:52 0:10:11 plana,mira,burnupi wip-10057 ubuntu rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml} 2
Failure Reason:

"2015-01-28 08:29:34.424849 mon.0 10.214.132.37:6789/0 3 : cluster [WRN] message from mon.2 was stamped 0.694134s in the future, clocks not synchronized" in cluster log

dead 726400 2015-01-28 15:08:52 2015-01-28 16:19:10 2015-01-28 17:57:18 1:38:08 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 726401 2015-01-28 15:08:52 2015-01-28 16:19:12 2015-01-28 17:39:18 1:20:06 0:50:41 0:29:25 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
fail 726402 2015-01-28 15:08:53 2015-01-28 16:19:46 2015-01-28 17:01:49 0:42:03 0:27:49 0:14:14 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
Failure Reason:

"2015-01-28 08:34:53.340361 mon.0 10.214.133.23:6789/0 5 : cluster [WRN] message from mon.1 was stamped 1.738668s in the future, clocks not synchronized" in cluster log

fail 726403 2015-01-28 15:08:53 2015-01-28 16:21:11 2015-01-28 16:35:12 0:14:01 0:08:51 0:05:10 plana,mira,burnupi wip-10057 ubuntu rados/singleton/{all/pg-removal-interruption.yaml fs/btrfs.yaml msgr-failures/many.yaml} 1
Failure Reason:

Command failed on burnupi61 with status 1: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-0'

pass 726404 2015-01-28 15:08:54 2015-01-28 16:23:02 2015-01-28 17:27:06 1:04:04 0:44:07 0:19:57 plana,mira,burnupi wip-10057 ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
dead 726405 2015-01-28 15:08:55 2015-01-28 17:55:42 2015-01-28 17:57:41 0:01:59 plana,mira,burnupi wip-10057 ubuntu rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/ec-radosbench.yaml}