Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 111512 2014-03-01 05:56:36 2014-03-01 05:56:36 2014-03-01 06:09:37 0:13:01 0:05:36 0:07:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 111513 2014-03-01 05:57:12 2014-03-01 05:57:12 2014-03-01 06:39:54 0:42:42 0:33:18 0:09:24 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.1 restart

pass 111514 2014-03-01 05:58:05 2014-03-01 05:58:05 2014-03-01 07:23:10 1:25:05 0:29:10 0:55:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 111515 2014-03-01 05:59:05 2014-03-01 05:59:05 2014-03-01 08:25:40 2:26:35 0:34:19 1:52:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111516 2014-03-01 05:59:14 2014-03-01 05:59:14 2014-03-01 06:41:25 0:42:11 0:26:16 0:15:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111517 2014-03-01 05:59:16 2014-03-01 05:59:16 2014-03-01 06:57:08 0:57:52 0:17:10 0:40:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
fail 111518 2014-03-01 06:00:22 2014-03-01 06:00:22 2014-03-01 06:54:40 0:54:18 0:44:06 0:10:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

failed to recover before timeout expired

pass 111519 2014-03-01 06:00:26 2014-03-01 06:00:26 2014-03-01 07:13:20 1:12:54 0:31:53 0:41:01 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111520 2014-03-01 06:00:44 2014-03-01 06:00:44 2014-03-01 07:04:07 1:03:23 0:14:41 0:48:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111521 2014-03-01 06:00:47 2014-03-01 06:00:47 2014-03-01 06:40:15 0:39:28 0:34:31 0:04:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111522 2014-03-01 06:00:50 2014-03-01 06:00:50 2014-03-01 06:10:02 0:09:12 0:05:29 0:03:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111523 2014-03-01 06:01:12 2014-03-01 06:01:12 2014-03-01 06:47:13 0:46:01 0:40:03 0:05:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
dead 111524 2014-03-01 06:01:35 2014-03-01 06:01:35 2014-03-01 19:51:25 13:49:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml}
pass 111525 2014-03-01 06:02:24 2014-03-01 06:02:24 2014-03-01 06:47:30 0:45:06 0:34:37 0:10:29 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 111526 2014-03-01 06:02:25 2014-03-01 06:02:25 2014-03-01 06:46:57 0:44:32 0:25:33 0:18:59 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
pass 111527 2014-03-01 06:02:28 2014-03-01 06:02:28 2014-03-01 07:18:28 1:16:00 0:16:06 0:59:54 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
pass 111528 2014-03-01 06:04:34 2014-03-01 06:04:34 2014-03-01 06:50:19 0:45:45 0:41:55 0:03:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
pass 111529 2014-03-01 06:04:56 2014-03-01 06:04:56 2014-03-01 06:53:56 0:49:00 0:29:48 0:19:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 111530 2014-03-01 06:05:04 2014-03-01 06:05:04 2014-03-01 06:25:24 0:20:20 0:16:49 0:03:31 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111531 2014-03-01 06:05:15 2014-03-01 06:05:15 2014-03-01 07:15:03 1:09:48 0:32:32 0:37:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111532 2014-03-01 06:06:30 2014-03-01 06:06:30 2014-03-01 06:29:05 0:22:35 0:05:28 0:17:07 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111533 2014-03-01 06:06:55 2014-03-01 06:06:55 2014-03-01 06:58:32 0:51:37 0:48:15 0:03:22 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111534 2014-03-01 06:07:00 2014-03-01 06:07:00 2014-03-01 07:16:40 1:09:40 0:27:31 0:42:09 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111535 2014-03-01 06:08:23 2014-03-01 06:08:23 2014-03-01 06:53:27 0:45:04 0:40:18 0:04:46 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111536 2014-03-01 06:08:25 2014-03-01 06:08:25 2014-03-01 06:39:19 0:30:54 0:25:29 0:05:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
pass 111537 2014-03-01 06:09:29 2014-03-01 06:09:29 2014-03-01 07:01:46 0:52:17 0:17:19 0:34:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111538 2014-03-01 06:10:39 2014-03-01 06:10:39 2014-03-01 07:12:09 1:01:30 0:57:09 0:04:21 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
pass 111539 2014-03-01 06:10:45 2014-03-01 06:10:45 2014-03-01 07:04:09 0:53:24 0:49:45 0:03:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111540 2014-03-01 06:10:53 2014-03-01 06:10:53 2014-03-01 07:58:27 1:47:34 0:17:39 1:29:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111541 2014-03-01 06:12:24 2014-03-01 06:12:24 2014-03-01 08:00:39 1:48:15 0:34:25 1:13:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111542 2014-03-01 06:12:37 2014-03-01 06:12:37 2014-03-01 06:45:28 0:32:51 0:05:09 0:27:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111543 2014-03-01 06:12:39 2014-03-01 06:12:39 2014-03-01 06:55:40 0:43:01 0:34:26 0:08:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111544 2014-03-01 06:12:40 2014-03-01 06:12:40 2014-03-01 06:58:43 0:46:03 0:29:34 0:16:29 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111545 2014-03-01 06:13:17 2014-03-01 06:13:17 2014-03-01 07:44:40 1:31:23 0:32:28 0:58:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111546 2014-03-01 06:18:52 2014-03-01 06:18:52 2014-03-01 06:51:07 0:32:15 0:25:40 0:06:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
fail 111547 2014-03-01 06:19:33 2014-03-01 06:19:33 2014-03-01 06:48:29 0:28:56 0:18:07 0:10:49 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2014-02-28 22:44:10.192443 osd.0 10.214.133.37:6805/16393 62 : [ERR] 70.2 scrub stat mismatch, got 17/17 objects, 0/0 clones, 17/15 dirty, 0/0 whiteouts, 2860/2860 bytes." in cluster log

pass 111548 2014-03-01 06:21:31 2014-03-01 06:21:31 2014-03-01 07:52:05 1:30:34 0:53:18 0:37:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111549 2014-03-01 06:22:59 2014-03-01 06:22:59 2014-03-01 09:18:04 2:55:05 0:29:53 2:25:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111550 2014-03-01 06:25:30 2014-03-01 06:25:30 2014-03-01 09:18:47 2:53:17 0:16:27 2:36:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111551 2014-03-01 06:27:12 2014-03-01 06:27:12 2014-03-01 07:24:58 0:57:46 0:34:34 0:23:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111552 2014-03-01 06:30:34 2014-03-01 06:30:34 2014-03-01 06:53:03 0:22:29 0:05:31 0:16:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111553 2014-03-01 06:32:05 2014-03-01 06:32:05 2014-03-01 07:23:13 0:51:08 0:31:45 0:19:23 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 111554 2014-03-01 06:38:03 2014-03-01 06:38:03 2014-03-01 07:23:22 0:45:19 0:28:37 0:16:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 111555 2014-03-01 06:40:31 2014-03-01 06:40:31 2014-03-01 07:20:22 0:39:51 0:33:07 0:06:44 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111556 2014-03-01 06:40:59 2014-03-01 06:40:59 2014-03-01 07:26:44 0:45:45 0:24:55 0:20:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111557 2014-03-01 06:41:31 2014-03-01 06:41:31 2014-03-01 07:03:08 0:21:37 0:17:23 0:04:14 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
pass 111558 2014-03-01 06:41:41 2014-03-01 06:41:41 2014-03-01 07:53:23 1:11:42 0:49:11 0:22:31 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
pass 111559 2014-03-01 06:46:50 2014-03-01 06:46:50 2014-03-01 07:21:52 0:35:02 0:30:07 0:04:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111560 2014-03-01 06:47:09 2014-03-01 06:47:09 2014-03-01 07:21:21 0:34:12 0:15:29 0:18:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111561 2014-03-01 06:47:40 2014-03-01 06:47:40 2014-03-01 07:58:42 1:11:02 0:32:10 0:38:52 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111562 2014-03-01 06:48:38 2014-03-01 06:48:38 2014-03-01 07:00:00 0:11:22 0:06:01 0:05:21 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111563 2014-03-01 06:48:50 2014-03-01 06:48:50 2014-03-01 07:27:52 0:39:02 0:37:29 0:01:33 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
pass 111564 2014-03-01 06:49:30 2014-03-01 06:49:30 2014-03-01 07:25:52 0:36:22 0:24:55 0:11:27 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
pass 111565 2014-03-01 06:49:43 2014-03-01 06:49:43 2014-03-01 08:07:02 1:17:19 0:29:40 0:47:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 111566 2014-03-01 06:50:51 2014-03-01 06:50:51 2014-03-01 07:21:15 0:30:24 0:20:43 0:09:41 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
pass 111567 2014-03-01 06:51:03 2014-03-01 06:51:03 2014-03-01 07:26:11 0:35:08 0:19:03 0:16:05 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
pass 111568 2014-03-01 06:52:59 2014-03-01 06:52:59 2014-03-01 07:48:30 0:55:31 0:52:47 0:02:44 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
pass 111569 2014-03-01 06:54:30 2014-03-01 06:54:30 2014-03-01 07:50:43 0:56:13 0:27:30 0:28:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 111570 2014-03-01 06:54:44 2014-03-01 06:54:44 2014-03-01 10:46:06 3:51:22 0:14:50 3:36:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111571 2014-03-01 06:55:08 2014-03-01 06:55:08 2014-03-01 07:52:49 0:57:41 0:34:41 0:23:00 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111572 2014-03-01 06:56:30 2014-03-01 06:56:30 2014-03-01 08:03:46 1:07:16 0:05:28 1:01:48 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111573 2014-03-01 06:56:55 2014-03-01 06:56:55 2014-03-01 09:07:46 2:10:51 0:48:50 1:22:01 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111574 2014-03-01 06:57:32 2014-03-01 06:57:32 2014-03-01 07:45:02 0:47:30 0:33:36 0:13:54 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111575 2014-03-01 06:57:40 2014-03-01 06:57:40 2014-03-01 08:44:16 1:46:36 0:34:18 1:12:18 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111576 2014-03-01 06:58:55 2014-03-01 06:58:55 2014-03-01 08:30:42 1:31:47 0:25:01 1:06:46 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
dead 111577 2014-03-01 06:59:09 2014-03-01 06:59:09 2014-03-01 19:51:21 12:52:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml}
pass 111578 2014-03-01 07:00:04 2014-03-01 07:00:04 2014-03-01 08:09:54 1:09:50 1:04:25 0:05:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
pass 111579 2014-03-01 07:00:40 2014-03-01 07:00:40 2014-03-01 07:53:56 0:53:16 0:32:36 0:20:40 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111580 2014-03-01 07:02:11 2014-03-01 07:02:11 2014-03-01 07:32:34 0:30:23 0:17:26 0:12:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111581 2014-03-01 07:02:56 2014-03-01 07:02:56 2014-03-01 07:52:26 0:49:30 0:40:14 0:09:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111582 2014-03-01 07:03:36 2014-03-01 07:03:36 2014-03-01 07:17:44 0:14:08 0:05:42 0:08:26 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111583 2014-03-01 07:03:47 2014-03-01 07:03:47 2014-03-01 08:11:32 1:07:45 0:34:10 0:33:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
dead 111584 2014-03-01 07:04:58 2014-03-01 07:04:58 2014-03-01 07:55:05 0:50:07 0:29:46 0:20:21 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111585 2014-03-01 07:05:02 2014-03-01 07:05:02 2014-03-01 07:49:31 0:44:29 0:34:23 0:10:06 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111586 2014-03-01 07:09:58 2014-03-01 07:09:58 2014-03-01 07:47:15 0:37:17 0:25:22 0:11:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 111587 2014-03-01 07:12:56 2014-03-01 07:12:56 2014-03-01 09:52:32 2:39:36 0:21:51 2:17:45 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111588 2014-03-01 07:14:42 2014-03-01 07:14:42 2014-03-01 08:45:21 1:30:39 0:52:14 0:38:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111589 2014-03-01 07:15:49 2014-03-01 07:15:49 2014-03-01 11:25:41 4:09:52 0:43:33 3:26:19 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111590 2014-03-01 07:16:26 2014-03-01 07:16:26 2014-03-01 07:41:51 0:25:25 0:16:50 0:08:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111591 2014-03-01 07:17:16 2014-03-01 07:17:16 2014-03-01 07:50:50 0:33:34 0:33:02 0:00:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111592 2014-03-01 07:18:42 2014-03-01 07:18:42 2014-03-01 07:26:16 0:07:34 0:05:52 0:01:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111593 2014-03-01 07:19:40 2014-03-01 07:19:40 2014-03-01 08:02:50 0:43:10 0:27:50 0:15:20 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 111594 2014-03-01 07:20:38 2014-03-01 07:20:38 2014-03-01 09:02:18 1:41:40 0:34:03 1:07:37 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 111595 2014-03-01 07:22:59 2014-03-01 07:22:59 2014-03-01 08:20:25 0:57:26 0:34:42 0:22:44 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111596 2014-03-01 07:22:59 2014-03-01 07:22:59 2014-03-01 09:14:06 1:51:07 0:25:54 1:25:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111597 2014-03-01 07:23:15 2014-03-01 07:23:15 2014-03-01 07:49:57 0:26:42 0:23:16 0:03:26 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
pass 111598 2014-03-01 07:23:32 2014-03-01 07:23:32 2014-03-01 08:05:39 0:42:07 0:39:43 0:02:24 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
pass 111599 2014-03-01 07:24:10 2014-03-01 07:24:10 2014-03-01 07:58:22 0:34:12 0:30:27 0:03:45 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111600 2014-03-01 07:24:13 2014-03-01 07:24:13 2014-03-01 07:42:04 0:17:51 0:15:17 0:02:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111601 2014-03-01 07:24:22 2014-03-01 07:24:22 2014-03-01 10:16:58 2:52:36 0:39:26 2:13:10 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111602 2014-03-01 07:25:32 2014-03-01 07:25:32 2014-03-01 07:37:19 0:11:47 0:05:42 0:06:05 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111603 2014-03-01 07:26:44 2014-03-01 07:26:44 2014-03-01 09:09:59 1:43:15 0:48:01 0:55:14 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
pass 111604 2014-03-01 07:27:08 2014-03-01 07:27:08 2014-03-01 08:11:54 0:44:46 0:27:49 0:16:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
pass 111605 2014-03-01 07:27:09 2014-03-01 07:27:09 2014-03-01 08:16:06 0:48:57 0:32:14 0:16:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
fail 111606 2014-03-01 07:27:36 2014-03-01 07:27:36 2014-03-01 08:40:53 1:13:17 0:55:50 0:17:27 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
Failure Reason:

failed to become clean before timeout expired

fail 111607 2014-03-01 07:28:58 2014-03-01 07:28:58 2014-03-01 08:43:27 1:14:29 0:48:15 0:26:14 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

failed to become clean before timeout expired

pass 111608 2014-03-01 07:33:34 2014-03-01 07:33:34 2014-03-01 08:29:47 0:56:13 0:38:55 0:17:18 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
pass 111609 2014-03-01 07:34:16 2014-03-01 07:34:16 2014-03-01 09:51:25 2:17:09 0:32:58 1:44:11 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 111610 2014-03-01 07:37:35 2014-03-01 07:37:35 2014-03-01 09:36:18 1:58:43 0:15:43 1:43:00 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111611 2014-03-01 07:37:58 2014-03-01 07:37:58 2014-03-01 08:30:06 0:52:08 0:36:41 0:15:27 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111612 2014-03-01 07:38:34 2014-03-01 07:38:34 2014-03-01 07:58:00 0:19:26 0:05:08 0:14:18 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111613 2014-03-01 07:41:55 2014-03-01 07:41:55 2014-03-01 08:33:49 0:51:54 0:46:09 0:05:45 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111614 2014-03-01 07:42:20 2014-03-01 07:42:20 2014-03-01 08:21:55 0:39:35 0:31:09 0:08:26 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111615 2014-03-01 07:42:34 2014-03-01 07:42:34 2014-03-01 08:24:57 0:42:23 0:37:39 0:04:44 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111616 2014-03-01 07:45:32 2014-03-01 07:45:32 2014-03-01 08:28:42 0:43:10 0:30:13 0:12:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
pass 111617 2014-03-01 07:45:39 2014-03-01 07:45:39 2014-03-01 09:38:57 1:53:18 0:20:10 1:33:08 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111618 2014-03-01 07:48:04 2014-03-01 07:48:04 2014-03-01 08:46:21 0:58:17 0:47:52 0:10:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
pass 111619 2014-03-01 07:49:07 2014-03-01 07:49:07 2014-03-01 08:21:53 0:32:46 0:31:52 0:00:54 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111620 2014-03-01 07:49:21 2014-03-01 07:49:21 2014-03-01 08:18:51 0:29:30 0:15:03 0:14:27 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111621 2014-03-01 07:51:08 2014-03-01 07:51:08 2014-03-01 08:36:17 0:45:09 0:43:46 0:01:23 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111622 2014-03-01 07:51:20 2014-03-01 07:51:20 2014-03-01 08:05:09 0:13:49 0:06:44 0:07:05 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111623 2014-03-01 07:51:21 2014-03-01 07:51:21 2014-03-01 11:25:44 3:34:23 0:48:16 2:46:07 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111624 2014-03-01 07:52:41 2014-03-01 07:52:41 2014-03-01 10:38:27 2:45:46 0:35:34 2:10:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111625 2014-03-01 07:53:06 2014-03-01 07:53:06 2014-03-01 08:32:12 0:39:06 0:38:13 0:00:53 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111626 2014-03-01 07:53:18 2014-03-01 07:53:18 2014-03-01 08:36:42 0:43:24 0:24:45 0:18:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 111627 2014-03-01 07:53:51 2014-03-01 07:53:51 2014-03-01 08:23:29 0:29:38 0:24:47 0:04:51 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111628 2014-03-01 07:53:51 2014-03-01 07:53:51 2014-03-01 12:23:01 4:29:10 0:43:00 3:46:10 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111629 2014-03-01 07:54:48 2014-03-01 07:54:48 2014-03-01 08:44:59 0:50:11 0:37:54 0:12:17 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111630 2014-03-01 07:55:07 2014-03-01 07:55:07 2014-03-01 08:15:35 0:20:28 0:14:54 0:05:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111631 2014-03-01 07:56:00 2014-03-01 07:56:00 2014-03-01 09:08:45 1:12:45 0:45:13 0:27:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111632 2014-03-01 07:58:37 2014-03-01 07:58:37 2014-03-01 08:23:29 0:24:52 0:06:21 0:18:31 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111633 2014-03-01 07:59:19 2014-03-01 07:59:19 2014-03-01 08:44:26 0:45:07 0:41:32 0:03:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
fail 111634 2014-03-01 07:59:50 2014-03-01 07:59:50 2014-03-01 08:32:21 0:32:31 0:23:07 0:09:24 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.4 restart

pass 111635 2014-03-01 08:00:15 2014-03-01 08:00:15 2014-03-01 09:45:46 1:45:31 0:31:01 1:14:30 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111636 2014-03-01 08:00:45 2014-03-01 08:00:45 2014-03-01 08:49:15 0:48:30 0:27:18 0:21:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111637 2014-03-01 08:03:49 2014-03-01 08:03:49 2014-03-01 08:41:34 0:37:45 0:24:13 0:13:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
pass 111638 2014-03-01 08:04:43 2014-03-01 08:04:43 2014-03-01 09:11:13 1:06:30 0:47:41 0:18:49 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
pass 111639 2014-03-01 08:05:23 2014-03-01 08:05:23 2014-03-01 08:49:37 0:44:14 0:33:58 0:10:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111640 2014-03-01 08:05:45 2014-03-01 08:05:45 2014-03-01 08:46:14 0:40:29 0:15:29 0:25:00 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111641 2014-03-01 08:07:54 2014-03-01 08:07:54 2014-03-01 08:48:06 0:40:12 0:38:08 0:02:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111642 2014-03-01 08:10:14 2014-03-01 08:10:14 2014-03-01 08:28:11 0:17:57 0:07:41 0:10:16 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111643 2014-03-01 08:11:58 2014-03-01 08:11:58 2014-03-01 10:45:24 2:33:26 0:20:24 2:13:02 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
dead 111644 2014-03-01 08:13:16 2014-03-01 08:13:16 2014-03-01 19:50:38 11:37:22 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml}
pass 111645 2014-03-01 08:13:56 2014-03-01 08:13:56 2014-03-01 08:49:20 0:35:24 0:33:11 0:02:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 111646 2014-03-01 08:14:12 2014-03-01 08:14:12 2014-03-01 08:58:29 0:44:17 0:24:38 0:19:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
pass 111647 2014-03-01 08:17:10 2014-03-01 08:17:10 2014-03-01 10:02:50 1:45:40 0:21:35 1:24:05 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
pass 111648 2014-03-01 08:17:16 2014-03-01 08:17:16 2014-03-01 09:01:51 0:44:35 0:38:20 0:06:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
pass 111649 2014-03-01 08:19:31 2014-03-01 08:19:31 2014-03-01 09:13:15 0:53:44 0:28:12 0:25:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 111650 2014-03-01 08:21:07 2014-03-01 08:21:07 2014-03-01 08:48:59 0:27:52 0:16:35 0:11:17 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111651 2014-03-01 08:22:25 2014-03-01 08:22:25 2014-03-01 09:14:42 0:52:17 0:31:17 0:21:00 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111652 2014-03-01 08:23:12 2014-03-01 08:23:12 2014-03-01 08:51:17 0:28:05 0:06:14 0:21:51 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111653 2014-03-01 08:24:00 2014-03-01 08:24:00 2014-03-01 09:18:24 0:54:24 0:49:39 0:04:45 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111654 2014-03-01 08:24:43 2014-03-01 08:24:43 2014-03-01 09:08:01 0:43:18 0:31:42 0:11:36 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111655 2014-03-01 08:26:39 2014-03-01 08:26:39 2014-03-01 09:21:20 0:54:41 0:39:43 0:14:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111656 2014-03-01 08:27:34 2014-03-01 08:27:34 2014-03-01 10:33:48 2:06:14 0:32:48 1:33:26 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
fail 111657 2014-03-01 08:28:16 2014-03-01 08:28:16 2014-03-01 08:56:04 0:27:48 0:22:35 0:05:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed on 10.214.132.7 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=123ff9e18a27c6517a291aab1e0613b27d70d711 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/workunit.client.0/rados/test.sh'

pass 111658 2014-03-01 08:29:39 2014-03-01 08:29:39 2014-03-01 09:17:15 0:47:36 0:47:23 0:00:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
pass 111659 2014-03-01 08:31:12 2014-03-01 08:31:12 2014-03-01 09:08:50 0:37:38 0:36:35 0:01:03 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111660 2014-03-01 08:31:41 2014-03-01 08:31:41 2014-03-01 08:52:47 0:21:06 0:16:02 0:05:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111661 2014-03-01 08:32:04 2014-03-01 08:32:04 2014-03-01 09:31:18 0:59:14 0:41:55 0:17:19 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111662 2014-03-01 08:33:13 2014-03-01 08:33:13 2014-03-01 08:47:38 0:14:25 0:06:02 0:08:23 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111663 2014-03-01 08:33:57 2014-03-01 08:33:57 2014-03-01 09:20:32 0:46:35 0:34:07 0:12:28 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111664 2014-03-01 08:34:05 2014-03-01 08:34:05 2014-03-01 09:29:26 0:55:21 0:36:31 0:18:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111665 2014-03-01 08:37:20 2014-03-01 08:37:20 2014-03-01 10:57:10 2:19:50 0:37:21 1:42:29 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111666 2014-03-01 08:37:26 2014-03-01 08:37:26 2014-03-01 09:18:44 0:41:18 0:29:21 0:11:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 111667 2014-03-01 08:38:05 2014-03-01 08:38:05 2014-03-01 09:09:13 0:31:08 0:21:32 0:09:36 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111668 2014-03-01 08:38:14 2014-03-01 08:38:14 2014-03-01 09:37:29 0:59:15 0:53:11 0:06:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111669 2014-03-01 08:41:51 2014-03-01 08:41:51 2014-03-01 09:56:50 1:14:59 0:35:44 0:39:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111670 2014-03-01 08:41:55 2014-03-01 08:41:55 2014-03-01 09:01:14 0:19:19 0:16:45 0:02:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111671 2014-03-01 08:45:10 2014-03-01 08:45:10 2014-03-01 09:59:19 1:14:09 0:40:13 0:33:56 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111672 2014-03-01 08:45:27 2014-03-01 08:45:27 2014-03-01 08:51:31 0:06:04 0:05:12 0:00:52 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111673 2014-03-01 08:46:01 2014-03-01 08:46:01 2014-03-01 10:24:57 1:38:56 0:23:13 1:15:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 111674 2014-03-01 08:46:55 2014-03-01 08:46:55 2014-03-01 09:18:56 0:32:01 0:29:38 0:02:23 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 111675 2014-03-01 08:47:00 2014-03-01 08:47:00 2014-03-01 09:40:52 0:53:52 0:33:01 0:20:51 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111676 2014-03-01 08:47:52 2014-03-01 08:47:52 2014-03-01 10:05:56 1:18:04 0:26:34 0:51:30 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111677 2014-03-01 08:48:20 2014-03-01 08:48:20 2014-03-01 09:08:01 0:19:41 0:16:26 0:03:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
fail 111678 2014-03-01 08:49:11 2014-03-01 08:49:11 2014-03-01 10:01:21 1:12:10 0:36:58 0:35:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

failed to recover before timeout expired

pass 111679 2014-03-01 08:49:22 2014-03-01 08:49:22 2014-03-01 09:39:20 0:49:58 0:29:18 0:20:40 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111680 2014-03-01 08:50:03 2014-03-01 08:50:03 2014-03-01 10:15:29 1:25:26 0:14:47 1:10:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111681 2014-03-01 08:50:04 2014-03-01 08:50:04 2014-03-01 10:30:31 1:40:27 0:33:49 1:06:38 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111682 2014-03-01 08:50:52 2014-03-01 08:50:52 2014-03-01 09:56:09 1:05:17 0:07:42 0:57:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 111683 2014-03-01 08:51:31 2014-03-01 08:51:31 2014-03-01 10:06:09 1:14:38 1:00:47 0:13:51 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

failed to become clean before timeout expired

pass 111684 2014-03-01 08:53:19 2014-03-01 08:53:19 2014-03-01 09:22:41 0:29:22 0:24:07 0:05:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
pass 111685 2014-03-01 08:53:32 2014-03-01 08:53:32 2014-03-01 09:31:41 0:38:09 0:35:36 0:02:33 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 111686 2014-03-01 08:53:44 2014-03-01 08:53:44 2014-03-01 10:03:07 1:09:23 0:23:44 0:45:39 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
fail 111687 2014-03-01 08:56:21 2014-03-01 08:56:21 2014-03-01 09:31:28 0:35:07 0:19:32 0:15:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2014-03-01 01:25:04.747842 osd.2 10.214.133.28:6810/13767 6 : [ERR] 70.0 scrub stat mismatch, got 23/23 objects, 0/0 clones, 23/22 dirty, 0/0 whiteouts, 2676/2676 bytes." in cluster log

pass 111688 2014-03-01 09:00:22 2014-03-01 09:00:22 2014-03-01 09:41:12 0:40:50 0:39:17 0:01:33 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
pass 111689 2014-03-01 09:02:00 2014-03-01 09:02:00 2014-03-01 09:34:04 0:32:04 0:26:00 0:06:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 111690 2014-03-01 09:02:58 2014-03-01 09:02:58 2014-03-01 09:46:04 0:43:06 0:15:24 0:27:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111691 2014-03-01 09:03:24 2014-03-01 09:03:24 2014-03-01 09:41:23 0:37:59 0:32:35 0:05:24 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111692 2014-03-01 09:08:27 2014-03-01 09:08:27 2014-03-01 09:14:35 0:06:08 0:05:15 0:00:53 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111693 2014-03-01 09:08:48 2014-03-01 09:08:48 2014-03-01 09:57:55 0:49:07 0:34:39 0:14:28 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111694 2014-03-01 09:09:19 2014-03-01 09:09:19 2014-03-01 09:58:37 0:49:18 0:27:06 0:22:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111695 2014-03-01 09:09:28 2014-03-01 09:09:28 2014-03-01 10:01:52 0:52:24 0:38:36 0:13:48 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111696 2014-03-01 09:10:12 2014-03-01 09:10:12 2014-03-01 11:08:46 1:58:34 0:31:16 1:27:18 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
fail 111697 2014-03-01 09:10:13 2014-03-01 09:10:13 2014-03-01 09:28:12 0:17:59 0:13:34 0:04:25 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

Command failed on 10.214.133.37 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=123ff9e18a27c6517a291aab1e0613b27d70d711 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/workunit.client.0/rados/test.sh'

fail 111698 2014-03-01 09:11:07 2014-03-01 09:11:07 2014-03-01 10:01:25 0:50:18 0:44:03 0:06:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

failed to recover before timeout expired

pass 111699 2014-03-01 09:12:56 2014-03-01 09:12:56 2014-03-01 09:48:41 0:35:45 0:35:23 0:00:22 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111700 2014-03-01 09:13:38 2014-03-01 09:13:38 2014-03-01 09:30:38 0:17:00 0:15:56 0:01:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111701 2014-03-01 09:15:23 2014-03-01 09:15:23 2014-03-01 10:56:49 1:41:26 0:32:48 1:08:38 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111702 2014-03-01 09:16:29 2014-03-01 09:16:29 2014-03-01 09:23:13 0:06:44 0:05:51 0:00:53 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 111703 2014-03-01 09:16:31 2014-03-01 09:16:31 2014-03-01 11:54:41 2:38:10 1:17:36 1:20:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
Failure Reason:

failed to recover before timeout expired

pass 111704 2014-03-01 09:17:47 2014-03-01 09:17:47 2014-03-01 10:02:37 0:44:50 0:43:47 0:01:03 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111705 2014-03-01 09:19:02 2014-03-01 09:19:02 2014-03-01 10:06:31 0:47:29 0:33:47 0:13:42 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111706 2014-03-01 09:19:30 2014-03-01 09:19:30 2014-03-01 09:51:33 0:32:03 0:30:10 0:01:53 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 111707 2014-03-01 09:19:35 2014-03-01 09:19:35 2014-03-01 12:23:11 3:03:36 0:19:29 2:44:07 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111708 2014-03-01 09:19:37 2014-03-01 09:19:37 2014-03-01 10:24:26 1:04:49 0:52:54 0:11:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111709 2014-03-01 09:20:11 2014-03-01 09:20:11 2014-03-01 10:23:29 1:03:18 0:34:46 0:28:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111710 2014-03-01 09:20:12 2014-03-01 09:20:12 2014-03-01 10:13:46 0:53:34 0:15:08 0:38:26 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111711 2014-03-01 09:22:11 2014-03-01 09:22:11 2014-03-01 10:00:38 0:38:27 0:37:55 0:00:32 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111712 2014-03-01 09:22:46 2014-03-01 09:22:46 2014-03-01 11:42:21 2:19:35 0:05:42 2:13:53 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111713 2014-03-01 09:23:25 2014-03-01 09:23:25 2014-03-01 10:46:15 1:22:50 0:29:07 0:53:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 111714 2014-03-01 09:24:42 2014-03-01 09:24:42 2014-03-01 10:23:58 0:59:16 0:31:24 0:27:52 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 111715 2014-03-01 09:28:34 2014-03-01 09:28:34 2014-03-01 11:16:28 1:47:54 0:34:56 1:12:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 111716 2014-03-01 09:28:37 2014-03-01 09:28:37 2014-03-01 10:16:18 0:47:41 0:24:51 0:22:50 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 111717 2014-03-01 09:30:21 2014-03-01 09:30:21 2014-03-01 10:00:57 0:30:36 0:19:29 0:11:07 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
pass 111718 2014-03-01 09:30:22 2014-03-01 09:30:22 2014-03-01 10:23:46 0:53:24 0:37:56 0:15:28 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
pass 111719 2014-03-01 09:31:52 2014-03-01 09:31:52 2014-03-01 10:17:04 0:45:12 0:31:15 0:13:57 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 111720 2014-03-01 09:31:53 2014-03-01 09:31:53 2014-03-01 09:56:39 0:24:46 0:14:39 0:10:07 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 111721 2014-03-01 09:32:37 2014-03-01 09:32:37 2014-03-01 10:26:29 0:53:52 0:34:53 0:18:59 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 111722 2014-03-01 09:32:47 2014-03-01 09:32:47 2014-03-01 09:41:57 0:09:10 0:05:36 0:03:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111723 2014-03-01 09:36:05 2014-03-01 09:36:05 2014-03-01 10:43:26 1:07:21 1:04:27 0:02:54 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
fail 111724 2014-03-01 09:37:58 2014-03-01 09:37:58 2014-03-01 10:27:59 0:50:01 0:21:40 0:28:21 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.1 restart

pass 111725 2014-03-01 09:38:45 2014-03-01 09:38:45 2014-03-01 10:27:33 0:48:48 0:30:39 0:18:09 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 111726 2014-03-01 09:39:37 2014-03-01 09:39:37 2014-03-01 11:21:12 1:41:35 0:22:35 1:19:00 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
fail 111727 2014-03-01 09:40:17 2014-03-01 09:40:17 2014-03-01 11:36:50 1:56:33 0:20:20 1:36:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

"2014-03-01 03:31:53.864619 osd.5 10.214.133.26:6814/9545 16 : [ERR] 70.5 scrub stat mismatch, got 25/25 objects, 0/0 clones, 25/21 dirty, 0/0 whiteouts, 4207/4207 bytes." in cluster log

pass 111728 2014-03-01 09:41:09 2014-03-01 09:41:09 2014-03-01 11:52:20 2:11:11 0:38:28 1:32:43 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
fail 111729 2014-03-01 09:41:36 2014-03-01 09:41:36 2014-03-01 10:34:22 0:52:46 0:29:48 0:22:58 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
Failure Reason:

"2014-03-01 02:09:34.236573 mon.1 10.214.131.15:6789/0 3 : [WRN] message from mon.0 was stamped 0.250935s in the future, clocks not synchronized" in cluster log

pass 111730 2014-03-01 09:42:46 2014-03-01 09:42:46 2014-03-01 10:28:31 0:45:45 0:14:41 0:31:04 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 111731 2014-03-01 09:42:49 2014-03-01 09:42:49 2014-03-01 11:10:50 1:28:01 0:28:25 0:59:36 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 111732 2014-03-01 09:46:02 2014-03-01 09:46:02 2014-03-01 10:19:46 0:33:44 0:05:00 0:28:44 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 111733 2014-03-01 09:46:58 2014-03-01 09:46:58 2014-03-01 10:29:16 0:42:18 0:32:22 0:09:56 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 111734 2014-03-01 09:47:04 2014-03-01 09:47:04 2014-03-01 10:33:33 0:46:29 0:30:41 0:15:48 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 111735 2014-03-01 09:49:02 2014-03-01 09:49:02 2014-03-01 10:35:44 0:46:42 0:37:07 0:09:35 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 111736 2014-03-01 09:51:38 2014-03-01 09:51:38 2014-03-01 10:26:36 0:34:58 0:30:24 0:04:34 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
pass 111737 2014-03-01 09:53:04 2014-03-01 09:53:04 2014-03-01 10:21:38 0:28:34 0:20:39 0:07:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
fail 111738 2014-03-01 09:53:42 2014-03-01 09:53:42 2014-03-01 10:48:02 0:54:20 0:38:57 0:15:23 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

timed out waiting for admin_socket to appear after osd.2 restart

pass 111739 2014-03-01 09:57:17 2014-03-01 09:57:17 2014-03-01 10:41:29 0:44:12 0:35:17 0:08:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 111740 2014-03-01 09:57:59 2014-03-01 09:57:59 2014-03-01 10:39:28 0:41:29 0:15:27 0:26:02 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 111741 2014-03-01 09:58:12 2014-03-01 09:58:12 2014-03-01 10:38:55 0:40:43 0:37:30 0:03:13 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 111742 2014-03-01 09:58:40 2014-03-01 09:58:40 2014-03-01 11:14:33 1:15:53 0:05:44 1:10:09 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
dead 111743 2014-03-01 09:59:23 2014-03-01 09:59:23 2014-03-01 19:49:38 9:50:15 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml}
dead 111744 2014-03-01 09:59:34 2014-03-01 09:59:34 2014-03-01 19:49:39 9:50:05 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml}
pass 111745 2014-03-01 10:00:14 2014-03-01 10:00:14 2014-03-01 11:16:20 1:16:06 0:37:04 0:39:02 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 111746 2014-03-01 10:01:27 2014-03-01 10:01:27 2014-03-01 10:42:05 0:40:38 0:39:26 0:01:12 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 111747 2014-03-01 10:02:19 2014-03-01 10:02:19 2014-03-01 11:30:46 1:28:27 0:22:16 1:06:11 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
pass 111748 2014-03-01 10:02:28 2014-03-01 10:02:28 2014-03-01 11:31:40 1:29:12 0:56:39 0:32:33 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
pass 111749 2014-03-01 10:05:24 2014-03-01 10:05:24 2014-03-01 11:04:36 0:59:12 0:35:17 0:23:55 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 111750 2014-03-01 10:05:29 2014-03-01 10:05:29 2014-03-01 10:39:17 0:33:48 0:15:29 0:18:19 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 111751 2014-03-01 10:03:39 2014-03-01 10:03:39 2014-03-01 10:58:35 0:54:56 0:40:10 0:14:46 plana firefly ubuntu rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2