Status Job ID Links Posted Started Updated
Runtime
Duration
In Waiting
Machine Teuthology Branch OS Type OS Version Description Nodes
pass 11866 2016-01-03 03:01:08 2016-01-04 11:51:52 2016-01-04 18:14:05 6:22:13 1:23:42 4:58:31 smithi master rados/upgrade/hammer-x-singleton/{0-cluster/start.yaml 1-hammer-install/hammer.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-mon/mona.yaml 5-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 6-next-mon/monb.yaml 7-workload/{radosbench.yaml rbd_api.yaml} 8-next-mon/monc.yaml 9-workload/{ec-rados-plugin=jerasure-k=3-m=1.yaml rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml test_cache-pool-snaps.yaml}} 3
pass 11867 2016-01-03 03:01:09 2016-01-04 11:55:01 2016-01-04 12:35:01 0:40:00 0:38:47 0:01:13 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/rados_api_tests.yaml} 2
pass 11868 2016-01-03 03:01:09 2016-01-04 11:56:22 2016-01-04 12:46:23 0:50:01 0:30:50 0:19:11 smithi master rados/singleton-nomsgr/{all/11429.yaml} 1
pass 11869 2016-01-03 03:01:10 2016-01-04 11:59:27 2016-01-04 13:33:29 1:34:02 1:15:18 0:18:44 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 11870 2016-01-03 03:01:10 2016-01-04 12:06:34 2016-01-04 14:14:38 2:08:04 0:41:32 1:26:32 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
fail 11871 2016-01-03 03:01:11 2016-01-04 12:16:33 2016-01-04 14:04:37 1:48:04 1:30:56 0:17:08 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (780) after waiting for 4680 seconds

pass 11872 2016-01-03 03:01:11 2016-01-04 12:17:04 2016-01-04 12:41:04 0:24:00 0:13:46 0:10:14 smithi master rados/singleton/{all/mon-config-keys.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 11873 2016-01-03 03:01:12 2016-01-04 12:17:51 2016-01-04 13:39:53 1:22:02 1:14:46 0:07:16 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
pass 11874 2016-01-03 03:01:12 2016-01-04 12:18:05 2016-01-04 13:02:06 0:44:01 0:19:45 0:24:16 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/rgw_snaps.yaml} 2
pass 11875 2016-01-03 03:01:13 2016-01-04 12:24:53 2016-01-04 12:46:52 0:21:59 0:18:18 0:03:41 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 11876 2016-01-03 03:01:13 2016-01-04 12:24:26 2016-01-04 13:42:28 1:18:02 1:06:32 0:11:30 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 11877 2016-01-03 03:01:14 2016-01-04 12:27:20 2016-01-04 13:49:23 1:22:03 1:14:51 0:07:12 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 11878 2016-01-03 03:01:14 2016-01-04 12:28:13 2016-01-04 12:46:13 0:18:00 0:17:44 0:00:16 smithi master rados/singleton/{all/mon-thrasher.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11879 2016-01-03 03:01:15 2016-01-04 12:29:16 2016-01-04 12:47:16 0:18:00 0:16:18 0:01:42 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 11880 2016-01-03 03:01:15 2016-01-04 12:29:30 2016-01-04 13:19:31 0:50:01 0:32:51 0:17:10 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 11881 2016-01-03 03:01:16 2016-01-04 12:29:50 2016-01-04 14:19:53 1:50:03 1:43:47 0:06:16 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 11882 2016-01-03 03:01:16 2016-01-04 12:30:58 2016-01-04 13:47:00 1:16:02 0:56:42 0:19:20 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 11883 2016-01-03 03:01:17 2016-01-04 12:35:16 2016-01-04 13:41:18 1:06:02 1:02:02 0:04:00 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 11884 2016-01-03 03:01:17 2016-01-04 12:35:29 2016-01-04 14:47:33 2:12:04 1:05:54 1:06:10 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
pass 11885 2016-01-03 03:01:18 2016-01-04 12:36:38 2016-01-04 12:50:38 0:14:00 0:09:16 0:04:44 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml} 2
pass 11886 2016-01-03 03:01:18 2016-01-04 12:40:19 2016-01-04 13:02:19 0:22:00 0:18:07 0:03:53 smithi master rados/singleton-nomsgr/{all/13234.yaml} 1
pass 11887 2016-01-03 03:01:19 2016-01-04 12:41:15 2016-01-04 12:55:15 0:14:00 0:11:13 0:02:47 smithi master rados/singleton/{all/osd-backfill.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 11888 2016-01-03 03:01:19 2016-01-04 12:41:48 2016-01-04 13:37:49 0:56:01 0:50:58 0:05:03 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 11889 2016-01-03 03:01:20 2016-01-04 12:43:23 2016-01-04 13:07:22 0:23:59 0:21:32 0:02:27 smithi master rados/objectstore/ceph_objectstore_tool.yaml 1
pass 11890 2016-01-03 03:01:20 2016-01-04 12:45:20 2016-01-04 13:37:20 0:52:00 0:50:19 0:01:41 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 11891 2016-01-03 03:01:21 2016-01-04 12:46:38 2016-01-04 14:22:40 1:36:02 1:09:07 0:26:55 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 11892 2016-01-03 03:01:21 2016-01-04 12:46:56 2016-01-04 13:32:56 0:46:00 0:40:47 0:05:13 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
fail 11893 2016-01-03 03:01:22 2016-01-04 12:47:22 2016-01-04 14:45:25 1:58:03 1:36:06 0:21:57 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (780) after waiting for 4680 seconds

dead 11894 2016-01-03 03:01:22 2016-01-04 12:47:38 2016-01-05 16:04:36 1 day, 3:16:58 smithi master rados/singleton/{all/osd-recovery-incomplete.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11895 2016-01-03 03:01:23 2016-01-04 12:50:38 2016-01-04 14:30:41 1:40:03 1:09:34 0:30:29 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
fail 11896 2016-01-03 03:01:23 2016-01-04 12:50:52 2016-01-04 13:48:53 0:58:01 0:18:12 0:39:49 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rgw_snaps.yaml} 2
Failure Reason:

Found coredumps on ubuntu@smithi016.front.sepia.ceph.com

pass 11897 2016-01-03 03:01:24 2016-01-04 12:55:28 2016-01-04 13:27:28 0:32:00 0:20:37 0:11:23 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 11898 2016-01-03 03:01:24 2016-01-04 13:02:19 2016-01-04 14:10:21 1:08:02 1:02:40 0:05:22 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 11899 2016-01-03 03:01:25 2016-01-04 13:02:31 2016-01-04 13:44:32 0:42:01 0:10:39 0:31:22 smithi master rados/multimon/{clusters/6.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/mon_clock_no_skews.yaml} 2
pass 11900 2016-01-03 03:01:25 2016-01-04 13:06:13 2016-01-04 15:02:16 1:56:03 1:23:03 0:33:00 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 11901 2016-01-03 03:01:26 2016-01-04 13:07:44 2016-01-04 17:41:53 4:34:09 0:19:29 4:14:40 smithi master rados/thrash-erasure-code-big/{cluster/12-osds.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 11902 2016-01-03 03:01:26 2016-01-04 13:12:22 2016-01-04 13:36:22 0:24:00 0:23:00 0:01:00 smithi master rados/singleton/{all/osd-recovery.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 11903 2016-01-03 03:01:27 2016-01-04 13:19:44 2016-01-04 13:45:44 0:26:00 0:11:35 0:14:25 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
fail 11904 2016-01-03 03:01:27 2016-01-04 13:27:56 2016-01-04 14:19:58 0:52:02 0:34:24 0:17:38 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-radosbench.yaml} 2
Failure Reason:

"2016-01-04 09:01:55.452358 osd.4 172.21.15.58:6800/5797 104 : cluster [ERR] OSD full dropping all updates 97% full" in cluster log

pass 11905 2016-01-03 03:01:28 2016-01-04 13:33:25 2016-01-04 14:13:24 0:39:59 0:34:22 0:05:37 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 11906 2016-01-03 03:01:28 2016-01-04 13:33:43 2016-01-04 13:49:43 0:16:00 0:11:39 0:04:21 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/rados_python.yaml} 2
pass 11907 2016-01-03 03:01:29 2016-01-04 13:34:00 2016-01-04 13:43:58 0:09:58 0:06:32 0:03:26 smithi master rados/singleton-nomsgr/{all/cache-fs-trunc.yaml} 1
pass 11908 2016-01-03 03:01:29 2016-01-04 13:36:52 2016-01-04 14:40:54 1:04:02 1:00:42 0:03:20 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 11909 2016-01-03 03:01:30 2016-01-04 13:37:43 2016-01-04 15:01:45 1:24:02 0:58:07 0:25:55 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 11910 2016-01-03 03:01:30 2016-01-04 13:38:11 2016-01-04 15:08:13 1:30:02 0:55:55 0:34:07 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
pass 11911 2016-01-03 03:01:31 2016-01-04 17:33:37 2016-01-04 17:43:36 0:09:59 0:08:33 0:01:26 smithi master rados/singleton/{all/peer.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11912 2016-01-03 03:01:32 2016-01-04 17:33:37 2016-01-04 18:43:39 1:10:02 0:58:11 0:11:51 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 11913 2016-01-03 03:01:32 2016-01-04 17:33:59 2016-01-04 19:24:02 1:50:03 1:26:50 0:23:13 smithi master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr/simple.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml workloads/snaps-few-objects.yaml} 2
pass 11914 2016-01-03 03:01:33 2016-01-04 17:34:19 2016-01-04 19:20:13 1:45:54 0:57:51 0:48:03 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 11915 2016-01-03 03:01:33 2016-01-04 17:33:47 2016-01-04 18:45:49 1:12:02 1:07:26 0:04:36 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 11916 2016-01-03 03:01:34 2016-01-04 17:34:54 2016-01-04 17:44:54 0:10:00 0:08:27 0:01:33 smithi master rados/objectstore/filejournal.yaml 1
pass 11917 2016-01-03 03:01:34 2016-01-04 17:34:44 2016-01-04 18:18:44 0:44:00 0:42:12 0:01:48 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
fail 11918 2016-01-03 03:01:35 2016-01-04 17:37:51 2016-01-04 18:21:51 0:44:00 0:28:35 0:15:25 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2016-01-04 13:02:02.369555 osd.4 172.21.15.57:6808/7761 262 : cluster [ERR] OSD full dropping all updates 97% full" in cluster log

pass 11919 2016-01-03 03:01:35 2016-01-04 17:34:34 2016-01-04 17:46:33 0:11:59 0:07:59 0:04:00 smithi master rados/singleton/{all/pg-removal-interruption.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 11920 2016-01-03 03:01:36 2016-01-04 17:36:34 2016-01-04 18:52:35 1:16:01 1:13:28 0:02:33 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
pass 11921 2016-01-03 03:01:36 2016-01-04 17:37:08 2016-01-04 19:07:10 1:30:02 0:14:21 1:15:41 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/rgw_snaps.yaml} 2
pass 11922 2016-01-03 03:01:37 2016-01-04 17:37:29 2016-01-04 18:15:29 0:38:00 0:21:12 0:16:48 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
dead 11923 2016-01-03 03:01:37 2016-01-04 17:35:41 2016-01-05 16:04:33 22:28:52 smithi master rados/thrash-erasure-code-shec/{clusters/fixed-4.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
pass 11924 2016-01-03 03:01:38 2016-01-04 17:35:04 2016-01-04 18:21:05 0:46:01 0:43:50 0:02:11 smithi master rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml validater/lockdep.yaml} 2
pass 11925 2016-01-03 03:01:38 2016-01-04 17:38:37 2016-01-04 19:46:41 2:08:04 1:03:04 1:05:00 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 11926 2016-01-03 03:01:39 2016-01-04 17:38:09 2016-01-04 19:22:12 1:44:03 1:21:45 0:22:18 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 11927 2016-01-03 03:01:39 2016-01-04 17:35:16 2016-01-04 17:55:15 0:19:59 0:16:56 0:03:03 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_stress_watch.yaml} 2
pass 11928 2016-01-03 03:01:40 2016-01-04 17:36:07 2016-01-04 18:42:08 1:06:01 0:20:42 0:45:19 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr-failures/few.yaml thrashers/fastread.yaml workloads/ec-small-objects-fast-read.yaml} 2
fail 11929 2016-01-03 03:01:40 2016-01-04 17:35:28 2016-01-04 17:45:27 0:09:59 0:06:50 0:03:09 smithi master rados/singleton-nomsgr/{all/ceph-post-file.yaml} 1
Failure Reason:

Command failed (workunit test post-file.sh) on smithi018 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=749c42422ec31e6ab4858fcb62d45c2c1a8da3f6 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/post-file.sh'

pass 11930 2016-01-03 03:01:41 2016-01-04 17:36:46 2016-01-04 17:56:46 0:20:00 0:12:13 0:07:47 smithi master rados/singleton/{all/radostool.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11931 2016-01-03 03:01:41 2016-01-04 17:42:07 2016-01-04 18:28:08 0:46:01 0:14:28 0:31:33 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 11932 2016-01-03 03:01:42 2016-01-04 17:43:08 2016-01-04 18:41:09 0:58:01 0:41:58 0:16:03 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
pass 11933 2016-01-03 03:01:42 2016-01-04 17:43:54 2016-01-04 19:05:54 1:22:00 0:52:33 0:29:27 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 11934 2016-01-03 03:01:43 2016-01-04 17:45:19 2016-01-04 19:17:21 1:32:02 1:03:02 0:29:00 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 11935 2016-01-03 03:01:43 2016-01-04 17:45:43 2016-01-04 18:51:44 1:06:01 1:01:32 0:04:29 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
pass 11936 2016-01-03 03:01:44 2016-01-04 17:46:47 2016-01-04 18:54:49 1:08:02 1:07:44 0:00:18 smithi master rados/singleton/{all/reg11184.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
fail 11937 2016-01-03 03:01:44 2016-01-04 17:48:46 2016-01-04 19:08:48 1:20:02 0:52:13 0:27:49 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
Failure Reason:

SELinux denials found on ubuntu@smithi012.front.sepia.ceph.com: ['type=AVC msg=audit(1451931237.151:8195): avc: denied { search } for pid=30751 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=8650942 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir']

fail 11938 2016-01-03 03:01:45 2016-01-04 17:50:06 2016-01-04 19:00:07 1:10:01 1:01:10 0:08:51 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
Failure Reason:

SELinux denials found on ubuntu@smithi016.front.sepia.ceph.com: ['type=AVC msg=audit(1451930193.445:5119): avc: denied { search } for pid=17642 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=40894655 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir']

pass 11939 2016-01-03 03:01:45 2016-01-04 17:50:35 2016-01-04 20:30:40 2:40:05 2:03:01 0:37:04 smithi master centos 7.1 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr-failures/osd-delay.yaml supported/centos_7.1.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 11940 2016-01-03 03:01:46 2016-01-04 17:55:17 2016-01-04 19:15:19 1:20:02 1:18:11 0:01:51 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 11941 2016-01-03 03:01:46 2016-01-04 17:55:31 2016-01-04 22:33:40 4:38:09 0:16:56 4:21:13 smithi master rados/thrash-erasure-code-big/{cluster/12-osds.yaml fs/btrfs.yaml msgr-failures/osd-delay.yaml thrashers/fastread.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml} 4
pass 11942 2016-01-03 03:01:47 2016-01-04 17:56:38 2016-01-04 19:06:39 1:10:01 0:37:32 0:32:29 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
fail 11943 2016-01-03 03:01:47 2016-01-04 17:57:03 2016-01-04 20:23:08 2:26:05 1:39:50 0:46:15 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (780) after waiting for 4680 seconds

pass 11944 2016-01-03 03:01:48 2016-01-04 18:00:48 2016-01-04 22:02:57 4:02:09 3:49:41 0:12:28 smithi master rados/objectstore/filestore-idempotent-aio-journal.yaml 1
pass 11945 2016-01-03 03:01:48 2016-01-04 18:11:19 2016-01-04 18:25:17 0:13:58 0:09:48 0:04:10 smithi master rados/singleton/{all/rest-api.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11946 2016-01-03 03:01:49 2016-01-04 18:14:09 2016-01-04 21:30:16 3:16:07 1:18:02 1:58:05 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
fail 11947 2016-01-03 03:01:49 2016-01-04 18:14:24 2016-01-04 18:38:23 0:23:59 0:17:38 0:06:21 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/rgw_snaps.yaml} 2
Failure Reason:

Found coredumps on ubuntu@smithi025.front.sepia.ceph.com

pass 11948 2016-01-03 03:01:50 2016-01-04 18:15:40 2016-01-04 19:43:42 1:28:02 0:29:02 0:59:00 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/rados_workunit_loadgen_big.yaml} 2
pass 11949 2016-01-03 03:01:50 2016-01-04 18:18:59 2016-01-04 18:28:58 0:09:59 0:07:55 0:02:04 smithi master rados/singleton-nomsgr/{all/export-after-evict.yaml} 1
pass 11950 2016-01-03 03:01:51 2016-01-04 18:21:21 2016-01-04 19:27:23 1:06:02 0:20:53 0:45:09 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 11951 2016-01-03 03:01:51 2016-01-04 18:22:09 2016-01-04 18:52:09 0:30:00 0:19:12 0:10:48 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/ec-small-objects.yaml} 2
pass 11952 2016-01-03 03:01:52 2016-01-04 18:25:43 2016-01-04 19:39:45 1:14:02 1:01:05 0:12:57 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
pass 11953 2016-01-03 03:01:52 2016-01-04 18:28:20 2016-01-04 20:48:25 2:20:05 1:25:48 0:54:17 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 11954 2016-01-03 03:01:53 2016-01-04 18:29:12 2016-01-04 19:57:15 1:28:03 0:12:00 1:16:03 smithi master rados/multimon/{clusters/9.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/mon_clock_with_skews.yaml} 3
pass 11955 2016-01-03 03:01:53 2016-01-04 18:34:15 2016-01-04 19:10:15 0:36:00 0:17:53 0:18:07 smithi master rados/singleton/{all/thrash-rados.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 2
pass 11956 2016-01-03 03:01:54 2016-01-04 18:38:41 2016-01-04 19:32:42 0:54:01 0:10:42 0:43:19 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 11957 2016-01-03 03:01:54 2016-01-04 18:41:28 2016-01-04 19:39:30 0:58:02 0:38:16 0:19:46 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/cache-agent-big.yaml} 2
pass 11958 2016-01-03 03:01:55 2016-01-04 18:42:26 2016-01-04 19:42:27 1:00:01 0:58:18 0:01:43 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/cache-agent-small.yaml} 2
pass 11959 2016-01-03 03:01:56 2016-01-04 18:43:52 2016-01-04 19:41:53 0:58:01 0:57:09 0:00:52 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 11960 2016-01-03 03:01:56 2016-01-04 18:46:22 2016-01-04 19:56:24 1:10:02 1:04:56 0:05:06 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps.yaml} 2
dead 11961 2016-01-03 03:01:57 2016-01-04 18:51:54 2016-01-05 16:04:39 21:12:45 smithi master rados/singleton/{all/thrash_cache_writeback_forward_none.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 2
pass 11962 2016-01-03 03:01:57 2016-01-04 18:52:26 2016-01-04 20:00:27 1:08:01 0:53:08 0:14:53 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-snaps.yaml} 2
pass 11963 2016-01-03 03:01:58 2016-01-04 18:52:49 2016-01-04 19:58:51 1:06:02 0:51:20 0:14:42 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache.yaml} 2
pass 11964 2016-01-03 03:01:58 2016-01-04 18:55:13 2016-01-04 20:25:16 1:30:03 1:16:00 0:14:03 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
fail 11965 2016-01-03 03:01:59 2016-01-04 19:00:43 2016-01-04 20:04:45 1:04:02 0:45:11 0:18:51 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/rados_api_tests.yaml} 2
Failure Reason:

SELinux denials found on ubuntu@smithi029.front.sepia.ceph.com: ['type=AVC msg=audit(1451935023.518:5417): avc: denied { search } for pid=23785 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=44826810 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir']

pass 11966 2016-01-03 03:01:59 2016-01-04 19:06:09 2016-01-04 20:02:10 0:56:01 0:17:33 0:38:28 smithi master rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/sync.yaml workloads/pool-create-delete.yaml} 2
fail 11967 2016-01-03 03:02:00 2016-01-04 19:07:01 2016-01-04 20:33:04 1:26:03 0:28:36 0:57:27 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2016-01-04 15:15:13.264460 osd.0 172.21.15.37:6800/11401 130 : cluster [ERR] OSD full dropping all updates 97% full" in cluster log

pass 11968 2016-01-03 03:02:00 2016-01-04 19:07:58 2016-01-04 20:05:47 0:57:49 0:21:54 0:35:55 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_workunit_loadgen_mix.yaml} 2
pass 11969 2016-01-03 03:02:01 2016-01-04 19:09:09 2016-01-04 19:17:04 0:07:55 0:06:21 0:01:34 smithi master rados/singleton-nomsgr/{all/full-tiering.yaml} 1
pass 11970 2016-01-03 03:02:01 2016-01-04 19:10:39 2016-01-04 23:46:42 4:36:03 4:22:32 0:13:31 smithi master rados/objectstore/filestore-idempotent.yaml 1
pass 11971 2016-01-03 03:02:02 2016-01-04 19:15:39 2016-01-04 19:23:35 0:07:56 0:06:37 0:01:19 smithi master rados/singleton/{all/watch-notify-same-primary.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 11972 2016-01-03 03:02:02 2016-01-04 19:17:41 2016-01-04 20:57:34 1:39:53 1:18:24 0:21:29 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/readwrite.yaml} 2
fail 11973 2016-01-03 03:02:03 2016-01-04 19:17:46 2016-01-04 19:43:45 0:25:59 0:18:23 0:07:36 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/rgw_snaps.yaml} 2
Failure Reason:

Found coredumps on ubuntu@smithi002.front.sepia.ceph.com

pass 11974 2016-01-03 03:02:03 2016-01-04 19:20:28 2016-01-04 21:24:30 2:04:02 1:28:33 0:35:29 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml} 2
pass 11975 2016-01-03 03:02:04 2016-01-04 19:22:28 2016-01-04 19:48:27 0:25:59 0:20:45 0:05:14 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/small-objects.yaml} 2
pass 11976 2016-01-03 03:02:04 2016-01-04 19:23:48 2016-01-04 20:41:50 1:18:02 1:01:57 0:16:05 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/snaps-few-objects.yaml} 2
pass 11977 2016-01-03 03:02:05 2016-01-04 19:24:15 2016-01-04 20:54:18 1:30:03 1:21:29 0:08:34 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 11978 2016-01-03 03:02:05 2016-01-04 19:27:36 2016-01-04 19:51:36 0:24:00 0:07:44 0:16:16 smithi master rados/singleton/{all/admin-socket.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
dead 11979 2016-01-03 03:02:06 2016-01-04 19:32:56 2016-01-05 16:05:40 20:32:44 smithi master rados/thrash-erasure-code-big/{cluster/12-osds.yaml fs/ext4.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}
pass 11980 2016-01-03 03:02:06 2016-01-04 19:39:52 2016-01-04 19:55:51 0:15:59 0:12:06 0:03:53 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 11981 2016-01-03 03:02:07 2016-01-04 19:40:09 2016-01-04 20:36:10 0:56:01 0:39:19 0:16:42 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-agent-big.yaml} 2
pass 11982 2016-01-03 03:02:08 2016-01-04 19:42:11 2016-01-04 20:50:12 1:08:01 1:00:49 0:07:12 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-agent-small.yaml} 2
pass 11983 2016-01-03 03:02:08 2016-01-04 19:42:51 2016-01-04 21:24:53 1:42:02 1:02:37 0:39:25 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 11984 2016-01-03 03:02:09 2016-01-04 19:43:59 2016-01-04 20:58:01 1:14:02 0:59:08 0:14:54 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-pool-snaps.yaml} 2
pass 11985 2016-01-03 03:02:09 2016-01-04 19:44:13 2016-01-04 20:20:14 0:36:01 0:28:55 0:07:06 smithi master rados/singleton/{all/cephtool.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
dead 11986 2016-01-03 03:02:10 2016-01-04 19:46:56 2016-01-05 16:05:37 20:18:41 smithi master rados/thrash-erasure-code-shec/{clusters/fixed-4.yaml fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
pass 11987 2016-01-03 03:02:10 2016-01-04 19:49:22 2016-01-04 20:19:14 0:29:52 0:13:48 0:16:04 smithi master rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml validater/valgrind.yaml} 2
pass 11988 2016-01-03 03:02:11 2016-01-04 22:02:59 2016-01-04 23:05:00 1:02:01 0:51:12 0:10:49 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml} 2
pass 11989 2016-01-03 03:02:11 2016-01-04 22:03:06 2016-01-04 23:13:08 1:10:02 0:58:40 0:11:22 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache.yaml} 2
pass 11990 2016-01-03 03:02:12 2016-01-04 22:03:17 2016-01-04 22:43:18 0:40:01 0:36:32 0:03:29 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/rados_workunit_loadgen_mostlyread.yaml} 2
pass 11991 2016-01-03 03:02:12 2016-01-04 22:03:54 2016-01-04 22:35:55 0:32:01 0:30:05 0:01:56 smithi master rados/singleton-nomsgr/{all/msgr.yaml} 1
pass 11992 2016-01-03 03:02:13 2016-01-04 22:04:21 2016-01-04 23:14:23 1:10:02 1:03:24 0:06:38 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 11993 2016-01-03 03:02:13 2016-01-04 22:06:08 2016-01-04 23:12:09 1:06:01 0:39:05 0:26:56 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/rados_api_tests.yaml} 2
fail 11994 2016-01-03 03:02:14 2016-01-04 22:07:54 2016-01-04 22:45:55 0:38:01 0:30:05 0:07:56 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

"2016-01-04 17:28:54.737673 osd.4 172.21.15.15:6805/25240 182 : cluster [ERR] OSD full dropping all updates 97% full" in cluster log

pass 11995 2016-01-03 03:02:14 2016-01-04 22:09:33 2016-01-05 00:57:40 2:48:07 2:47:48 0:00:19 smithi master rados/singleton/{all/divergent_priors.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 11996 2016-01-03 03:02:15 2016-01-04 22:12:19 2016-01-04 23:30:22 1:18:03 1:15:31 0:02:32 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/readwrite.yaml} 2
pass 11997 2016-01-03 03:02:15 2016-01-04 22:12:53 2016-01-04 22:34:53 0:22:00 0:20:22 0:01:38 smithi master rados/objectstore/objectcacher-stress.yaml 1
pass 11998 2016-01-03 03:02:16 2016-01-04 22:13:08 2016-01-05 00:01:11 1:48:03 1:23:58 0:24:05 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml} 2
pass 11999 2016-01-03 03:02:16 2016-01-04 22:14:02 2016-01-04 22:34:00 0:19:58 0:15:59 0:03:59 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/rgw_snaps.yaml} 2
pass 12000 2016-01-03 03:02:17 2016-01-04 22:14:31 2016-01-04 23:04:32 0:50:01 0:20:34 0:29:27 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/small-objects.yaml} 2
pass 12001 2016-01-03 03:02:17 2016-01-04 22:16:32 2016-01-04 23:58:35 1:42:03 1:04:33 0:37:30 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 12002 2016-01-03 03:02:18 2016-01-04 22:18:47 2016-01-04 23:42:51 1:24:04 1:18:33 0:05:31 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 12003 2016-01-03 03:02:18 2016-01-04 22:26:07 2016-01-04 22:54:05 0:27:58 0:23:29 0:04:29 smithi master rados/singleton/{all/divergent_priors2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 12004 2016-01-03 03:02:19 2016-01-04 22:29:27 2016-01-04 22:45:27 0:16:00 0:14:37 0:01:23 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 12005 2016-01-03 03:02:19 2016-01-04 22:30:46 2016-01-04 23:12:56 0:42:10 0:34:54 0:07:16 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache-agent-big.yaml} 2
pass 12006 2016-01-03 03:02:20 2016-01-04 22:31:38 2016-01-04 23:23:39 0:52:01 0:49:00 0:03:01 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-agent-small.yaml} 2
pass 12007 2016-01-03 03:02:20 2016-01-04 22:33:12 2016-01-04 23:35:14 1:02:02 0:53:03 0:08:59 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 12008 2016-01-03 03:02:21 2016-01-04 22:33:51 2016-01-05 01:21:56 2:48:05 0:25:27 2:22:38 smithi master rados/multimon/{clusters/21.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/mon_recovery.yaml} 3
pass 12009 2016-01-03 03:02:21 2016-01-04 22:34:11 2016-01-04 23:38:12 1:04:01 1:00:46 0:03:15 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/cache-pool-snaps.yaml} 2
pass 12010 2016-01-03 03:02:22 2016-01-04 22:35:05 2016-01-04 23:13:06 0:38:01 0:18:57 0:19:04 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/repair_test.yaml} 2
pass 12011 2016-01-03 03:02:22 2016-01-04 22:36:09 2016-01-04 22:58:09 0:22:00 0:13:55 0:08:05 smithi master rados/singleton-nomsgr/{all/multi-backfill-reject.yaml} 2
pass 12012 2016-01-03 03:02:23 2016-01-04 22:37:26 2016-01-05 00:45:32 2:08:06 1:32:50 0:35:16 smithi master ubuntu 14.04 rados/thrash-erasure-code-isa/{arch/x86_64.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr-failures/fastclose.yaml supported/ubuntu_14.04.yaml thrashers/pggrow.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} 2
pass 12013 2016-01-03 03:02:23 2016-01-04 22:38:07 2016-01-04 22:54:07 0:16:00 0:09:24 0:06:36 smithi master rados/singleton/{all/dump-stuck.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 12014 2016-01-03 03:02:24 2016-01-04 22:42:56 2016-01-04 23:36:58 0:54:02 0:50:14 0:03:48 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/cache-snaps.yaml} 2
pass 12015 2016-01-03 03:02:24 2016-01-04 22:43:18 2016-01-05 00:07:20 1:24:02 0:54:28 0:29:34 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/cache.yaml} 2
pass 12016 2016-01-03 03:02:25 2016-01-04 22:43:27 2016-01-05 00:39:32 1:56:05 1:02:45 0:53:20 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/pool-snaps-few-objects.yaml} 2
pass 12017 2016-01-03 03:02:25 2016-01-04 22:45:38 2016-01-04 23:25:39 0:40:01 0:37:36 0:02:25 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/rados_api_tests.yaml} 2
fail 12018 2016-01-03 03:02:26 2016-01-04 22:46:06 2016-01-05 00:42:09 1:56:03 1:33:41 0:22:22 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (780) after waiting for 4680 seconds

pass 12019 2016-01-03 03:02:26 2016-01-04 22:48:51 2016-01-04 23:14:56 0:26:05 0:10:22 0:15:43 smithi master rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml fs/xfs.yaml msgr/simple.yaml msgr-failures/mon-delay.yaml thrashers/force-sync-many.yaml workloads/rados_5925.yaml} 2
dead 12020 2016-01-03 03:02:27 2016-01-04 22:54:14 2016-01-05 16:04:48 17:10:34 smithi master rados/thrash-erasure-code-big/{cluster/12-osds.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml}
fail 12021 2016-01-03 03:02:27 2016-01-04 22:54:22 2016-01-04 23:10:22 0:16:00 0:11:41 0:04:19 smithi master rados/singleton/{all/ec-lost-unfound.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
Failure Reason:

'new_1'

pass 12022 2016-01-03 03:02:28 2016-01-04 22:58:19 2016-01-05 00:00:20 1:02:01 0:46:26 0:15:35 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr-failures/osd-delay.yaml thrashers/fastread.yaml workloads/ec-radosbench.yaml} 2
pass 12023 2016-01-03 03:02:29 2016-01-04 23:04:42 2016-01-05 00:32:44 1:28:02 1:19:20 0:08:42 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/readwrite.yaml} 2
fail 12024 2016-01-03 03:02:29 2016-01-04 23:05:09 2016-01-04 23:43:10 0:38:01 0:15:46 0:22:15 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/rgw_snaps.yaml} 2
Failure Reason:

Found coredumps on ubuntu@smithi056.front.sepia.ceph.com

pass 12025 2016-01-03 03:02:30 2016-01-04 23:10:31 2016-01-05 00:10:33 1:00:02 0:59:06 0:00:56 smithi master rados/objectstore/objectstore.yaml 1
pass 12026 2016-01-03 03:02:30 2016-01-04 23:12:19 2016-01-04 23:34:19 0:22:00 0:21:13 0:00:47 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/small-objects.yaml} 2
pass 12027 2016-01-03 03:02:31 2016-01-04 23:13:05 2016-01-05 00:39:07 1:26:02 1:08:01 0:18:01 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/snaps-few-objects.yaml} 2
pass 12028 2016-01-03 03:02:31 2016-01-04 23:13:15 2016-01-05 00:57:18 1:44:03 1:19:24 0:24:39 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 12029 2016-01-03 03:02:32 2016-01-04 23:13:24 2016-01-04 23:23:24 0:10:00 0:07:06 0:02:54 smithi master rados/singleton/{all/lost-unfound-delete.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 12030 2016-01-03 03:02:32 2016-01-04 23:35:00 2016-01-05 00:17:01 0:42:01 0:17:28 0:24:33 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/admin_socket_objecter_requests.yaml} 2
pass 12031 2016-01-03 03:02:33 2016-01-04 23:35:23 2016-01-05 00:11:24 0:36:01 0:33:39 0:02:22 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/cache-agent-big.yaml} 2
pass 12032 2016-01-03 03:02:33 2016-01-04 23:37:08 2016-01-04 23:55:08 0:18:00 0:11:27 0:06:33 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml tasks/scrub_test.yaml} 2
pass 12033 2016-01-03 03:02:34 2016-01-04 23:37:43 2016-01-04 23:49:43 0:12:00 0:06:08 0:05:52 smithi master rados/singleton-nomsgr/{all/valgrind-leaks.yaml} 1
pass 12034 2016-01-03 03:02:34 2016-01-04 23:38:21 2016-01-05 01:26:25 1:48:04 0:59:42 0:48:22 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/cache-agent-small.yaml} 2
pass 12035 2016-01-03 03:02:35 2016-01-04 23:43:00 2016-01-05 00:47:01 1:04:01 0:58:16 0:05:45 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/cache-pool-snaps-readproxy.yaml} 2
pass 12036 2016-01-03 03:02:35 2016-01-04 23:43:19 2016-01-05 00:45:21 1:02:02 0:58:55 0:03:07 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/cache-pool-snaps.yaml} 2
pass 12037 2016-01-03 03:02:36 2016-01-04 23:45:59 2016-01-04 23:57:59 0:12:00 0:10:56 0:01:04 smithi master rados/singleton/{all/lost-unfound.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 12038 2016-01-03 03:02:36 2016-01-04 23:46:51 2016-01-05 00:44:53 0:58:02 0:49:51 0:08:11 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/mapgap.yaml workloads/cache-snaps.yaml} 2
pass 12039 2016-01-03 03:02:37 2016-01-04 23:49:52 2016-01-05 00:51:54 1:02:02 0:53:24 0:08:38 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml workloads/cache.yaml} 2
fail 12040 2016-01-03 03:02:37 2016-01-04 23:50:45 2016-01-05 00:06:45 0:16:00 0:03:56 0:12:04 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/pool-snaps-few-objects.yaml} 2
Failure Reason:

Command failed on smithi018 with status 1: 'sudo yum install ceph-debuginfo -y'

pass 12041 2016-01-03 03:02:38 2016-01-04 23:55:17 2016-01-05 00:39:18 0:44:01 0:36:52 0:07:09 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rados_api_tests.yaml} 2
fail 12042 2016-01-03 03:02:38 2016-01-04 23:58:11 2016-01-05 01:48:14 1:50:03 1:33:27 0:16:36 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/mapgap.yaml workloads/radosbench.yaml} 2
Failure Reason:

reached maximum tries (780) after waiting for 4680 seconds

pass 12043 2016-01-03 03:02:39 2016-01-04 23:58:49 2016-01-05 00:26:48 0:27:59 0:20:20 0:07:39 smithi master rados/thrash-erasure-code/{clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/ec-small-objects-fast-read.yaml} 2
pass 12044 2016-01-03 03:02:39 2016-01-05 00:00:30 2016-01-05 00:20:30 0:20:00 0:13:16 0:06:44 smithi master rados/singleton/{all/mon-config-keys.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml} 1
pass 12045 2016-01-03 03:02:40 2016-01-05 00:01:20 2016-01-05 01:27:23 1:26:03 1:16:10 0:09:53 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/morepggrow.yaml workloads/readwrite.yaml} 2
pass 12046 2016-01-03 03:02:40 2016-01-05 00:06:54 2016-01-05 00:34:55 0:28:01 0:18:09 0:09:52 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/pggrow.yaml workloads/rgw_snaps.yaml} 2
pass 12047 2016-01-03 03:02:41 2016-01-05 00:07:29 2016-01-05 00:57:30 0:50:01 0:17:53 0:32:08 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/default.yaml workloads/small-objects.yaml} 2
pass 12048 2016-01-03 03:02:41 2016-01-05 00:10:43 2016-01-05 01:32:45 1:22:02 0:59:25 0:22:37 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/ext4.yaml msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/mapgap.yaml workloads/snaps-few-objects.yaml} 2
dead 12049 2016-01-03 03:02:42 2016-01-05 00:11:32 2016-01-05 16:04:04 15:52:32 smithi master rados/thrash-erasure-code-shec/{clusters/fixed-4.yaml fs/xfs.yaml msgr-failures/fastclose.yaml thrashers/default.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml}
pass 12050 2016-01-03 03:02:42 2016-01-05 00:12:23 2016-01-05 00:52:24 0:40:01 0:14:07 0:25:54 smithi master rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml validater/lockdep.yaml} 2
pass 12051 2016-01-03 03:02:43 2016-01-05 00:17:10 2016-01-05 00:29:10 0:12:00 0:07:32 0:04:28 smithi master rados/objectstore/alloc-hint.yaml 1
pass 12052 2016-01-03 03:02:43 2016-01-05 00:20:39 2016-01-05 02:06:42 1:46:03 1:18:38 0:27:25 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-1-min-size.yaml 1-pg-log-overrides/normal_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml msgr/simple.yaml msgr-failures/few.yaml thrashers/morepggrow.yaml workloads/write_fadvise_dontneed.yaml} 2
pass 12053 2016-01-03 03:02:44 2016-01-05 00:26:58 2016-01-05 01:16:59 0:50:01 0:37:40 0:12:21 smithi master rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml} 2
fail 12054 2016-01-03 03:02:44 2016-01-05 00:29:20 2016-01-05 00:33:19 0:03:59 0:02:49 0:01:10 smithi master rados/singleton-nomsgr/{all/11429.yaml} 1
Failure Reason:

Command failed on smithi018 with status 1: 'sudo yum install ceph-radosgw -y'

pass 12055 2016-01-03 03:02:45 2016-01-05 00:32:54 2016-01-05 00:54:55 0:22:01 0:20:18 0:01:43 smithi master rados/singleton/{all/mon-thrasher.yaml fs/btrfs.yaml msgr/simple.yaml msgr-failures/many.yaml} 1
pass 12056 2016-01-03 03:02:45 2016-01-05 00:33:29 2016-01-05 00:51:29 0:18:00 0:16:22 0:01:38 smithi master rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/2-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/btrfs.yaml msgr/simple.yaml msgr-failures/osd-delay.yaml thrashers/pggrow.yaml workloads/admin_socket_objecter_requests.yaml} 2