Comments on: Testing Hadoop http://www.cloudera.com/blog/2008/12/testing-hadoop/ Hadoop and Cloudera's Products and Services Wed, 23 May 2012 10:20:43 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: tom http://www.cloudera.com/blog/2008/12/testing-hadoop/comment-page-1/#comment-55 tom Wed, 21 Jan 2009 09:43:42 +0000 http://www.cloudera.com/blog/?p=159#comment-55 Nick, Thanks for the offer. Apache already has a Clover license and we're trying to (re-)integrate Clover into the Hadoop nightly builds. See https://issues.apache.org/jira/browse/HADOOP-3921 Cheers, Tom Nick,

Thanks for the offer. Apache already has a Clover license and we’re trying to (re-)integrate Clover into the Hadoop nightly builds. See https://issues.apache.org/jira/browse/HADOOP-3921

Cheers,
Tom

]]> By: Nick Pellow http://www.cloudera.com/blog/2008/12/testing-hadoop/comment-page-1/#comment-52 Nick Pellow Wed, 21 Jan 2009 01:06:22 +0000 http://www.cloudera.com/blog/?p=159#comment-52 Hi Steve, Part of Clover's Test Optimization is to automatically re-order your tests from fastest to slowest so that the failures are detected as early as possible in the build. This removes the need to manually separate fast and slow tests in Hadoop. Being Open Source, Hadoop is eligible for a free Clover2 license. I'll also have a look at integrating Clover into the Hadoop build. Cheers, Nick Hi Steve,

Part of Clover’s Test Optimization is to automatically re-order your tests from fastest to slowest so that the failures are detected as early as possible in the build. This removes the need to manually separate fast and slow tests in Hadoop.

Being Open Source, Hadoop is eligible for a free Clover2 license. I’ll also have a look at integrating Clover into the Hadoop build.

Cheers,
Nick

]]> By: Steve Loughran http://www.cloudera.com/blog/2008/12/testing-hadoop/comment-page-1/#comment-29 Steve Loughran Mon, 22 Dec 2008 11:56:10 +0000 http://www.cloudera.com/blog/?p=159#comment-29 the 3h test cycles really irritate me too. There is some stuff in ant 1.7 for recording which tests fail and rerunning just those, so you could work on the failing tests. But then you need to go back to the big run. Maybe we could split into fast+slow, and so do test runs that run all fast tests and if that works, run the slow set. the 3h test cycles really irritate me too. There is some stuff in ant 1.7 for recording which tests fail and rerunning just those, so you could work on the failing tests. But then you need to go back to the big run. Maybe we could split into fast+slow, and so do test runs that run all fast tests and if that works, run the slow set.

]]> By: tom http://www.cloudera.com/blog/2008/12/testing-hadoop/comment-page-1/#comment-26 tom Wed, 17 Dec 2008 13:48:56 +0000 http://www.cloudera.com/blog/?p=159#comment-26 Nick, Thanks for the link to the post on Clover's Test Optimization, very interesting. Lots of Hadoop's tests launch a new JVM for the mapper and reducer code, but this may not matter too much as Clover will still have a good idea of the tests that run a particular application class's code. This could be improved by changing the tests to run tasks in the same JVM, see https://issues.apache.org/jira/browse/HADOOP-3675. Tom Nick,

Thanks for the link to the post on Clover’s Test Optimization, very interesting. Lots of Hadoop’s tests launch a new JVM for the mapper and reducer code, but this may not matter too much as Clover will still have a good idea of the tests that run a particular application class’s code. This could be improved by changing the tests to run tasks in the same JVM, see https://issues.apache.org/jira/browse/HADOOP-3675.

Tom

]]> By: Nick Pellow http://www.cloudera.com/blog/2008/12/testing-hadoop/comment-page-1/#comment-25 Nick Pellow Wed, 17 Dec 2008 06:32:53 +0000 http://www.cloudera.com/blog/?p=159#comment-25 More information on how Clover's Test Optimization works can be found here: http://blogs.atlassian.com/developer/2008/11/stop_testing_so_much.html . It will currently only work for tests that run in the same JVM as your application code. Please don't hesitate to ask if you have any questions regarding Clover and/or Test Optimization. There should be plenty of room to optimize your current 3 hours of tests! Cheers, Nick Pellow Atlassian - Clover. More information on how Clover’s Test Optimization works can be found here: http://blogs.atlassian.com/developer/2008/11/stop_testing_so_much.html .

It will currently only work for tests that run in the same JVM as your application code.

Please don’t hesitate to ask if you have any questions regarding Clover and/or Test Optimization.

There should be plenty of room to optimize your current 3 hours of tests!

Cheers,
Nick Pellow
Atlassian – Clover.

]]>