Running Individual Test Cases from Ant Blog

Version 2


    JUnit and Ant are indispensabl e tools for Java development. This article discusses how to use them together more effectively, allowing you more control over which test cases get run. We'll start by showing how to run a specificTest class from Ant, and then move on to selectively running individual test case methods inside aTest.

    But first, why would you want to do this? Test-first design and Extreme Programming suggest that we should run all of the tests, all of the time. However, not everyone has the luxury of using the test-first approach. The techniques discussed in this article were developed to deal with writing test cases for an existing, completely untested system. In such a system, tests are often just used as an easier way to make sure code doesn't blow up, rather than testing it with the UI.

    The main reason I have found for wanting to run only selected test cases is in the case of bug fixes. I'll write a test method (or three) to exercise the bug, and then run those test cases repeatedly as I work on the bug. In an untested system, tests rarely are completely self-verifying, so I'll need to look at the log output. Being able to look at the output from only one test case is helpful for debugging. Another reason for wanting to run only a few tests is that tests often take a long time to run. Running test cases in isolation from each other is also warranted because, as often as not, the problem causing a test to fail is in the test itself.

    I certainly recommend running all of the tests on a regular basis. But if you are already modifying your Ant build or editing your source code in order to selectively run your tests, you may find that the techniques discussed in this article will help shorten your code-compile-test cycle, resulting in better productivity for you.

    For the purposes of this article, I assume you've got Ant installed and configured correctly so you can use the optional<junit> task. To do this, you need to make surejunit.jar and optional.jar are in theANT_HOME/lib directory of your Ant install. If you need more help, check out the Ant documentation (in particular, the section on library dependencies).

    Running Tests from a Single Class

    It's easy to run all tests from Ant (see Cooking with Java XP), but what if you only want to run a specifiedTest? To do this, create a target calledruntest that uses an Ant property set on the command line. This target assumes you have a src property for your source code, a compile target to compile the code before running the tests, and a classpath reference set to your classpath.

    <target name="ensure-test-name" unless="test"> <fail message="You must run this target with -Dtest=TestName"/> </target> <target name="runtest" description="Runs the test you specify on the command line with -Dtest=" depends="compile, ensure-test-name"> <junit printsummary="withOutAndErr" fork="yes"> <classpath refid="classpath" /> <formatter type="plain" usefile="false"/> <batchtest> <fileset dir="${src}"> <include name="**/${test}.java"/> </fileset> </batchtest> </junit> </target>

    The ensure-test-name target verifies that thetest property is set. If test is not set,ensure-test-name fails and warns the user to set the property.

    The runtest target does the work. It depends oncompile to make sure it's run with your latest code. The <junit> task runs the Test you specify with the test property. The task is configured to print all standard out and error output on the console. Theformatter is plain withusefile="false" to print all messages to the console in plain text format, and printsummary="withOutAndErr"will include all messages printed to standard out and error (if you use Log4J, set the appender to ConsoleAppender to see your log messages on standard out). I also fork the JUnit JVM to avoid "Jar Hell;" this is optional. For more configuration of the<junit> task, see the Ant documentation for it.

    The runtest target uses<batchtest> instead of<test>, so we don't have to specify the fully qualified class name of the Test. If no class matches the <include> criteria, no test cases will be run and the build will be successful (the summary report will show no test cases were run). If a class that's not a Testmatches, the build will fail when JUnit tries to run it.

    To run only the Test MyTest, you use the command:

    ant runtest -Dtest=MyTest

    Here, -D is the flag to pass a property to Ant. In the runtest target, you are passing in the name of the test class to run as the value of the testproperty.

    Because runtest uses<batchtest>, you can use wildcard matching with the test property value. For example, to run all the classes ending in "Test", execute this command:

    ant runtest -Dtest=*Test

    Running a Test from a Single Method

    After implementing running a test from a singleTestCase class, I found myself wishing I could run specific test case methods inside of a TestCase class. Unfortunately, there's no way to do this in JUnit without changing the TestCase's suite method and recompiling. But by modifying the runtest target to take another parameter, and with a little bit of Java cleverness, we can dynamically create a TestSuite that only contains the test case methods we want.

    First, let's modify the runtest target to take an optional argument that describes which test cases to run, as a comma-separated list of test case names. This Ant property will be used to create a JVM property.

    <target name="runtest" description="Runs the test you specify on the command line with -Dtest=" depends="compile, ensure-test-name"> <junit printsummary="withOutAndErr" fork="yes"> 
    <sysproperty key="tests" value="${tests}"/> <classpath refid="classpath" /> <formatter type="plain" usefile="false"/> <batchtest> <fileset dir="${src}"> <include name="**/${test}.java"/> </fileset> </batchtest> </junit> </target>

    As you can see, the only modification to this version ofruntest is to set an Ant property namedtests as a JVM property using the<sysproperty> element. The testsproperty is optional, so there is no checking for it as in the case of the ensure-test-name target. Since there is no way to conditionally change a variable in Ant, if tests is not set on the command line, the JVM property will have the literal value "${tests}". The Java code that reads the JVM property will need to be aware of this.

    To run all the test cases in a Test, we'll use the old command line:

    ant runtest -Dtest=MyTest

    To run only a few test cases, we'll use a command like this:

    ant runtest -Dtest=MyTest -Dtests=testFoo,testBar

    As you can see, the tests property is optional.

    Supplementing JUnit

    Now we need some code that can be used by Tests to dynamically create a TestSuite from thetests JVM property. This can be implemented as a subclass of TestCase that your tests then extend instead of TestCase, or it can be implemented with a few static methods in a utility class. I favor the latter approach because it makes it easier for developers to integrate the code into existing Tests.

    We need two public methods: one to determine if thetests property is set and one to construct theTestSuite from the tests property. Both of these methods need to be static so that they can be called from the static suite method of a TestCase. The code uses reflection to dynamically construct instances of theTestCase subclass with each test case name from thetests property. Reflection must be used because eachTestCase object needs to be constructed at runtime with the name of a test case from the testsproperty.

    public final class TestUtils { private static final String TEST_CASES = "tests"; private static final String ANT_PROPERTY = "${tests}"; private static final String DELIMITER = ","; /** * Check to see if the test cases property is set. Ignores Ant's * default setting for the property (or null to be on the safe side). **/ public static boolean hasTestCases() { return System.getProperty( TEST_CASES ) == null || System.getProperty( TEST_CASES ).equals( ANT_PROPERTY ) ? false : true; } /** * Create a TestSuite using the TestCase subclass and the list * of test cases to run specified using the TEST_CASES JVM property. * * @param testClass the TestCase subclass to instantiate as tests in * the suite. * * @return a TestSuite with new instances of testClass for each * test case specified in the JVM property. * * @throws IllegalArgumentException if testClass is not a subclass or * implementation of junit.framework.TestCase. * * @throws RuntimeException if testClass is written incorrectly and does * not have the approriate constructor (It must take one String * argument). **/ public static TestSuite getSuite( Class testClass ) { if ( ! TestCase.class.isAssignableFrom( testClass ) ) { throw new IllegalArgumentException ( "Must pass in a subclass of TestCase" ); } TestSuite suite = new TestSuite(); try { Constructor constructor = testClass.getConstructor( new Class[] { String.class } ); List testCaseNames = getTestCaseNames(); for ( Iterator testCases = testCaseNames.iterator(); testCases.hasNext(); ) { String testCaseName = (String); suite.addTest( (TestCase) constructor. newInstance( new Object[] { testCaseName } ) ); } } catch ( Exception e ) { throw new RuntimeException ( testClass.getName() + " doesn't have the proper constructor" ); } return suite; } /** * Create a List of String names of test cases specified in the * JVM property in comma-separated format. * * @return a List of String test case names * * @throws NullPointerException if the TEST_CASES property * isn't set **/ private static List getTestCaseNames() { if ( System.getProperty( TEST_CASES ) == null ) { throw new NullPointerException( "Test case property is not set" ); } List testCaseNames = new ArrayList(); String testCases = System.getProperty( TEST_CASES ); StringTokenizer tokenizer = new StringTokenizer( testCases, DELIMITER ); while ( tokenizer.hasMoreTokens() ) { testCaseNames.add( tokenizer.nextToken() ); } return testCaseNames; } }

    As you can see, the hasTestCases method ignores thetests property if it is set to the default of "${tests}", and the private helper methodgetTestCaseNames uses a StringTokenizerto parse the comma-separated list of test case names, adding each one to a List. The getSuite method uses this List to reflectively construct newTestCase subclass instances and add them to theTestSuite that it returns. If a test case name is bogus or empty, nothing bad will happen. JUnit will not be able to find the bogus test case and it will be skipped.

    Running the Tests You Want

    Now, all the pieces are in place for you to choose which test cases to execute at run time without recompiling.

    public static TestSuite suite() { if ( TestUtils.hasTestCases() ) { return TestUtils.getSuite( MyTest.class ); } TestSuite suite = new TestSuite( MyTest.class ); return suite; }

    When writing a TestCase, useTestUtils.hasTestCases to check for thetests property, and useTestUtils.getSuite with the Class object for your TestCase subclass to return the dynamically constructed TestSuite. Otherwise, construct and return the TestSuite as usual.