JUnit Reloaded Blog

Version 2


    Let's face it, JUnit is the most widely used (unit-) testing tool in the Java world. There are other powerful test frameworks out there, such as TestNG (which is very comprehensive), but they've never enjoyed the broad acceptance JUnit has. With version 4, Kent Beck and Erich Gamma introduced the first significant API changes in the last few years. When the first release candidate was available back in 2005, you could hardly use it in a productive working environment due to the lack of tool support at that time. By now, most build tools and IDEs come with support for JUnit 4, so it's about time to give it a try. This article describes what's different compared to JUnit 3.8.x.

    What's New?

    Let's start with the most obvious: JUnit 4 is completely based on annotations. Ah, now I know--and what exactly does that mean? First of all, you no longer need to extend classTestCase. Also, your test-method names do not have to start with the prefix test. All you have to do is to mark your test method with a @Test annotation. Take a look at this example:

    import junit.framework.TestCase; 
    import org.junit.Test;  public class CalculatorTest 
    extends TestCase { 
    @Test public void 
    testadd() { .... } }

    Okay, not having to extend TestCase might be helpful in some environments. But what about theassert... methods I used to inherit? They come to you using another Java5 feature: static imports.

    import org.junit.Test; 
    import static org.junit.Assert.*; public class CalculatorTest { @Test public void add() { ... 
    assertEquals( 4, calculator.add( 1, 3 ) ); } }

    Setting Up the Test Environment

    So writing a test isn't that different with JUnit 4. Wait, but what happened tosetUp() and tearDown()? Here we go: You may now decorate any method with the annotations@Before and @After:

    public class CalculatorTest { 
    @Before  public void prepareTestData() { ... } 
    @After  public void cleanupTestData() { ... } }

    You may even have multiple @Before and@After methods, but be aware that nothing is said about the order in which they are executed:

    public class CalculatorTest { 
    @Before  public void prepareTestData() { ... } 
    @Before  public void setupMocks() { ... } 
    @After  public void cleanupTestData() { ... } }

    One thing worth noticing is that inherited @Beforeand @After methods are executed symmetrically. This means if we have a test class ScientificCalculatorTestthat extends CalculatorTest, they are executed as follows:

    CalculatorTest#Before ScientificCalculatorTest#Before ScientificCalculatorTest#After CalculatorTest#After

    One feature a lot of people had been missing is the ability to define a setup/teardown for a whole set of tests. This is useful if you have a very expensive setup that does not need to be run for every test, such as setting up a database connection. This can now be done using the annotations@BeforeClass and @AfterClass:

    public class CalculatorTest { 
    @BeforeClass  public 
    static void setupDatabaseConnection() { ... } 
    @AfterClass  public 
    static void teardownDatabaseConnection() { ... } }

    The same rules apply as for @Before and@After, meaning you can have multiple@BeforeClass and @AfterClass methods and also inherit from super classes. Be aware that these methods must be static.

    Testing for Exceptions

    Checking that our code produces correct results is one thing, but what about error handling? When things go wrong, usually an exception is thrown. Testing that your code behaves correctly in an exceptional case is as important (or even more important) than thereal functionality. And that's where one of the great benefits of unit tests pays off: you can drive the code into a failure situation in a controlled manner (maybe with the help of some mocks) and check if it throws the expected exception. In the JUnit 3.8.x world, the pattern for testing exceptions was:

    public class CalculatorTest { public void testDivisionByZero() { 
    try {  new Calculator().divide( 4, 0 ); 
    } catch (ArithmeticException e) {}  } }

    With JUnit 4, you declare an expected exception in the@Test annotation:

    public class CalculatorTest { @Test
    (expected=ArithmeticException.class) public void testDivisionByZero() { new Calculator().divide( 4, 0 ); } }

    Test with Timeout

    Unit tests should be short-running so you're willing to run them frequently. But sometimes you have some tests that take their time, particularly if network connectivity is involved. So whenever you have reasonable doubts that your test will finish in time, you should make sure that it will be canceled after a specified amount of time. With JUnit 3.8.x, you had to use additional libraries for that or, even worse, do it on your own by utilizing newThreads. Now this task has been eased; all you need to do is to specify a timeout as a parameter of the @Testannotation:

    (timeout=5000) public void testLengthyOperation() { ... }

    If the timeout occurs before the test is finished, you'll get an appropriate failure message:

    java.lang.Exception: test timed out after 5000 milliseconds

    Omitting Tests

    There might be the case that you want some tests to be ignored by the test runner. Maybe the current release of the third-party library you're using has a bug, or the malicious infrastructure prevents your test from being run successfully. Whatever your (more or less) good reason is, in JUnit 3.8.x you had to comment out your tests or exclude them from the test suite. With JUnit 4 you may decorate your method with the @Ignore annotation.

    public class CalculatorTest { 
    @Ignore("Not running because 
    <fill in a good reason here>
    ")  @Test public void testTheWhatSoEverSpecialFunctionality() { } }

    The text handed to the @Ignore annotation will be reported by the test runners. Even if it is optional, you should always provide a comment about why the test is ignored so you won't forget about it. The TestRunner built into Eclipse marks ignored tests visually by rendering it strikethrough. Alas, the comment is not provided (yet).

    Test Suites

    The good old suite() method also found its annotation-based replacement in JUnit 4. Instead of

    public class AllTests extends TestCase { public static Test suite() { TestSuite suite = new TestSuite(); suite.addTestSuite(CalculatorTest.class); suite.addTestSuite(AnotherTest.class); return suite; }

    you write:

    @RunWith(value=Suite.class) @SuiteClasses(value={CalculatorTest.class, AnotherTest.class}) public class AllTests { ... }

    Alright, what's that? Well, JUnit 4 introduces a hook to specify different test runners. In this case, we told JUnit toRunWith the Suite runner to execute this test. The @SuiteClasses annotation is evaluated by theSuite runner, it specifies the tests that belong to this suite. Boy, I sure liked that old suite-style better.Maybe so, but this new way to define a suite also has an advantage: You can specify @BeforeClass and@AfterClass methods that will be executed before the first of the suite and after the last test, respectively. With this, you can define a suite-wide test setup.

    Parameterized Tests

    Besides the Suite runner, JUnit 4 comes with another special runner: Parameterized. It allows you to run the same test(s) with different datasets. Let's do this by example: we will write a test for a method that calculates the factorial for a given number n:

    @RunWith(value=Parameterized.class) public class FactorialTest { private long 
    expected; private int 
    value; @Parameters public static Collection data() { return Arrays.asList( new Object[][] { { 
    0 }, // expected, value { 
    1 }, { 
    2 }, { 
    4 }, { 
    7 }, }); } public FactorialTest(
    long expected, int value) { this.expected = expected; this.value = value; } @Test public void factorial() { Calculator calculator = new Calculator(); assertEquals(
    expected, calculator.factorial(
    value)); } }

    What the Parameterized runner is doing is to run all tests in the FactorialTest (we have just one here ;-) with the data provided by the method decorated with@Parameters. In this case, we have five items in ourList. Each item consists of an array that will be used as constructor arguments for the FactorialTest. Ourfactorial() test then uses this data in theassertEquals(). After all, this means our test is run five times with the following data:

    factorial#0: assertEquals( 
    1, calculator.factorial( 
    0 ) ); factorial#1: assertEquals( 
    1, calculator.factorial( 
    1 ) ); factorial#2: assertEquals( 
    2, calculator.factorial( 
    2 ) ); factorial#3: assertEquals( 
    24, calculator.factorial( 
    4 ) ); factorial#4: assertEquals( 
    5040, calculator.factorial( 
    7 ) );

    Miscellaneous Debris

    The new JUnit classes come within a new package:org.junit. The old test framework is still contained in the package junit.framework for compatibility reasons. Another subtle little change is that instead ofjunit.framework.AssertionFailedError, the(java.lang.)AssertionError is now thrown.

    Something really useful is the new assert for checking the equality of arrays, which first checks if the arrays have an equal length and then compares the array's items usingequals().

    assertEquals(Object[] expected, Object[] actual)

    JUnit 4 no longer supports a UI-based TestRunner; this is left to the IDE developers. But there is still a command-line tool you can use to manually run tests. Just call the classorg.junit.runner.JUnitCore and pass the (fully qualified) names of your test classes:

    java -cp ... 
    org.junit.runner.JUnitCore CalculatorTest AnotherTest

    Hidden Surprise

    Finally, here's some practical joking. When I was fooling around with JUnit 4, one of my first tests was for the trivialCalculator.add() method:

    public class Calculator { public long add(long number1, long number2) { return number1 + number2; } } public class CalculatorTest { @Test public void add() throws Exception { Calculator calculator = new Calculator(); assertEquals(4, calculator.add(1, 3)); } }

    Easy going, eh? But when I was running the test, I got:

    expected:<4> but was:<4>

    What?! Why is that? That's autoboxing, my friend. In JUnit 4, there is no more need for specialassertEquals() for primitive types; there is justassertEquals(Object expected, Object actual). If you pass two ints, they are autoboxed toInteger. And now our problem enters the catwalk:Calculator.add() returns a long, but by default Java uses int for number. So together with autoboxing we get something like this:

    new Integer(4
    new Long(calculator.add(1, 3)

    OK, that's not what I've been expecting, but is that a problem? Yep, have a look at the equals()implementation of Integer:

     public boolean equals(Object obj) { 
    if (obj instanceof Integer) { return value == ((Integer)obj).intValue(); 
    } return false;  }

    Unfortunately, we don't compare with anotherInteger but a Long, soequals() always returns false. This could be an issue when you transfer your old tests to JUnit 4. In JUnit3.8.x, this test would have run, since there areassertEquals() for all kinds of primitive types, so in our case the compiler would have chosen assertEquals(long expected, long actual). To get around this, you have to make sure that long is used in this case:

    (long)4, calculator.add(1, 3) );

    So What?

    JUnit 4 brings some fancy new stuff to us: it is annotation-based, test setup has been significantly improved, there are extension hooks for new runners, and it even adds the much-wanted assertEquals() for comparing arrays.Other frameworks like TestNG introduced this kind of stuff long ago! You're right, but JUnit is still the most widely used (Java) testing framework out there, so it makes sense to introduce a more recent technical base. JUnit's advance is that every major development tool comes with out-of-the-box support: no need to install a plugin, no need to pimp-up my build process. And due to its open annotation-based architecture, extensions are already appearing. Enough of the big words, just give it a try.

    There is a difference between knowing the path and walking the path
    -- Morpheus