Overview Parameterized Tests with JUnit

Gerard Meszaros's describes in his great book with the title xUnit Test Patterns a pattern with the name Parameterized Test. This blog post describes how the pattern can be implemented in a JUnit 4.X test. The post compares three different options.

1. JUnit Parameterized Test

The JUnit framework comes with a runner for parameterized tests. The parameters are defined by a static function which is marked with the annotation @Parameters. The JUnit Runner is called Parameterized.

Example: 

	import static org.junit.Assert.*;
	import java.util.Arrays;
	import java.util.Collection;
	import org.junit.Test;
	import org.junit.runner.RunWith;
	import org.junit.runners.Parameterized;
	import org.junit.runners.Parameterized.Parameters;
	@RunWith(Parameterized.class)
	public class JUnitParameterizedTest {
	static class Person {
	final String name;
	public Person(String name) {
	this.name = name;
	}
	@Override
	public String toString() {
	return String.format("Person[name: %s]", name);
	}
	}
	@Parameters
	public static Collection<Object[]> data() {
	return Arrays.asList(new Object[][]{
	new Object[]{"Christian"},
	new Object[]{"Joshua"},
	});
	}
	String name;
	public JUnitParameterizedTest(String name) {
	this.name = name;
	}
	@Test
	public void createPersonWithName(){
	Person person = new Person(name);
	assertEquals(name, person.name);
	}
	}

view raw JUnitParameterizedTest.java hosted with ❤ by GitHub

Output - Eclipse JUnit View: 

Advantages
No extra framework or library for parameterized tests is needed. The JUnit view in eclipse and also in other IDEs works fine. One test with one defined parameter set can be invoked via the JUnit view in eclipse.

Disadvantages
The output from the tests is not clear. The output shows only the index number of the used test parameter. Only one test data model and parameter set per test class.

2. More JUnit Parameterized Test in a Test Class

It is possible to have more then one parameterized JUnit test in one test class by using the experimental runner "Enclosed".

Example: 

	import static org.junit.Assert.*;
	import java.util.Arrays;
	import java.util.Collection;
	import org.junit.Test;
	import org.junit.experimental.runners.Enclosed;
	import org.junit.runner.RunWith;
	import org.junit.runners.Parameterized;
	import org.junit.runners.Parameterized.Parameters;
	@RunWith(Enclosed.class)
	public class JUnitEnclosedParameterizedTest {
	static class Person {
	final String name;
	public Person(String name) {
	this.name = name;
	}
	@Override
	public String toString() {
	return String.format("Person[name: %s]", name);
	}
	}
	@RunWith(Parameterized.class)
	public static class PersonNameTest {
	@Parameters
	public static Collection<Object[]> data() {
	return Arrays.asList(new Object[][]{
	new Object[]{"Christian"},
	new Object[]{"Joshua"},
	});
	}
	String name;
	public PersonNameTest(String name) {
	this.name = name;
	}
	@Test
	public void createPersonWithName(){
	Person person = new Person(name);
	assertEquals(name, person.name);
	}
	}
	@RunWith(Parameterized.class)
	public static class PersonTest {
	@Parameters
	public static Collection<Object[]> data() {
	return Arrays.asList(new Object[][]{
	new Object[]{new Person("Christian")},
	new Object[]{new Person("Joshua")},
	});
	}
	Person person;
	public PersonTest(Person person) {
	this.person = person;
	}
	@Test
	public void savePerson() {
	// TODO: Implementation test logic
	}
	}
	}

view raw JUnitEnclosedParameterizedTest.java hosted with ❤ by GitHub

Output - Eclipse JUnit View:

Advantages
Grouping logic tests together in one class. Each test can be run from the JUnit view, a single test can be executed for debugging.

Disadvantages
Lots of boilerplate code of the embedded classes.

3. TwiP

TwiP is JUnit extension for parameterized tests. The library brings a JUnit runner, which is named "TwiP".

Example: 

	import static org.junit.Assert.*;
	import net.sf.twip.AutoTwip;
	import net.sf.twip.TwiP;
	import net.sf.twip.Values;
	import org.junit.Test;
	import org.junit.runner.RunWith;
	@RunWith(TwiP.class)
	public class TwiPTest {
	static class Person {
	final String name;
	public Person(String name) {
	this.name = name;
	}
	@Override
	public String toString() {
	return String.format("Person[name: %s]", name);
	}
	}
	public static final String[] names =new String[]{
	"Christian",
	"Joshua"
	};
	@Test
	public void createPersonWithName(@Values("names") String name){
	Person person = new Person(name);
	assertEquals(name, person.name);
	}
	public static final Person[] defaultPersons = new Person[]{
	new Person("Christian"),
	new Person("Joshua")
	};
	@Test
	public void savePerson(@Values("defaultPersons") Person person) {
	// TODO: Implementation test logic
	}
	@AutoTwip
	public static Person randomPersons(String name){
	return new Person(name);
	}
	@Test
	public void savePersonWithRandomName(Person person){
	// TODO: Implementation test logic
	}
	}

view raw TwiPTest.java hosted with ❤ by GitHub

Output - Eclipse JUnit View: 

Advantages
More then one parameterized test in a class is possible. Mixing parameterized and not parameterized tests is possible in one test class. Clear test output toString() method is used for the test parameters.

Disadvantages
A single test could not be chosen from the JUnit output (Eclipse JUnit view) and could not be invoked. This makes debugging the tests difficult because always all test combination must be executed to debug one failing test with one special combination.

4. JUnit Params

JUnit Params is another JUnit extension for parameterized tests like TwiP. The essential difference between TwiP and JUnit Params are the syntax how to define the parameterized tests.

Example: 

	package demo;
	import static org.junit.Assert.*;
	import junitparams.JUnitParamsRunner;
	import junitparams.Parameters;
	import org.junit.Test;
	import org.junit.runner.RunWith;
	@RunWith(JUnitParamsRunner.class)
	public class JUnitParamsTest {
	static class Person {
	final String name;
	public Person(String name) {
	this.name = name;
	}
	@Override
	public String toString() {
	return String.format("Person[name: %s]", name);
	}
	}
	@Test
	@Parameters({"Christian", "Joshua"})
	public void createPersonWithName(String name) {
	Person person = new Person(name);
	assertEquals(name, person.name);
	}
	@Test
	@Parameters(method="defaultPersons")
	public void savePerson(Person person) {
	}
	Object[] defaultPersons() {
	return new Object[]{new Person("Christian"), new Person("Joshua")};
	}
	}

view raw JUnitParamsTest.java hosted with ❤ by GitHub

Output - Eclipse JUnit View: 

Advantages
Same advantages as TwiP and a little bit clearer test output then TwiP.

Disadvantages
Same disadvantages as TwiP a single test could not be invoked.

5. Summary

A perfect solution for a parameterized test in JUnit does not exists. TwiP and JUnitParams provided the way I like to write parameterized tests. But TwiP and also JUnitParams test cannot be debugged a single test can not be executed. I think the problems lies in the JUnit design for the descriptions and not in the TwiP or the JUnitParams project. Hope there will be a better solution in the future. So it's a difficult choice