Parameterized test

You have already learned about unit testing principles and the JUnit framework and familiarized yourself with lifecycle annotations used to control the execution of tests. Now it's time to study new advanced features of JUnit, which will help you to manipulate data supplied to your tests.

Getting started

In this topic, we will expand our Calculator class from the introductory JUnit topic by adding a new method, which will calculate the maximum of two arguments:

class Calculator {
    fun maxOf(a: Int, b: Int): Int {
        return if (a >= b) {
            a
        } else {
            b
        }
    }
}

We will also write the necessary tests to make sure that this method works correctly. We need to test three cases: when the first argument is greater than the second one, when the first argument is less than the second one, and when both arguments are equal.

import org.junit.jupiter.api.Assertions.*
import org.junit.jupiter.api.Test

internal class CalculatorTest {
    @Test
    fun testMaxFirstArgGreaterThanSecondArg() {
        val calculator = Calculator()
        val result = calculator.maxOf(2, 1)
        val expected = 2
        assertEquals(expected, result)
    }

    @Test
    fun testMaxFirstArgLessThanSecondArg() {
        val calculator = Calculator()
        val result = calculator.maxOf(1, 2)
        val expected = 2
        assertEquals(expected, result)
    }

    @Test
    fun testMaxFirstArgEqualToSecondArg() {
        val calculator = Calculator()
        val result = calculator.maxOf(2, 2)
        val expected = 2
        assertEquals(expected, result)
    }
}

Now, let's run these tests to make sure that our implementation of the maxOf method successfully passes all the tests. Running the test using Gradle gives the following output:

However, if you look at these tests, you will notice that they are nearly identical and the only difference is the values we use in their bodies. Is there a way to write such tests in a cleaner manner? JUnit provides us such an option: it is called parameterized tests.

First, let's add the following dependency to our project so that JUnit will be able to work with parameterized tests.

Gradle:

dependencies {
    testImplementation("org.junit.jupiter:junit-jupiter-params:5.9.0")
}

Maven:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.9.0</version>
    <scope>test</scope>
</dependency>

@ParameterizedTest

@ParameterizedTest allows us to invoke a single test method multiple times, passing different arguments to it. Look at the following code snippet:

import org.junit.jupiter.api.Assertions.*
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.CsvSource

internal class CalculatorTest {
    @ParameterizedTest
    @CsvSource("2, 1, 2", "1, 2, 2", "1, 1, 1")
    fun testMax(first: Int, second: Int, expected: Int) {
        val calculator = Calculator()
        val result = calculator.maxOf(first, second)
        assertEquals(expected, result)
    }
}

In this example, we use the @ParameterizedTest annotation instead of @Test to specify that the corresponding test should be executed multiple times with different arguments. We also use the @CsvSource annotation to provide an array of such arguments. JUnit has plenty of annotations for different sources of arguments, such as @ValueSource, @EnumSource, @MethodSource, @CsvSource, @CsvFileSource, and @ArgumentsSource, some of which we are going to discuss below.

Note that the test method now has three parameters: first: Int, second: Int, and expected: Int, which are used in the body of the test method, and respective arguments are supplied by JUnit at runtime based on the specified argument source. Let's run this test:

The default output consists of the current invocation index and the list of arguments. We can specify a custom message format for a test using attributes and placeholders, for example:

@ParameterizedTest(name = "{index} => maxOf({0}, {1}) == {2}")

The execution of the same test with such a custom display name looks as follows:

With the help of custom display names, you can easily and conveniently provide pretty and informative test outputs.

Sources of arguments

JUnit provides a number of annotations to define the source of arguments. Such arguments may be a sequence of test method arguments of the same type. They accept a single test method argument or a sequence of arguments of the same or different types, which, in turn, accept multiple arguments. We will discuss @ValueSource, @CsvSource, and @CsvFileSource in detail and have a glimpse of some other annotations. You can find the full information about them in the official guide.

@ValueSource

@ValueSource is an argument source that supplies an array of literal values for test methods with a single parameter. Such literal values may be of any of the following types: Short, Byte, Int, Long, Float, Double, Char, Boolean, java.lang.String, and java.lang.Class.

Let us add another method to our Calculator class, which will accept a single Int argument and return Boolean:

fun isEven(a: Int): Boolean {
    return a % 2 == 0
}

After that, we will use the following test method, which will be invoked multiple times with different integer arguments supplied by @ValueSource:

import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.ValueSource

internal class CalculatorTest {

    @ParameterizedTest
    @ValueSource(ints = [0, 2, 4, 1000])
    fun testIsEven(arg: Int) {
        assertTrue(Calculator().isEven(arg))
    }
}

For non-primitive types, you can use @EmptySource, @NullSource, or @NullAndEmptySource annotations to pass null and empty values. In order to avoid writing too much code, we will be using more abstract examples to illustrate how to pass different types of arguments to test methods. The following code snippet demonstrates passing an empty argument as well as an empty and then null arguments:

import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.EmptySource
import org.junit.jupiter.params.provider.NullAndEmptySource

internal class CalculatorTest {

    @ParameterizedTest
    @EmptySource
    fun testEmpty(arg: IntArray) {
        assertEquals(0, arg.size)
    }

    @ParameterizedTest
    @NullAndEmptySource
    fun testNullAndEmpty(arg: List<String?>?) {
        assertTrue(arg == null || arg.isEmpty()) // we can also write this line 
                                                 // assertTrue(arg.isNullOrEmpty())
    }
}

You can even combine these annotations with @ValueSource values to check the whole range of test cases in a single test method.

@MethodSource

This annotation allows you to use a method of your test class or an external class as a source of arguments. Each such method must satisfy the following requirements: it must not accept any arguments and must return a stream, an array, or a collection of arguments.

import junit.framework.TestCase.assertFalse
import org.junit.jupiter.api.TestInstance
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.MethodSource

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class CalculatorTest {

    @ParameterizedTest
    @MethodSource("stringFactory")
    fun testStrings(str: String) {
        assertFalse(str.isEmpty())
    }

    fun stringFactory(): List<String>? {
        return listOf("apple", "banana", "lemon", "orange")
    }
}

If a parameterized test method has multiple parameters, your argument source method needs to return a collection, a stream, or an array of Arguments or an array of Object:

import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.TestInstance
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.Arguments
import org.junit.jupiter.params.provider.Arguments.arguments
import org.junit.jupiter.params.provider.MethodSource

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class CalculatorTest {

    @ParameterizedTest
    @MethodSource("argFactory")
    fun testStringLength(str: String, length: Int) {
        assertEquals(length, str.length)
    }

    fun argFactory(): List<Arguments?>? {
        return listOf(arguments("apple", 5), arguments("watermelon", 10))
    }
}

See the official JUnit documentation for more detailed information about @MethodSource.

Working with CSV

In the first example of a parameterized test, we used the annotation @CsvSource. It allows you to supply a list of arguments as comma-separated values (CSV format), for example:

@CsvSource({ "apple, 5", "strawberry, 10", "cherry, 6" })

In this case, each value is represented by a String literal containing a list of arguments separated by a comma, which serves as the default delimiter. @CsvSource also has a number of attributes to define the format of the arguments. You can change the default delimiter to another character or even a String literal, as well as define the representation of empty and null values; however, all these attributes are optional and can be used when needed.

Due to its flexibility, @CsvSource is well suited for supplying arguments for methods with multiple parameters of different types.

In addition to@CsvSource, JUnit has the @CsvFileSource annotation, which is used to load a CSV file from the classpath or the local file system. Each line from a CSV file serves as the source of arguments for one invocation of the parameterized test. You may skip the desired number of lines in the file by setting the numLinesToSkip attribute. Also, if you want any lines in the CSV file to be ignored, you can use the symbol # at the beginning of the respective lines to comment them out.

Here is an example of a CSV file:

String, Length
apple, 5
strawberry, 10
# commented line
cherry, 6

And here's an example of the @CsvFileSource annotation:

@CsvFileSource(resources = "/dataset.csv", numLinesToSkip = 1)

This way, you can use large sets of input data for your tests. If you are interested in detailed instructions on how to work with CSV argument sources, check out the corresponding sections of the official JUnit5 guide.

Conclusion

In this topic, you've learned about the concept of parameterized tests and familiarized yourself with the tools JUnit provides for working with them. Parameterized tests are a convenient tool for writing effective and concise tests. Instead of multiple test methods, you can have a single method denoted by @ParameterizedTests, which takes parameters so that you can supply different arguments to it. This allows you to reuse your code efficiently and improve the readability of your tests.

JUnit has many options to set up argument sources for your tests, including @ValueSource, @MethodSource, @CsvSource, and a number of other annotations. By combining different sources, you can use sets of input data of any size to cover as many test cases as possible. This allows for testing units of code with a very complex logic and extremely large numbers of execution paths, which otherwise could not be reliably tested.