JUnit Tutorial

Parameterised Tests

Literal value inputs

• This is the most basic parameterisation method for instantiating and inserting test data, useful for inserting in literals of all java primitive and data types, as well as testing with nulls (and empty values for strings).

/*

* PARAMETERISED TEST METHODS

* */

@ParameterizedTest(name = "Run: {index} => {arguments}")

@NullSource

@EmptySource

// @NullAndEmptySource should be used instead of the two annotations combination above

// @ValueSource can take strings, ints and all other primitive types

@ValueSource(strings = {"This", "is", "a", "Parameterized", "Test"})

void emptyTest3(String params) {

   System.out.println(params);

}

@ParameterizedTest(name = "Test Case: {index} => Value: {arguments}")

@NullSource

// @EmptySource can only be used for strings,

// @NullSource must use ints @ValueSource with Integer wrapper class type as local parameter

@ValueSource(ints = {1,2,3,4,5})

void emptyTest4(Integer params) {

   System.out.printf("This is Parameterised Test Case: %d\n", params);

}

Comma Separated Values inputs

• Values of any primitive / data type can be inserted into a given test method using the csv format. 

Note: the values are Strings that are then cast / transformed into the java type by the test method.

// Using @CsvSource annotation

// With just one value type

@ParameterizedTest(name = "Test Case: {index} => {arguments}")

@CsvSource(value = {"This,is,a,CSVSource,Test"})

void emptyTest5_StringStringStringStringString(String p1, String p2, String p3, String p4, String p5) {

   System.out.printf("%s %s %s %s %s\n", p1, p2, p3, p4, p5);

}

// With both strings and integer type values

@ParameterizedTest(name = "Test Case: {index} => {arguments}")

@CsvSource(value = {"This,is,CSVSource,Test,-,number,2"})

void emptyTest6_StringStringStringStringStringStringInteger(String s1, String s2, String s3, String s4, String s5, String s6, int i1) {

   System.out.printf("%s %s %s %s %s %s %d\n", s1, s2, s3, s4, s5, s6, i1);

}

// With a strings, integers and use of '' to escape a comma from being delimited

@ParameterizedTest(name = "Test Case: {index} => {arguments}")

@CsvSource(value = {"This,is,CSVSource,'Test,',number,3"})

void emptyTest7_StringStringStringStringStringStringInteger(String s1, String s2, String s3, String s4, String s5, int i1) {

   System.out.printf("%s %s %s %s %s %d\n", s1, s2, s3, s4, s5, i1);

}

// Replacing comma with another delimiter token, in this case a semi-colon ';'

@ParameterizedTest(name = "Test Case: {index} => {arguments}")

@CsvSource(value = {"This;is;CSVSource;Test,;number;4"}, delimiter = ';')

void emptyTest8_StringStringStringStringStringStringInteger(String s1, String s2, String s3, String s4, String s5, int i1) {

   System.out.printf("%s %s %s %s %s %d\n", s1, s2, s3, s4, s5, i1);

}

@ParameterizedTest(name = "Test Case: {index} => {arguments}")

@CsvSource(value = {"This---is---CSVSource---Test,---number---5"}, delimiterString = "---")

void emptyTest9_StringStringStringStringStringStringInteger(String s1, String s2, String s3, String s4, String s5, int i1) {

   System.out.printf("%s %s %s %s %s %d\n", s1, s2, s3, s4, s5, i1);

}

• You can also insert csv values from a file:

// Using @CsvFileSource annotation

@ParameterizedTest

@CsvFileSource(files = "src/test/resources/testData.csv", numLinesToSkip = 1)

void emptyTest10_StringStringStringInteger(String name, String address, String contact, int age) {

   System.out.printf("Name = %s, Address = %s, Contact = %s, Age = %d\n", name, address, contact, age);

}

• Files can be kept anyway but in order to stick to the best practices convention, place it under the src/test/resources area:

Test Order Sequencing

• The @TestMethodOrderer annotation can be used to set the order by the test method name, display name, randomly or (probably the most appropriate one) by a tagging each test with an @Order([n]) where n is a positive integer representing the order in which the tests will run; e.g. 1 would be before 2 which is run before 3, etc. :

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)

• The @Order tag is placed before each @ParameterizedTest / Test:

@Order(1)

@Test

@DisplayName("Placeholder Test2: Placeholder for actual test")

void emptyTest2() {

   System.out.println("This is a test @Test!");

}

/*

* PARAMETERISED TEST METHODS

* */

@Order(2)

//Using @ValueSource annotation

@ParameterizedTest(name = "Run: {index} => {arguments}")

@NullSource

@EmptySource

// @NullAndEmptySource should be used instead of the two annotations combination above

// @ValueSource can take strings, ints and all other primitive types

@ValueSource(strings = {"This", "is", "a", "Parameterized", "Test"})

void emptyTest3(String params) {

   System.out.println(params);

}

Extending Assertions

• Hamcrest is a very useful match and assert library that can be added to any test class through adding the library to the project via the package manager:

<dependency>

   <groupId>org.hamcrest</groupId>

   <artifactId>hamcrest</artifactId>

   <version>2.2</version>

</dependency>

• And then mporting it to any given test class import org.hamcrest.Matchers;

• You can then use it to assert many test outcomes based on some very useful and powerful matcher methods:

@Order(14)

@ParameterizedTest

@MethodSource(value = "org.example.GeneratedTestData#sourceStream_StringDouble")

void emptyTest14_StringDoubleStream(String constant, Double value) {

   System.out.printf("Constant %s has a value of %.8f to 8 decimal places\n", constant, value);

   var testData = new GeneratedTestData(); // Instantiate a Generated Test Data object

   testData.setExpectedValues(); // Set the expected key / value pairs to test against

   // get the test data value based on constant parameter and evaluate

   // against the actual resulting value derived from the same constant

   assertThat(testData.getMap(),Matchers.hasKey("pi"));

   // assertAll method can be used to group several assertions for a test outcome together more succinctly

   assertAll(

           () -> assertThat(testData.getMap(),Matchers.hasKey("e")),

           () -> assertThat(testData.getMap(),Matchers.hasKey("Root2"))

   );

}

Configuring Test Runs

• Tests can be run in various combinations, through the powerful build configuration tool within an IDE:

  • Run multiple Test Classes

  • Run a single Test Class

  • Run a group of Tests within a Test Class

  • Run a single Test

• Tag type annotations can be used to group tests based on logical groupings such as test type:

  • @unit = unit test tag annotation group member

  • @acceptance = acceptance test tag annotation group member

• Add the @Tag("unit") tag above the Class declaration statement in the CalculatorTest.Class

• Add the @Tag("acceptance") tag above a couple of the tests in the Annotations.Class

• Create a new junit build configuration called unit tests

• Add Tag "unit" and then select apply | ok

• copy this run config and rename the copy run acceptance tests

• replace the unit tag with acceptance tag value and then save this configuration too

• Running the run unit tests run configuration, results in only the calculator tests being run, as denoted by the unit tag at the test class level

• Running the run acceptance tests run configuration, results in only two of the annotation class tests being run, as denoted by the acceptance tag above each of those tests.

Conditional Test Runs

@Annotations

• Tests can be run based on meeting certain conditions (e.g. Only execute tests if the input meets a certain criteria, such as amount spent < 1000) using the @Assumptions tag.

/*

* ASSUMPTIONS

**/

// Assumptions can be used to determine whether to run a test based value comparisons

// The test below will run all test cases that input a value less than 5; i.e.

// AssumingThat outputs a message if the value is 4 or 5

// AssumeTrue throws an exception and stops the execution of this test

@Order(15)

@ParameterizedTest(name = "Test Case: {index} => Value: {arguments}")

@ValueSource(ints = {1, 2, 3, 4, 5})

void emptyTest15(Integer params) {

   assumingThat(params > 3, () -> System.out.println("\nUsing value: " + params + "\n"));

   assumeTrue(params < 5);

   System.out.printf("This is Parameterised Test Case: %d\n", params);

}

@Disabled

Tests can be disabled, using @Disabled annotation at the head of each test where it is applicable, but it can also be extended to disable tests based on attributes such as system or environmental properties:

• Add a new test with @DisabledIfSystemProperty set to disable this test if at runtime the system property "env" is set to "prod" (Useful for stopping destructive tests from running on production:

/*

* DISABLED TESTS: Disable test based on annotation value

*/

@Order(16)

@DisabledIfSystemProperty(named = "env", matches = "prod", disabledReason = "Disabled test as this is a PRODUCTION RUN")

@ParameterizedTest(name = "Test Case: {index} => Value: {arguments}")

@ValueSource(ints = {1, 2, 3, 4, 5})

void emptyTest16(Integer params) {

   System.out.printf("This is Parameterised Test Case: %d\n", params);

}

• Now set the System property at runtime by editing the run configuration in the IDE (For IntelliJ):

• Run the test - as part of a suite:

• The test will not be executed, and will be displayed as such in the console output:

@RepeatedTests

• This annotation can be used instead of the @ParameterizedTest one, when the objective is to loop theough a sequence 'n' number of times, the n value can also be used as an argument passed into the test method:

/*

* REPEAT TEST EXECUTION: Similar to the for loop in JAVA, use counter 1 to n,

* where n is the value set in the annotation's value argument

*/

@Order(17)

@RepeatedTest(value = 5, name = "Running test: {currentRepetition} of (totalRepetitions}")

void emptyTest17(RepetitionInfo repCount) {

   System.out.printf("This is Parameterised Test Case: %s\n", repCount);

}

Reporting with Event Listeners

• JUnit uses reporting plugins such as Maven SureFire, to capture reporting output from each test execution and outcome event.

• This method of capturing can be customised through declaring a listener class that overrides the default methods

• Create a Listener.Class in a new separate package within the same src/test/java classpath and set the class to implement the JUnit TestWatcher module:

package listeners;

import org.junit.jupiter.api.extension.TestWatcher;

public class Listener implements TestWatcher {

}

• Use the IDE tools to implement all of the existing methods (this will automatically add overridden existing methods provided JUnit

• Add custom console output messages for each method and then run the tests again to see this output appear based on each type of event:

package listeners;

import org.junit.jupiter.api.extension.ExtensionContext;

import org.junit.jupiter.api.extension.TestWatcher;

import java.util.Optional;

public class Listener implements TestWatcher {

   @Override

   public void testDisabled(ExtensionContext context, Optional<String> reason) {

       TestWatcher.super.testDisabled(context, reason);

       System.out.printf("\nContext: %s\n Reason: %s \n", context, reason);

   }

   @Override

   public void testSuccessful(ExtensionContext context) {

       TestWatcher.super.testSuccessful(context);

       System.out.printf("\nTest %s ran successfully!!!, \nContext: %s\n", context.getTestMethod(), context.getTestClass());

   }

   @Override

   public void testAborted(ExtensionContext context, Throwable cause) {

       TestWatcher.super.testAborted(context, cause);

       System.out.printf("\nTest %s was ABORTED!!!, \nCause: %s\n", context.getTestMethod(), cause.getCause());

   }

   @Override

   public void testFailed(ExtensionContext context, Throwable cause) {

       TestWatcher.super.testFailed(context, cause);

       System.out.printf("\nTest %s FAILED!!!, \nCause: %s\nDetails: %s", context.getTestMethod(), cause.getCause(), cause.getMessage());

   }

}

Setting Timeouts for Tests

• An @Timeout tag can be used to state how long a test execution can run for before it will be considered failed, the default values for this tag are in seconds, however any time unit available in Java can be used instead:

/*

* TIMEOUT: Constrain the running time for a test in milliseconds, seconds, minutes, etc.

* very useful for testing validating things like system response time criteria

*/

@Timeout(4) //default unit for timeout is seconds

@Order(18)

@RepeatedTest(value = 5, name = "Running test: {currentRepetition} of {totalRepetitions}")

void emptyTest18(RepetitionInfo repCount) throws InterruptedException {

   System.out.printf("This is Parameterised Test Case: %s\n", repCount.getCurrentRepetition());

   Thread.sleep(repCount.getCurrentRepetition()* 1000L);

}

• The test above will timeout after 4 seconds execution time and will execute 5 times, using the repetition count value to pause for that many seconds.

• This means that tests 4 and 5 will timeout and result in failures: