A pyramid scheme

Most people may be familiar with the testing pyramid, a concept originally developed by Martin Cohn but popularised by Martin Fowler. For those that are not, it attempts to convey how tests should be distributed within a project - lots of fast, cheap and easy to write unit tests, thinning out (i.e. not duplicated at each layer) as we go further up the pyramid, ultimately to more complex, time consuming and potentially brittle automated GUI tests. This guide will talk through applying the test pyramid to a Spring Boot web service. The service is a typical RESTful service - it exposes API endpoints, has a number of services that implement the required business logic and surfaces data maintained in a traditional SQL database.

Automated Unit Tests

A relatively simple definition (a more thorough discussion can again by found by referring to Martin Fowler) of a unit test is that its a technique that checks that a single unit of code performs the expected behaviour. Numerous articles show contrived examples to explain this. However, this guide, as previously stated, is focused around a Spring Boot service - so lets dive into some real examples!

Database

To access our data store we utilise JPA. The repository below provides a sample query against a simple UpdateLog entity:

public interface UpdateLogRepository extends JpaRepository<UpdateLog, Long> {

    @Query("FROM UpdateLog ul " +
            "WHERE ul.status = :status " +
            "ORDER BY ul.createdDate")
    List<UpdateLog> findRequestsInState(@Param("status") UpdateLog.Status status);
}

From the above we can see that there are a number of things that we might want to test:

The Long data type in JpaRepository<UpdateLog, Long> is the type of primary key for the underlying entity. We need to ensure that the auto-generated JPA methods (such as findById) do indeed work when a Long data type is provided.
We need to ensure that the method returns something when all query parameters are valid. We can see that the UpdateLog.Status is most likely an enum
Equally, we want to check that nothing is returned if data isn’t present in the correct state e.g. the table is empty
Finally, we want to ensure that the data is returned in date order

From this single unit of code we can see that there are a number of unit tests that we need to perform. Again, once configured its very easy and relatively quick to test all of the required conditions.

A sample test case that covers one of the above scenarios is below:

@Test
void findRequestsInStateShouldReturnPendingRequestsOrderedByDate() {
    // Arrange
    final UpdateLog updateLog = buildUpdateLog(...);
    testEntityManager.persistAndFlush(updateLog);
    testEntityManager.clear();

    final UpdateLog otherUpdateLog = buildUpdateLog(...);
    testEntityManager.persistAndFlush(otherUpdateLog);
    testEntityManager.clear();

    // Act
    final List<UpdateLog> found = testSubject.findRequestsInState(UpdateLog.Status.PENDING);

    // Assert
    assertThat(found.isEmpty(), is(not(true)));
    assertThat(found, contains(
            samePropertyValuesAs(updateLog),
            samePropertyValuesAs(otherUpdateLog)
    ));
}

To summarise the above:

Uses @DataJpaTest Spring annotation to indicate to Spring that we are testing a JPA class and it should only load the necessary components
Has @Autowired the TestEntityManager and UpdateLogRepository dependencies
Uses Junit Jupiter denoted by the @Test annotation for running the test
Persist test data to an in memory H2 database via the autowired TestEntityManager
Execute the desired repository method to fetch the inserted data
Use the Hamcrest library to provide matchers for asserting (checking conditions). For example the contains matcher ensures that a collection contains the required objects in the specified order

Service

To re-enforce the above, lets look at another unit of code - this time one of our business logic services. An example service is provided below:

@Service
@AllArgsConstructor
public class UpdateLogService {

    private final UpdateLogRepository updateLogRepository;

    @Override
    public List<UpdateLog> findUpdateLogs() {
        return updateLogRepository.findRequestsInState(UpdateLog.status.PENDING);
    }
}

From the above we want to ensure that our business logic works as intended. Again to achieve this we can make use of automated unit tests:

@ExtendWith(MockitoExtension.class)
class UpdateLogServiceTest {
    @InjectMocks
    private UpdateLogService testSubject;

    @Mock
    private UpdateLogRepository updateLogRepository;

    @Test
    void shouldGetPendingUpdateLogs() {
        // Arrange
        final UpdateLog expectedResponse = UpdateLog.builder().build();
        when(updateLogRepository.findRequestsInState(UpdateLog.status.PENDING)).thenReturn(Collections.singletonList(expectedResponse));

        // Act
        final List<UpdateLog> response = testSubject.findUpdateLogs();

        // Assert
        assertThat(response, contains(expectedResponse));
        verifyNoMoreInteractions(updateLogRepository);
    }
}

We will again walk through the above:

Make use of mockito to mock external dependencies of the unit. This is useful as it enables us stub responses from external dependencies - allowing us to test all possible responses from that dependency programmatically e.g. the typical response, empty responses or potential error states.
Make use of @InjectMocks annotation to inject any of our mocked external dependencies (via constructor injection)
Mock any required dependencies though @Mock
Again use Junit Jupiter to run tests (as indicated by @Test)
Use when to invoke the mocked object (asserting that the correct parameters are passed during execution, if not the test will fail) and thenReturn to return a desired response for this invocation
Use Hamcrest to assert that the desired outcome occurred

Automated Component tests

Moving up through the pyramid, the next layer focuses on testing components. A component is sometimes referred to as a module. You can think of a component as a logical block slightly larger than a unit (it will be made up of a number of units). Again we want to ensure that any component functions as expected.

Downstream services

A good example of component testing is checking that a call to a downstream service functions as expected. An example of this might look like:

@Service
@RequiredArgsConstructor
public class AService {
    private final WebClient aServiceWebClient;

    public void anAction(final ARequest request, final RequestMetadata requestMetadata) {
        postAction("/a/list", request, requestMetadata);
    }

    private void postAction(final String uri, final Object payload, final RequestMetadata requestMetadata) {
        aServiceWebClient.post()
                .uri(uri)
                .header(HttpHeaders.AUTHORIZATION, "Bearer " + requestMetadata.getForwardingAuth())
                .header(REQUEST_ID_HEADER, requestMetadata.getRequestId().toString())
                .accept(MediaType.APPLICATION_JSON)
                .bodyValue(requestPayload)
                .retrieve()
                .onStatus(HttpStatus::isError, this::handleHttpStatusError)
                .toBodilessEntity()
                .onErrorMap(this::isUnhandledException, thrown -> new AServiceException(UNEXPECTED_ERROR_MESSAGE, thrown))
                .block();
    }
}

We can test this service as follows:

@JsonTest
class AServiceTest {
    private AService auditService;
    private MockWebServer mockWebServer;

    @Autowired
    private ObjectMapper objectMapper;

    @BeforeEach
    void setUp() throws IOException {
        mockWebServer = new MockWebServer();
        mockWebServer.start();

        final WebClient webClient = WebClient.builder()
                .baseUrl("http://localhost:" + mockWebServer.getPort())
                .codecs(configurer -> {
                    configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(objectMapper));
                    configurer.defaultCodecs().jackson2JsonEncoder(new Jackson2JsonEncoder(objectMapper));
                })
                .build();
        aService = new AService(webClient);
    }

    @AfterEach
    void tearDown() throws IOException {
        mockWebServer.shutdown();
    }

    @Test
    void shouldCallAnAction() throws Exception {
        // Arrange
        final ARequest request = buildARequest();
        final RequestMetadata requestMetadata = buildRequestMetadata();

        mockWebServer.enqueue(new MockResponse().setResponseCode(HttpStatus.NO_CONTENT.value()));

        // Act
        aService.anAction(request, requestMetadata);

        // Assert
        final RecordedRequest recordedRequest = mockWebServer.takeRequest();
        assertThat(recordedRequest.getMethod(), is(HttpMethod.POST.name()));
        assertThat(recordedRequest.getPath(), is("/a/list"));
        assertThat(recordedRequest.getHeader(HttpHeaders.AUTHORIZATION), is("Bearer " + AUTH));
        assertThat(recordedRequest.getHeader(HttpHeaders.ACCEPT), is(MediaType.APPLICATION_JSON_VALUE));
        assertThat(recordedRequest.getHeader(HEADER_REQUEST_ID), is(REQUEST_ID.toString()));

        final String expectedBody = readClassPathResource("api/request.json");
        JSONAssert.assertEquals(expectedBody, recordedRequest.getBody().readUtf8(), JSONCompareMode.STRICT);
    }
}

As you can see from the above there is quite a bit going on. One of the main things to note is that the service itself is not mocked. Instead we make use of MockWebServer to intercept requests sent through the component.

The above example makes use of @BeforeEach and @AfterEach annotations to setup and tear-down the mock server between test executions. This prevents previous test executions contaminating future invocations. We also use the @BeforeEach to instantiate our component.

Again, Junit 5 is used to execute the test. MockWebServer is able to observe requests sent to the server. This allows us to inspect the request. Through this we can perform a number of actions, such as, recordedRequest.getMethod() which checks that the HTTP request sent out by our service was a HTTP POST. Hamcrest is used as our assertion library of choice.

The final part of the test checks that the POST’ed request body matches a known JSON body (made available in the resources directory of of project). To do this we utilise JSONAssert.

Integration Tests

The next layer of the pyramid focuses on integration testing. Integration testing aims to test all of our interconnected components, without mocking or stubbing dependencies, other than those external to our service e.g. downstream services or databases. Again, we have provided a sample integration test below:

@SpringBootTest(
        webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
        classes = {TestDBConfiguration.class}
)
@ActiveProfiles({"test", "text-logging"})
class AServiceIT {

    @LocalServerPort
    private int port;

    @Value("${a-test-port}")
    private int aTestPort;

    private MockWebServer mockAService;

    @Autowired
    private WebTestClient webTestClient;

    @Autowired
    private Clock clock;

    @Autowired
    private ObjectMapper objectMapper;

    @Autowired
    private TransactionTemplate transactionTemplate;

    @Autowired
    private TestEntityManager testEntityManager;

    @BeforeEach
    void setUp() throws IOException {
        mockAService = new MockWebServer();
        mockAService.start(aTestPort);
    }

    @AfterEach
    void tearDown() throws IOException {
        mockAService.shutdown();
        tearDownDb();
    }

    @Test
    private void getListShouldReturnAList() throws IOException, InterruptedException {
        // Arrange
        final UUID requestId = UUID.randomUUID();
        setUpUpdateLogs(1L, buildUpdateLog());

        final AResponse expectedResponse = AResponse.builder()
                .build();

        // Act
        webTestClient.get()
                .uri(getURI(BASE_PATH, A_PATH))
                .headers(standardHeaders(requestId, JWT))
                .exchange()
                .expectStatus().isOk()
                .expectHeader().contentType(MediaType.APPLICATION_JSON)
                .expectBody(AResponse.class).isEqualTo(expectedResponse);

        // Assert
        final RecordedRequest request = mockAService.takeRequest();
        assertThat(request.getMethod(), is("POST"));
        assertThat(request.getPath(), is("/a/list"));
        assertThat(request.getHeader(HttpHeaders.AUTHORIZATION), is("Bearer " + JWT));
        assertThat(request.getHeader(HttpHeaders.ACCEPT), is(MediaType.APPLICATION_JSON_VALUE));
        assertThat(request.getHeader(HEADER_REQUEST_ID), is(requestId.toString()));
        assertThat(request.getHeader(HEADER_REQUEST_SIGNATURE), notNullValue());
        assertThat(request.getHeader(HEADER_REQUEST_SIGNATURE_KEY_ID), notNullValue());

        final ARequest actual = objectMapper.readValue(request.getBody().readUtf8(), ARequest.class);
        assertThat(actual, is(ARequest.builder().build()));
    }
}

A number of things are happening in the above code:

We configure the tests to start all of the necessary Spring Boot internals required to run a full integration test.
We again make use of OK HTTP’s MockWebServer to mock our downstream external services
Autowire required dependencies, such as the TestEntityManager which will be used for writing data to our in memory database.
Test fixture setup and tear-down are added to run before each test executes (as denoted by @BeforeEach and @AfterEach respectively). In these methods we instantiate/clean-up our mocked external dependencies and clean down the database between test executions.
JUnit is again used to execute the test
The test inserts some data into our database required to make the test run (via setUpUpdateLogs)
We invoke our service and check that the returned response is what we expected.
As an added bonus, as covered previously, we use MockWebServer to monitor requests sent to external mocked services.

API tests

Phew, we have reached the final layer of the pyramid for our sample application! This layer looks at executing against the entire stack - the service, database, dependent services etc. These tests typically simulate end user interactions and are often referred to as End-to-End (E2E) tests.

For executing E2E API tests we do not need to use the same programming language as used by the rest of our application. In this instance we could choose to use Python, more specifically pytest. Pytest is used for a number of reasons, but here are some high-level highlights:

Flexibility: Pytest allows us create scoped fixtures (e.g. at a session, module or method level). This removes the need for tear-down methods typically present in other popular frameworks e.g. TestNG
Fixtures: Pytest is capable of creating parametrised test cases through the use of annotations This enables us to write a single test function that can be used to test multiple scenarios
Discoverability: Pytest is excellent at discovering test cases. It is able to execute a subset of tests, such as, an individual test case, a class of tests, a directory of tests, tests with a name that matches a particular pattern and more
Logging: Pytest clearly outputs which of the executed tests fail and why!

A full example of how to utilise pytest is beyond the scope of this blog. For now you can find a good example of how to use pytest here - it covers building an API in Python and testing it through pytest.

A pyramid scheme#

Automated Unit Tests#

Database#

Service#

Automated Component tests#

Downstream services#

Integration Tests#

API tests#