Web Application Factory VS Real HTTP Calls - Take Two

I’d say that Web-Application-Factory-based integration tests are really close to the bottom border of the "integration" section of the test pyramid.

You might want to have a good level of isolation to be able to run them as a part of the CI/CD pipeline and .NET provides many options such as:

• Mocking services entirely;
• Using EF Core In-Memory Database Provider or SQLite-InMemory;
• Using Docker for tests.

By favouring isolation, however, we sacrifice the accuracy of testing. Issues related to the transport layer, third-party services, or DB can’t be found when using mocks because all of these are isolated from tests. On the other hand, the higher we climb the test pyramid the more expensive and fragile the tests become. Luckily, it’s possible to reuse a lot of code that we already have to build integration tests that would adjoin the upper side of the “integration tests” section of the pyramid.

Let’s say we have a Post controller method implemented somewhat like this:

public async Task<PostResult> Post(InputParams input) {
    var valueFromDb = await GetValueFromDbAsync(input);
    var valueThirdPartyService = await
        GetValueFromThirdPartyServiceAsync(valueFromDb);

    return new PostResult(
        valueFromDb, valueThirdPartyService);
}

Both GetValueFromDbAsync and GetValueFromThirdPartyServiceAsync are isolated by injecting mock services to the web application factory. To make sure that the controller method works not only with mocks but with the real DB and third-party service we are going to implement a test that would call the real API endpoint deployed to the test environment.

The code of the web-application-factory-based test works just fine

public class MyControllerTest:
    ControllerTestBase<MyController> 
{
    [Theory]
    [ClassData(typeof(SophisticatedDataProvider))]
    public async Task ReturnsSucces(
        InputParams input, ExpectedOutput expectedOutput)
    {
        // Arrange
        var client = _factory.CreateControllerClient();

        // Act
        var outputData = await client.SendAsync(
            (MyController c) => c.Post(input));

        // Assert
        ComplexAssertions(expectedOutput, outputData);
    }
}

except for this time we need the real HttpClient to do real calls to the real environment unlike the HttpClient implementation provided by the Web Application Factory.

Applying Template Method Pattern

Let’s refactor the original test by applying Template Method pattern.

public abstract class HttpClientTestBase<T>
    where T: IHttpClientFactory
{
    protected readonly T HttpClientFactory;

    public HttpClientTestsBase(T http)
    {
        HttpClientFactory = http;
    }

    protected HttpClient CreateHttpClient(string name = "")
        => HttpClientFactory.CreateClient(name);
}

public class ControllerTestBase<
    TController,
    THttpClientFactory
>:
    HttpClientTestsBase<THttpClientFactory> 
    where THttpClientFactory : IHttpClientFactory
{
    public ControllerTestBase(THttpClientFactory http) :
        base(http) {}

    public ControllerClient<TController>
        CreateControllerClient(string name = "") => 
            new(CreateHttpClient(name));
}

public abstract class MyControllerTestBase:
    ControllerTestBase<MyController, TFactory> 
    IClassFixture<TFactory> 
    where TFactory : class, IHttpClientFactory
{
    protected MyControllerTestBase(THttpClientFactory http) :
        base(http) {}

    [Theory]
    [ClassData(typeof(SophisticatedDataProvider))]
    public async Task ReturnsSucces(
        InputParams input, ExpectedOutput expectedOutput)
    {
        // Arrange
        var client = CreateControllerClient();

        // Act
        var outputData = await client.SendAsync(
            (MyController c) => c.Post(input));

        // Assert
        ComplexAssertions(expectedOutput, outputData);
    }

Complete source code of HttpClientTestBase and ControllerTestBase is available on GitHub.

Albeit CreateControllerClient method is not abstract, it only delegates all work to HttpClientFactory, which in turn will be provided in the derived class. Thus, this is a variant of the “template method” implementation that uses composition instead of inheritance.

public class MyControllerWafTest:
    MyControllerTestBase<MoqHttpClientFactory>
{
    MyControllerWafTest(MoqHttpClientFactory factory):
        base(factory){}
}

public class MyControllerHttpTest:
    MyControllerTestBase<HttpClientFactory>
{
    MyControllerWafTest(HttpClientFactory factory):
        base(factory){}
}

Test frameworks such as xUnit or NUnit will skip the base class since it’s abstract but will execute tests from both derived classes. As a result, we shared the test code between web-application-factory-based and real-http-client-based tests.

Configuring Mocks

For the web-application-factory-based test we need mocks set up. Microsoft recommends using ConfigureTestServices to do so. That’s a nice and convenient way to override service registrations, but in order to keep as much code as possible shared between web-application-factory-based and real-http-client-based tests we'll move mock registration from the test method to the class constructor.

MyControllerWafTest:
    MyControllerTestBase<MoqHttpClientFactory>
{
    MyControllerWafTest(MoqHttpClientFactory factory):
        base(factory)
    {
        _factory.ConfigureMocks(nameof(ReturnsSucces), m => {
            m
               .Mock<ISomeRepository>
               .Setup(x => x.GetAll())
               .Returns(new [] {/*...*/});

        });
    }
}

[Theory]
[ClassData(typeof(SophisticatedDataProvider))]
public async Task ReturnsSucces(
    InputParams input, ExpectedOutput expectedOutput)
{
    // Arrange
    var client =
        CreateControllerClient(nameof(ReturnsSucces));

    // Act
    var outputData = await client.SendAsync(
        (MyController c) => c.Post(input));

    // Assert
    ComplexAssertions(expectedOutput, outputData);
 }

The code above benefits from the IHttpClientFactory definition. Mocks for each named instance of HttpClient are configured independently. We only needed to update the client creation code to get the named HttpClient instance with mocks configured for this method:
var client = CreateControllerClient(nameof(ReturnsSucces));

What if I don’t want to share some tests

Just don’t do it. Put tests that you want to run in both modes into the base class and add feel free to add additional tests to derived test classes when needed. The complete code can be found on GitHub. You can also use the nuget package for you projects.

A note on modular architecture

This type of tests is better aligned with modular architecture.