Mocking dependencies in Go

#go #tests #mock

Go makes obvious, non-magical dependency injection easy. Lots of new developers, or experienced people coming from other languages often run into troubles when non-simple dependencies are introduced such as databases and third party services (eg. API clients). This is where I along with many others get stuck and need to find a way to test code without hitting the dependencies code. I intend to demonstrate one strategy that has worked well for me so far.

Let's start by taking a thousand foot view of the problem we are going to solve. Our software uses a third party service which is accessed by a restful HTTP API. Every call we make to this API costs money making it unfeasible to call the API with our unit tests. Our goal is to be able to switch this API client out with one we control during our test cycle.

Let's say we have a main.go that looks like:

// main.go
package main 

import thirdPartySvc "url.com/thirdsPartySvcClient"
import "myservice"

// returns a new third party service client
client, err := thirdPartySvc.New("OurAPIKey")
handleError(err)

// we create a new instance of our service
svc := myservice.New(client)
// start the service
svc.Start()

and our myservice/svc.go file looks like:

// myservice/svc.go
package myservice

import thirdPartySvc "url.com/thirdsPartySvcClient"

type MyService struct {
    client thirdPartySvc.Client
}

func New(client thirdPartySvc.Client) MyService {
    return MyService{client}
}

func (s *MyService) Start() {
    // start the service
    data := s.client.Get()
    // do something with `data`
    data = data + "-validated"
    return data
}

This looks pretty good right? If you've already wallowed with the problem that is about to unveil itself you may already be cringing. The issue with this setup is that the myservice package has a dependency on the thirdPartySvc library. When we test it, we have to pass in a struct created from that library which will want to call the real world service.

Our first step to making this testable is to remove the thirdPartySvc dependency. This can be achieved with an interface.

// myservice/svc.go
package myservice

type ThirdPartyInterface interface {
    // functions that we use from the third party service client
    func Get() string
}

type MyService struct {
    client ThirdPartyInterface
}

func New(client ThirdPartyInterface) MyService {
    return MyService{client}
}

func (s *MyService) Start() string {
    // start the service
    data := s.client.Get()
    // do something with `data`
    data = data + "-validated"
    return data
}

We added an interface that defines only the functions that we use in the thirdPartySvc struct. This is cool for a couple reasons, there is no hard dependency on the external library, we can pass in any struct that matches the interface signature, and we know exactly what functions are being consumed and how they are being used.

Now we can test our MyService.Start() function without calling the external service.

// svc_test.go
package myservice

type MockThirdPartySvc struct {}

func (m *MockThirdPartySvc) Get() string {
    return "test-string"
}

func TestMyService_Start(t *testing.T) {
    expected := "test-string-validated"
    startString := New(MockThirdPartySvc{})

    if startString != expected {
        t.Errorf("Expected %s but got %s", expected, startString)
    }
}

This test works because our MockThirdPartySvc struct functions match those the ThirdPartyInterface defined in svc.go. Note, that while the thirdPartySvc struct probably has way more functions then we use, we only implement the ones we need, making testing easier and understanding the scope of how we use that library.

At this point you may be saying, this is great and all but starts to break down when I have many packages that use varying combinations of the functions that are offered by a library. Creating complex mocks in every package is a ton of work and painful to maintain. And you're totally correct.

I use a two part approach that creates a mocks package that can be accessible across all of your tests and contains the most complete API interface that the project uses. Diving in a bit deeper, public structs defined in *_test.go files are not accessible to other test files. Creating a mock package allows this accessibility. By defining a interface in your mock package that includes all of the functions your codebase uses for that library you guarantee it will work even though in your non-test code you define subset interfaces.

The implementations of interfaces in the mocks package can be created in all sorts of clever ways, one way is to hand create simple implementations where the caller can pass in functions.

package mocks

type ThirdPartyInterface interface {
    // all of the functions we use from the third party client
    func Get() string
    func Post(string) error
}

type ThirdPartyMock struct {
    GetFunc: func() string
    PostFunc: func(string) error
}

func (m *ThirdPartyMock) Get() string {
    return m.GetFunc()
}

func (m *ThirdPartyMock) Post(in string) error {
    return m.PostFunc(string)
}

This method lets the test define the inner functions as needed for that specific test case so the same mock can be used for a set of tests with bespoke outcomes.

Mock Generation

This method of mock creation is tedious and can be made easier with tools and code generation. Some tools like golang/mock or my preferred library matryer/moq.

Generations can be done in a few ways, including one off generations, inline generations, or as part of a build command. One off generations are, merely a command being run (eg. moq -out ThirdParty_test.go -pkg mocks myservice ThirdPartyInterface).

There is also inline generation where you annotate the mocks that are needed (eg. //go:generate moq -out ThirdParty_test.go -pkg mocks myservice ThirdPartyInterface) which will generate the mocks when go generate is run.

The last and my preferred method is identifying the mocks that are wanted and generating them externally. An example of this can be found in the digitalocean/doctl project, a regenmocks.sh script that is wired up to their Makefile, specifically:

# executed with make mocks
.PHONY: mocks
mocks:
    @echo "==> update mocks"
    @echo ""
    @scripts/regenmocks.sh

A note for library authors

I often find including a complete public interface with a library is a nice courtesy to developers who are using your libraries. I believe that interfaces should be created where they are used and libraries return structs (eg. creation of a client). That being said having a complete interface in the library is a nice way for consumers to easily generate mocks. If your library is going to be commonly mocked out for testing you may even want to consider adding a mocks directory with generated mocks, you can use those mocks for testing the library internally and your consumers can use them too as a nice bonus.

Final

I hope this painted a picture on one way to improve the quality of your tests and make your life a bit easier when it comes to testing code that has dependencies that are hard to test. If you have any feedback or questions, feel free to reach out to me on twitter or email jon@jonfriesen.ca.