API testing with simulation Vol.2

In Part 1, we introduced Hoverfly as a tool for API simulation and explored its basic setup. We saw how it can capture API interactions and replay them during tests, making our test suite more reliable.

However, real-world applications present some real-world challenges that go beyond those basic examples. Let's be honest — if the simple scenarios from first part were sufficient for your needs, you probably don't need this series.

This part explores solutions to common challenges. We'll focus on specific scenarios that frequently appear in applications, and I believe you can adapt these examples for your own needs far beyond what we'll cover here.

Challenge One: Dynamic Data

Let's look at a test scenario:

{
    name: "create activity",
    testFunc: func(t *testing.T, client *ClientWithResponses) {
        var (
            ctx     = context.Background()
            id      = int32(1)
            title   = randomTitleForResource("activity")
            dueDate = time.Now()
        )

        resp, err := client.PostApiV1ActivitiesWithApplicationJSONV10BodyWithResponse(
            ctx,
            PostApiV1ActivitiesApplicationJSONV10RequestBody{
                Id:        tests.ToPtr(id),
                Title:     tests.ToPtr(title),
                DueDate:   tests.ToPtr(dueDate),
                Completed: tests.ToPtr(false),
            },
        )
        require.NoError(t, err)
        require.NotNil(t, resp, "expected resp, got nil")
        require.NotNil(t, resp.ApplicationjsonV10200, "expected resp body, got nil")

        assert.Equal(t, http.StatusOK, resp.StatusCode())
        assert.Equal(t, id, *resp.ApplicationjsonV10200.Id)
        assert.Equal(t, title, *resp.ApplicationjsonV10200.Title)
        assert.Equal(
            t,
            dueDate.UTC().Format(time.RFC3339),
            resp.ApplicationjsonV10200.DueDate.UTC().Format(time.RFC3339),
        )
        assert.Equal(t, false, *resp.ApplicationjsonV10200.Completed)
    },
}

With a helper function to generate random titles:

// Generate a random title to test the possibility of random data in Hoverfly
func randomTitleForResource(resource string) string {
    return fmt.Sprintf("%s-%s-title-%s", fakePrefix, resource, tests.RandomString(5))
}

A couple of points that may catch our attention:

• Random data generation: The test creates a unique random title for each test run
• Current timestamps: The test uses the current time as the dueDate

These scenarios are quite common in testing.

But when we run these tests with simulations, they'll fail. Why? Because our captured simulation contains specific values, but our test generates new ones each time. The simulation can't match the new requests to the captured responses.

Similarly, look at this test with UUIDs:

{
    name: "create activity with UUID in title",
    testFunc: func(t *testing.T, client *ClientWithResponses) {
        ctx := context.Background()

        // Create multiple activities with different UUIDs
        for i := 0; i < 3; i++ {
            id := int32(1 + i)
            title := uuid.New().String()

            resp, err := client.PostApiV1ActivitiesWithApplicationJSONV10BodyWithResponse(
                ctx,
                PostApiV1ActivitiesApplicationJSONV10RequestBody{
                    Id:        tests.ToPtr(id),
                    Title:     tests.ToPtr(title),
                    DueDate:   tests.ToPtr(time.Now()),
                    Completed: tests.ToPtr(false),
                },
            )
            require.NoError(t, err)
            require.NotNil(t, resp)
            require.NotNil(t, resp.ApplicationjsonV10200)
            assert.Equal(t, http.StatusOK, resp.StatusCode())
            assert.NotNil(t, resp.ApplicationjsonV10200)
            assert.Equal(t, title, *resp.ApplicationjsonV10200.Title)
        }
    },
}

Each iteration generates a new UUID, creating a unique request that won't match our captured simulation.

Understanding Hoverfly Simulations

Let's peek inside the simulation file. Hoverfly works by storing pairs of requests and responses. When it receives an incoming request, it compares it against stored requests to find a match, then returns the corresponding response. Here's what a captured request-response pair looks like:

{
  "request": {
    "path": [
      {
        "matcher": "exact",
        "value": "/api/v1/Activities/1"
      }
    ],
    "method": [
      {
        "matcher": "exact",
        "value": "GET"
      }
    ],
    "destination": [
      {
        "matcher": "exact",
        "value": "fakerestapi.azurewebsites.net"
      }
    ],
    "scheme": [
      {
        "matcher": "exact",
        "value": "https"
      }
    ],
    "body": [
      {
        "matcher": "exact",
        "value": ""
      }
    ]
  },
  "response": {
    "status": 200,
    "body": "{\"id\":1,\"title\":\"Activity 1\",\"dueDate\":\"2025-03-04T23:24:16.4781493+00:00\",\"completed\":false}",
    "encodedBody": false,
    "headers": {
      "Api-Supported-Versions": [
        "1.0"
      ],
      "Content-Type": [
        "application/json; charset=utf-8; v=1.0"
      ],
      "Date": [
        "Tue, 04 Mar 2025 22:24:15 GMT"
      ],
      "Hoverfly": [
        "Was-Here"
      ],
      "Server": [
        "Kestrel"
      ]
    },
    "templated": false
  }
}

Request Matchers

When Hoverfly captures a request, it creates a Request Matcher for each field in the request. A Request Matcher consists of:

1. The request field name (path, method, destination, etc.)
2. The type of match to use for comparison
3. The request field value

By default, Hoverfly sets the type of match to exact for each field, which means the incoming request must exactly match the stored request for each field. This is why random data in our tests breaks the simulation—the new random values won't exactly match the captured ones.

Hoverfly supports multiple matcher types including exact, glob, regex, and more. You can find details in Hoverfly's official documentation.

Response Templating

The templated field in the response determines whether Hoverfly should use templating. When enabled, you can inject data from the request into the response using templating syntax.

For example, this templated response includes the URL path from the original request:

"response": {
  "status": 200,
  "body": "You requested: {{ Request.Path.[0].Value }}",
  "encodedBody": false,
  "headers": {
    "Content-Type": ["text/plain"]
  },
  "templated": true
}

It's worth noting that Hoverfly's templating is far more powerful than just simple value injection. You can:

• Use conditional logic with {{if}}, {{else}}
• Extract data using JSONPath or XPath expressions
• Perform string manipulations (substring, lowercase, etc.)
• Generate random data, etc.

When Hoverfly receives a request, it will extract the values from the request body and inject them into the response, creating a consistent experience even with random data.

This is exactly what we need to solve our problems. By combining request matchers with templated responses, we can create simulations that work with any random values our tests generate.

For our dynamic data problem, we'll use these features to:

1. Change the matcher types for fields with dynamic data
2. Enable response templating to inject request values into responses
3. Apply these changes automatically with our simulation processor

Creating a Simulation Processor

Now that we understand how Hoverfly processes requests and responses, let's build a solution for our problems. In my code, I'll use the internal implementations from the Hoverfly package since it's open-source. However, remember this is just a JSON file, so if you use any language other than Go, you can easily implement the same approach.

The Goal

Our processor needs to achieve several objectives:

1. Identify dynamic data patterns in both requests and responses
2. Replace exact matchers with matchers for fields containing UUIDs, timestamps, prefixed random strings, etc.
3. Enable templating in responses to use values from the request
4. Support static responses for specific endpoints that need predetermined behavior

We'll use a declarative configuration file to define our processing rules, for example:

version: "1.0"
settings:
  debug: false

patterns:
  # All UUIDs will be replaced with the default regex pattern
  - type: "uuid"

  # Dates will be replaced
  - type: "datetime"
    formats:
      - "2006-01-02T15:04:05Z07:00"
      - "2006-01-02"

  # Prefix matching
  - type: "prefix"
    pattern: "fake-api-test-"
    length: 5

endpoints:
  # Replace the response of the endpoint with the static_response
  - method: "GET"
    path: "/api/v1/Activities/30"
    status: 200
    static_response: |
      {
        "id": 77,
        "title": "John Doe",
        "dueDate": "2025-02-19T17:12:52.127Z",
        "completed": false
      }

With this configuration, we aim to automatically transform our simulation files to handle dynamic data without manually editing them.

I don't want to go deep into the implementation details in this article (if you're interested, you can find everything in the code), so I'll just show you the main concepts.
Here's the entry point of the processor implementation:

func (p *PostProcessor) Process(simulation *v2.SimulationViewV5) error {
    if simulation == nil {
        return fmt.Errorf("nil simulation provided")
    }

    for i := range simulation.RequestResponsePairs {
        pair := &simulation.RequestResponsePairs[i]

        // 1. Check if this is a static endpoint rule
        if p.endpointProcessor != nil {
            if rule := p.endpointProcessor.FindMatchingRule(pair); rule != nil {
                if err := p.endpointProcessor.ApplyRule(pair, rule); err != nil {
                    return fmt.Errorf("applying static response for pair %d: %w", i, err)
                }
                continue
            }
        }

        // 2. Process the pair
        if err := p.processingStrategy.Process(pair); err != nil {
            return fmt.Errorf("processing pair %d: %w", i, err)
        }
    }

    return nil
}

Let's skip the static endpoint for now and look closer at the processing of each pair.
For prefixed patterns in requests:

func (p *PrefixProcessor) ProcessRequest(value string) (string, bool) {
    if p.replaceWith != "" {
        return p.replaceWith, true
    }

    // Properly escape any regex special characters in the prefix
    escapedPrefix := regexp.QuoteMeta(p.prefix)

    // Check if we detected a resource name embedded in the value
    if resourceName := p.detectResourceName(value); resourceName != "" {
        p.includeResourceName = true
        p.resourceName = resourceName

        // Pattern with resource name included
        return fmt.Sprintf("%s%s-[a-zA-Z0-9]{%d}", escapedPrefix, resourceName, p.randomLength), true
    }

    // Standard pattern without resource name
    return fmt.Sprintf("%s[a-zA-Z0-9]{%d}", escapedPrefix, p.randomLength), true
}

And for responses:

func (p *PrefixProcessor) ProcessResponse(field string, value string, modifiedFields map[string]bool) (string, bool) {
    if !p.Match(value) {
        return value, false
    }

    // If this field was modified in the request and no fixed replacement
    if modifiedFields[field] && p.replaceWith == "" {
        return fmt.Sprintf("{{ Request.Body 'jsonpath' '$.%s' }}", field), true
    }

    if p.replaceWith != "" {
        return p.replaceWith, true
    }

    return value, false
}

This approach:

1. Changes the matcher for fields to use regex matcher
2. Uses Hoverfly's templating system to extract values from the incoming request
3. Uses those same values in the response, maintaining consistency

Following a similar principle, let's look at the UUID example:

func (p *UUIDProcessor) ProcessRequest(_ string) (string, bool) {
    if p.replaceWith != "" {
        return p.replaceWith, true
    }
    // Use proper regex pattern for UUIDs that will work with Hoverfly's RegexMatch
    return `[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}`, true
}

func (p *UUIDProcessor) ProcessResponse(field string, value string, modifiedFields map[string]bool) (string, bool) {
    if !p.Match(value) {
        return value, false
    }

    // If we have a fixed replacement, use it regardless of modified fields
    if p.replaceWith != "" {
        return p.replaceWith, true
    }

    // If this field was modified in the request and modifiedFields is provided
    if modifiedFields != nil && modifiedFields[field] {
        return fmt.Sprintf("{{ Request.Body 'jsonpath' '$.%s' }}", field), true
    }

    return value, false
}

All these processors are orchestrated:

// Process handles both the request and response processing for a pair
func (s *DefaultProcessingStrategy) Process(pair *v2.RequestMatcherResponsePairViewV5) error {
    // First try to process the request
    modifiedFields, err := s.ProcessRequest(pair)
    if err != nil {
        return err
    }

    // Then process the response
    if len(modifiedFields) > 0 {
        // If we modified fields in the request, process response with them
        if err = s.ProcessResponse(pair, modifiedFields); err != nil {
            return err
        }
        return nil
    }

    // If we didn't modify any request fields (e.g., no request body or no matches),
    // try to process the response independently
    if err = s.processResponseForRequestsWithoutBody(pair); err != nil {
        return err
    }

    return nil
}

Here we:

1. First process the request matchers, identifying dynamic fields
2. Tracks which fields were modified in the request
3. Process the response, applying templating where appropriate
4. Handles special cases like requests without bodies

Note: I understand that the code snippets above can be a bit complex to digest on first reading. Don't worry if you don't grasp all the implementation immediately - they're included primarily for those who want to dive deeper. If you prefer, you can focus on the overall concept rather than the specific code. The full source code is available on GitHub if you want to explore further.

To summarize the main idea: We modify the output JSON files, telling Hoverfly to use the request data in the response. You can, of course, modify this approach to make responses with static data, random data, or whatever else you need.

API Retries

Sometimes things don't work, at least not after the first attempt. Our application might need to handle temporary failures by retrying requests.
Let's look at a test server that simulates an unreliable API:

func (ts *TestServer) handleGetActivity(w http.ResponseWriter, r *http.Request) {
    attempt := ts.attempts.Add(1)

    // Extract the id from the URL path
    pathParts := strings.Split(r.URL.Path, "/")
    if len(pathParts) < 5 {
        http.Error(w, "Invalid URL", http.StatusBadRequest)
        return
    }
    idStr := pathParts[4]
    id, err := strconv.Atoi(idStr)
    if err != nil {
        http.Error(w, "Invalid ID", http.StatusBadRequest)
        return
    }

    // Simulate different responses based on attempt number
    switch attempt {
    case 1:
        // First attempt - Service Unavailable
        w.WriteHeader(http.StatusServiceUnavailable)
        json.NewEncoder(w).Encode(map[string]string{
            "error": "Service Temporarily Unavailable",
            "code":  "SERVER_BUSY",
        })
    case 2:
        // Second attempt - Gateway Timeout
        w.WriteHeader(http.StatusGatewayTimeout)
    case 3:
        // Third attempt - Success
        w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
        _ = json.NewEncoder(w).Encode(map[string]interface{}{
            "id":        int32(id),
            "title":     "Test Activity",
            "dueDate":   "2025-02-20T09:58:59.009Z",
            "completed": false,
        })
    default:
        // Any subsequent attempts - Bad Request
        w.WriteHeader(http.StatusTooManyRequests)
        _ = json.NewEncoder(w).Encode(map[string]string{
            "error": "Too many attempts",
        })
    }
}

This server:

1. Returns a 503 (Service Unavailable) on the first attempt
2. Returns a 504 (Gateway Timeout) on the second attempt
3. Succeeds with a 200 (OK) on the third attempt

By default, Hoverfly would only capture the final successful response. And of course our test will fail, but like we have a trick for that as well.

Enabling Stateful Capture

To capture this sequence of responses, we need to enable stateful capture mode in Hoverfly:

hoverctl mode capture --stateful

How Stateful Capture Works

When in stateful capture mode, Hoverfly adds metadata to the captured request/response pairs to track their sequence:

1. requiresState - Added to the request to indicate which state is required to match this entry
2. transitionsState - Added to the response to indicate how the state should change after this response

Let's look at how this appears in the simulation file:

Here's how the sequence works:

1. Hoverfly automatically creates a state variable called sequence:1 and initializes it to "1"
2. The first request matches when the state is "1"
3. After serving the first response, the state transitions to "2"
4. The second request matches when the state is "2"
5. And so on…
6. After serving the last response, there's no further transition defined

Testing with Stateful Simulations

When we run tests against this simulation, Hoverfly will play back the sequence of responses in order, perfectly simulating our retry scenario:

func TestActivitiesWithRetries(t *testing.T) {
    serverUrl, ok := os.LookupEnv("API_SERVER_URL")
    if !ok || serverUrl == "" {
        ts := tests.NewTestServer() // Start our test server
        defer ts.Close()
        serverUrl = ts.URL()
    }

    httpClient := tests.NewHttpClient(t)
    client, err := NewClientWithResponses(serverUrl, WithHTTPClient(httpClient))
    if err != nil {
        t.Fatalf("failed to inistialize client: %v", err)
    }
    defer func() {
        // Reset the server state
        _, _ = httpClient.Post(serverUrl+"/reset", "application/json", nil)
    }()

    scenarios := []testCase{
        {
            name: "create activity with retries",
            testFunc: func(t *testing.T, client *ClientWithResponses) {
                var (
                    ctx   = context.Background()
                    id    = int32(7474)
                    title = "Test Activity"
                )

                var lastErr error
                for attempt := 1; attempt <= 4; attempt++ {
                    resp, err := client.GetApiV1ActivitiesIdWithResponse(
                        ctx,
                        id,
                    )
                    if err != nil {
                        lastErr = err
                        continue
                    }

                    // Success!
                    if resp.StatusCode() == http.StatusOK {
                        require.NotNil(t, resp.ApplicationjsonV10200)
                        assert.Equal(t, id, *resp.ApplicationjsonV10200.Id)
                        assert.Equal(t, title, *resp.ApplicationjsonV10200.Title)
                        return
                    }

                    // Continue if it's a retryable status
                    if resp.StatusCode() == http.StatusServiceUnavailable ||
                        resp.StatusCode() == http.StatusGatewayTimeout {
                        continue
                    }

                    // Non-retryable error
                    t.Fatalf("Got non-retryable status: %d", resp.StatusCode())
                }

                // If we got here, all retries failed
                t.Fatalf("All retries failed. Last error: %v", lastErr)
            },
        },
    }

    for _, tt := range scenarios {
        t.Run(tt.name, func(t *testing.T) {
            tt.testFunc(t, client)
        })
    }
}

Benefits of Stateful Simulation

Using stateful capture may provide several benefits:

1. Test retry mechanisms properly - Verify that your client correctly handles temporary errors
2. Simulate workflows - Test multi-step processes that change state with each request
3. Reproduce edge cases - Capture and replay rare error conditions

By combining our pattern processors with stateful capture, we gain the ability to test both dynamic data and complex stateful interactions without changing our test code.

Static Response Replacement

Sometimes you want to provide a completely different response for specific endpoints. This is useful for:

1. Creating test-specific behavior - Returning predefined responses for certain test cases
2. Handling edge cases - Simulating error conditions or special scenarios
3. Maintaining consistent test data - Ensuring tests run with known values

Our processor supports this through the endpoints section in the configuration:

endpoints:
  - method: "GET"
    path: "/api/v1/Activities/30"
    status: 200
    static_response: |
      {
        "id": 77,
        "title": "John Doe",
        "dueDate": "2025-02-19T17:12:52.127Z",
        "completed": false
      }

This completely overrides any captured response for the specified endpoint, giving you precise control over the behavior in your tests.

Conclusion

We've explored several powerful techniques for API testing with simulations:

1. Understanding Hoverfly's matching system - Learning how request matchers work
2. Handling dynamic data - Making simulations work with UUIDs, random strings, and timestamps
3. Testing retry logic - Using stateful capture to simulate unreliable APIs
4. Static response replacement - Overriding specific endpoints for test scenarios

Of course, this approach may not be for everyone. If you require much more flexibility or you're starting a new project, then tools like WireMock might be a better choice. But there are specific use cases where this approach is particularly helpful and more effective than alternatives, making Hoverfly a useful tool to have in your testing arsenal.

In the final part of this series, we'll explore integrating these tools into CI pipelines and development workflows.

Source Code

The complete source code for this solution is available on GitHub.