Idempotency in API Design

#api #webdev #architecture #productivity

The API is what gives a web developer his power. It's a technology used by all living things. It surrounds us and penetrates us. It binds the web together.

The modern web is powered by APIs, and the ability to not only code an API, but properly architect one is an absolutely crucial skill.
A well-designed RESTful API should be several things:

Well-documented
Designed around the resource for which it is providing
Using a stateless request model – this means that you shouldn’t be storing any data between requests. Each should be its own operation.

But above all, reliability and safety are key. An unreliable API is a useless API. Very often the measure of a good API is not the technology itself, but rather what people are able to build with it. It's very important you are not dissuading potential users with an unreliable API.

So what do reliability and safety look like at the architectural level?

Idempotency and Safety

RESTful APIs serve up information based on HTTP requests made by clients. These HTTP requests have an attribute referred to as “safety,” and understanding safety is the first step.

SAFE HTTP REQUESTS:

GET - YES
PUT - NO
POST - NO
DELETE - NO
PATCH – NO

An HTTP request type is considered “safe” if it doesn’t mutate the data when it is invoked. Naturally, this would make GET requests a “safe” request type, therefore, they are considered idempotent from the get-go.

But that doesn’t answer the question of idempotence. Idempotence is the principle that an API method, once invoked a single time, will not modify the server state further upon any subsequent identical requests.

That’s a lot. What does that mean? Imagine it as a mathematical function.

f(x) = x^2 + 2

If we say x = 5, then f(x) will always be equal to 27. If we do it again, f(x) is still equal to 27, because the input data hasn’t changed. Similarly, if your user sends a request to the server that deletes a row from a database, and sends that same request again, they should get back the same “row successfully deleted” result each time, but the state will have only changed once.

Implementation

From the client end, a POST request to the API may look something like this:

import axios from ‘axios’;

static callAction(url, parameters = {}) {
    return new Promise((resolve, reject) => {
        return axios.post(API_URL + url, parameters).then(x => {
            resolve(x.data.data);
        });
    });
}

This is fine, it’ll return the value, but what happens if there is a network error? It would be nice to have auto-retry.

This is where idempotency comes in. Network errors run both ways. In the same way that the client’s packets may be disrupted on path to the server, the response may be disrupted on its way back to the client, triggering an auto-retry. Therefore, your API methods must be prepared to handle this eventuality.

Luckily, implementing idempotency is very easy. Simply attach our good friend the GUID, or Globally Unique Identifier (also called a UUID or Universally Unique Identifier but I like to say “gwid”). This 128-bit number can be generated by the client, and attached as a header value to the request.

Upon authentication by the API, handling is cake. Simply save the GUID and server response for a defined period of time (make sure this data auto-removes itself, or you’ll suffer from some pretty severe bloat).

A simple relational example:

Once that is done, all there is to do is place our callAction function within a loop:

async function callPost() {
    let remainingAttempts = 3;
    let exceptions = []
    try {
        while(remainingAttempts > 0) {
            await callAction(url, parameters);
            remainingAttempts--;
        }
    } catch(e) {
        exceptions.add(e);
    } finally {
        // this is where you handle the exception, whether it’s logging
        // to the console, etc
    }
}

Thanks for reading!

Got questions? Want to talk about The Mandalorian? Follow me on Twitter!

Top comments (4)

Mike Amundsen • Dec 24 '19 • Edited

happy to see your post. wanted to follow up on some HTTP-foo that I think makes supporting imdempotence writes easier for APIs.

HTTP has built-in features to make supporting idempotence easy (esp. handy w/ APIs.

services SHOULD:
1) support PUT /collection/{guid}, not POST /collection/ to create resources
2) require If-None-Match: * header on create (PUT) to prevent key collisions and return 412 Precondition Failed if the PUT fails
3) return ETag: {etag-hash} header on GET to ensure content idempotence
4) require If-Match: {etag-hash} header on update (PUT) to prevent content collisions and return 412 Precondition Failed if the PUT fails

clients SHOULD:
1) generate their own resource keys (you mention this)
2) send If-None-Match: * on create (PUT) & accept 412 Precondition Failed in response
3) send If-None-Match: {guid-hash} on GET & accept 204 No Content in response
4) send If-Match {guid-hash} headers on update (PUT) & accept 412 Precondition Failed in response

cheers