FetchApp: UrlFetchApp with Retries

Google Apps Script is often used to pull data from various services via HTTP requests. However, these requests sometimes fail due to network or service issues. The default behavior of UrlFetchApp is to throw an exception, which you have to catch. Otherwise, the script execution will be interrupted.

We often need more: send the request again instead of failing. There is no built-in way to do retries in Apps Script.

Solution

To solve this problem and not copy-and-paste code snippets from project to project, I created FetchApp (GitHub) -- an open-source Google Apps Script library.

Its main features are:

1. Optional Retries: Depending on the response code received.
2. Delay Strategies: Choose between linear or exponential delay between retries.
3. Custom Callbacks: Implement callbacks on failed attempts for tailored actions and logic.
4. Enhanced Type Hints: Improved hints for UrlFetchApp's params argument.
5. Automatic Logging: Logs failed attempts automatically.

I use this library in many projects in production, especially where I have to communicate with unreliable third-party APIs, or there is a chain of related requests that I do not want to repeat if one of them randomly fails.

If you find this library useful, please give the repository a star and share the link with others.

Features, Use Cases, Examples

There are multiple ways and use cases to use FetchApp.

Drop-in Replacement for UrlFetchApp

FetchApp is designed to work as a drop-in replacement for UrlFetchApp. Caveat: FetchApp sets muteHttpExceptions: true in params unless explicitly specified otherwise.

// `url` and `params` are defined elsewhere

// regular UrlFetchApp
const response1 = UrlFetchApp.fetch(url, params);

// FetchApp without configuration is a pass-through to UrlFetchApp
const response2 = FetchApp.fetch(url, params);

// FetchApp with retries and delay enabled
const config = {
  maxRetries: 5,
  successCodes: [200],
  delay: 500,
};
const response3 = FetchApp.fetch(url, params, config);

// If there are no `params`, pass an empty object
const response4 = FetchApp.fetch(url, {}, config);

Configurable Client

If you need to use FetchApp multiple times, you can initiate a client to reuse the configuration:

// FetchApp with retries and delay enabled
const config = {
  maxRetries: 5,
  retryCodes: [500, 502, 503, 504],
  delay: 500,
};

const client = FetchApp.getClient(config);

// All client's fetch calls will use this config
const response1 = client.fetch(url, params);

// Partially modify the config for a specific request
const response2 = client.fetch(url, params, { successCodes: [200] });

Success or Retry Response Codes

FetchApp retries requests depending on the response code received in two different modes:

• successCodes: deem responses with these codes successful and return the response. If provided, retryCodes are ignored.
• retryCodes: requests with these codes are not successful; retry.

Examples:

const response1 = FetchApp.fetch(
  url,
  params,
  {
    successCodes: [200],  // Everything else leads to retries
    maxRetries: 3,
  },
)

const response2 = FetchApp.fetch(
  url,
  params,
  {
    retryCodes: [500, 502, 503],  // Everything else is deemed successful
    maxRetries: 3,
  },
)

const response3 = FetchApp.fetch(
  url,
  params,
  {
    successCodes: [200],  // Takes priority over retryCodes
    retryCodes: [500, 502, 503],  // Ignored
    maxRetries: 3,
  },
)

Delay Between Requests

FetchApp supports constant or exponential delay between retries:

const response1 = FetchApp.fetch(
  url,
  params,
  {
    successCodes: [200],  // Everything else leads to retries
    maxRetries: 3,
    delay: 300,  // Constant delay of 300ms after each request
  },
)

const response2 = FetchApp.fetch(
  url,
  params,
  {
    successCodes: [200],  // Everything else is deemed successful
    maxRetries: 3,
    // Exponential delay of 1, 2, 4, 8, etc. seconds
    delay: 1000,
    delayFactor: 2,
  },
)

const response2 = FetchApp.fetch(
  url,
  params,
  {
    successCodes: [200],  // Everything else is deemed successful
    maxRetries: 10,
    // Exponential delay of 1, 2, 4, 8, 10 seconds.
    delay: 1000,
    delayFactor: 2,
    maxDelay: 10000,  // Limit delay to maximum 10 seconds
  },
)

Throw Exceptions via Callbacks

If you need to throw an exception if all attempts fail, you can do it via a onAllRequestsFailure callback:

// Throw an exception
const throwException = ({ retries, url }) => {
  throw new Error(`All ${retries + 1} requests to ${url} failed`);
};

const config = {
  successCodes: [200],
  maxRetries: 5,
  delay: 500,
  onAllRequestsFailure: throwException,
};

const response = FetchApp.fetch(url, params, config);

Send Notifications via Callbacks

One of the difficulties with Apps Script is that it functions as a black box unless you add logging and notifications to the script. With the onRequestFailure and onAllRequestsFailure callbacks, you can send notifications about failed requests.

// Define the `sendNotification` function elsewhere

// Send notification if access is denied, maybe because the credentials expired
const accessDenied = ({ url, response }) => {
  const responseCode = response.getResponseCode();
  if ([401, 403].includes(responseCode)) {
    sendNotification(`Received ${responseCode} when accessing ${url}`);
  }
};

// Send a notification if all attempts failed
const allAttemptsFailed = ({ retries, url }) => {
  throw new Error(`All ${retries + 1} requests to ${url} failed`);
};

const config = {
  successCodes: [200],
  maxRetries: 5,
  delay: 500,
  onRequestFailure: accessDenied,
  onAllRequestsFailure: allAttemptsFailed,
};

const response = FetchApp.fetch(url, params, config);

Autocomplete and Type Hints

FetchApp comes with JSDoc declarations that enable type hints and autocomplete in Google Apps Script IDE, both for UrlFetchApp's params and for FetchApp's config:

Unfortunately, due to the IDE limitations, rich autocomplete doesn't work when you attach FetchApp as a library (as opposed to copying the code). Nevertheless, you still have the description of possible options in the type hints:

---

Contributions are welcome. Feel free to submit pull requests or issues on GitHub.