Dealing with Promises In an Array with async/await

Promises and async/await is a welcomed addition to the newer versions of JavaScript. If you are not using it yet and are trapped in the callback hell, you might want to check it out and start using it already. Believe me, it's awesome! The MDN docs would be a good place to start, and CSS-Tricks has a good article on it as well.

But it can be a little bit tricky when using async/await to deal with a collection of promises. Thankfully, here is my cheatsheet for dealing with them, created based on my experience.

p.s. No external libraries! 😉

Now, let's get started! Imagine we have the following asynchronous functions:

const resolveInTwoSeconds = () => {
  return new Promise((resolve) => {
    setTimeout(() => resolve(2), 2000);
  })
};

const resolveInThreeSeconds = () => {
  return new Promise((resolve) => {
    setTimeout(() => resolve(3), 3000);
  })
};

const resolveInFiveSeconds = () => {
  return new Promise((resolve) => {
    setTimeout(() => resolve(5), 5000);
  })
};

1. Wait for all promises to complete with Promise.all

Promise.all accepts an array of promises and returns a new promise that resolves only when all of the promises in the array have been resolved. The promise resolves to an array of all the values that the each of the promise returns.

(async function() {
  const asyncFunctions = [
    resolveInTwoSeconds(),
    resolveInThreeSeconds(),
    resolveInFiveSeconds()
  ];
  const results = await Promise.all(asyncFunctions);
  // outputs `[2, 3, 5]` after five seconds
  console.log(results);
})();

2. Wait for at least one promise to complete with Promise.race

Promise.race accepts an array of promises and returns a new promise that resolves immediately when one of the promises in the array have been resolved, with the value from that promise.

(async function() {
  const asyncFunctions = [
    resolveInTwoSeconds(),
    resolveInThreeSeconds(),
    resolveInFiveSeconds()
  ];
  const result = await Promise.race(asyncFunctions);
  // outputs `2` after two seconds
  console.log(result);
})();

3. Wait for all promises to complete one-by-one

The easiest, most straightforward way to achieve this is by using plain old for loops. It is readable and easy to reason about.

(async function() {
  const asyncFunctions = [resolveInTwoSeconds, resolveInThreeSeconds, resolveInFiveSeconds];
  // outputs 2 after 2 seconds
  // outputs 3 after 5 seconds
  // outputs 5 after 8 seconds
  for (const asyncFn of asyncFunctions) {
    const result = await asyncFn();
    console.log(result);
  }
})();

Update: The approach below was the original one on the post, but after years of new experience, I've come to realize that this unnecessarily complicates things, and I was just hacking reduce to behave like a for loop. Advice: Just use for loops for this. Still keeping it here if you're interested

There's no native methods on Promise class that can do this quickly, but we can make use of Array.prototype.reduce method to achieve the goal.

(async function() {
  const asyncFunctions = [resolveInTwoSeconds, resolveInThreeSeconds, resolveInFiveSeconds];
  // outputs 2 after 2 seconds
  // outputs 3 after 5 seconds
  // outputs 5 after 8 seconds
  await asyncFunctions.reduce(async (previousPromise, nextAsyncFunction) => {
    await previousPromise;
    const result = await nextAsyncFunction();
    console.log(result);
  }, Promise.resolve());
})();

This is less straight-forward than the previous implementations, but I am going to write a separate post to explain this. Let's keep this post just for quick cheatsheets 😉.

4. Run async functions batch-by-batch, with each batch of functions executed in parallel

This is really helpful if you want to avoid hitting the rate limit of some API service. This makes use of the same concept in #3, where we have an array of promises resolved sequentially, combined with a two-dimensional array of promises and the use of Promise.all.

The key here is to build the collection of async functions in a two-dimensional array first. Once we have that, we can iterate over each collection of async functions and execute them in parallel, and use Promise.all to wait for each of those functions to complete. Until all of the promises in the current batch resolve, we are not going to process the next batch.

(async function() {
  const asyncFunctionsInBatches = [
    [resolveInTwoSeconds, resolveInTwoSeconds],
    [resolveInThreeSeconds, resolveInThreeSeconds],
    [resolveInFiveSeconds, resolveInFiveSeconds],
  ];

  // Outputs [2, 2] after two seconds
  // Outputs [3, 3] after five seconds
  // Outputs [5, 5] after eight seconds
  for (const currentBatch of asyncFunctionsInBatches) {
    const currentBatchPromises = currentBatch.map(asyncFn => asyncFn())
    const batchResults = await Promise.all(currentBatchPromises)
    console.log(batchResults)
  }
})();

Update: Again, the approach below was the original one on the post, but after years of new experience, I've come to realize that this unnecessarily complicates things, and I was just hacking reduce to behave like a for loop. Advice: Just use for loops for this. Still keeping it here if you're interested

Here's the full implementation of the above concept:

(async function() {
  const asyncFunctionsInBatches = [
    [resolveInTwoSeconds, resolveInTwoSeconds],
    [resolveInThreeSeconds, resolveInThreeSeconds],
    [resolveInFiveSeconds, resolveInFiveSeconds],
  ];

  // Outputs [2, 2] after two seconds
  // Outputs [3, 3] after five seconds
  // Outputs [5, 5] after eight seconds
  await asyncFunctionsInBatches.reduce(async (previousBatch, currentBatch, index) => {
    await previousBatch;
    console.log(`Processing batch ${index}...`);
    const currentBatchPromises = currentBatch.map(asyncFunction => asyncFunction())
    const result = await Promise.all(currentBatchPromises);
    console.log(result);
  }, Promise.resolve());
})();

Keep in mind that I'm building the batches of async functions through hard-coding here. In a real application, you might have a dynamic length of array returned from an API call or the likes, so you will have to split them yourselves. A quick implementation for this task:

const splitInBatch = (arr, batchSize) => {
  return arr.reduce((accumulator, element, index) => {
    const batchIndex = Math.floor(index / batchSize);
    if (Array.isArray(accumulator[batchIndex])) {
      accumulator[batchIndex].push(element);
    } else {
      accumulator.push([element]);
    }
    return accumulator;
  }, []);
}

// outputs [[1, 2, 3], [4, 5, 6]]
console.log(splitInBatch([1, 2, 3, 4, 5, 6], 3));

Or, you can also opt for libraries such as lodash to help you with this task.

import chunk from 'lodash.chunk';

// outputs [[1, 2, 3], [4, 5, 6]]
console.log(chunk([1, 2, 3, 4, 5, 6], 3));

5. Bonus Tip: Do not pass an async function to forEach

Remember, the difference between Array.prototype.map and Array.prototype.forEach is that the latter does not return the result of each iteration. If we pass async functions to forEach, we have no way of retrieving the returned promise to do anything useful with it. Unless you want to fire the async function and forget about it, passing async functions to forEach is never something you want to do.

Conclusion

There you go! That is all 5 cheatsheets on what to do and not to do with an array of Promises. I hope this has been useful to you all 😁, and please, please, let me know in the comments section if there's anything I ought to improve upon.

See you again!

Comments

[Gleb]

You forget about for await to run async functions one-by-one

async function start() {
  for await (let asyncFunction of asyncFunctions ) {
    console.log(await asyncFunction())
  }
}
start();

[iranrodrigues]

An added bonus is that you can easily break or return based on the result of each function, if you had to.

[Afif Sohaili]

Updated!

[JZ]

Hi Afif, thanks for the post, it was helpful. With help from a user on Reddit, there's another entry you might want to add, "async pools". If it's useful, please copy anything you want from this public gist: gist.github.com/jzohrab/a6701d0087...

Cheers and regards! jz

[Afif Sohaili]

Thanks! That's interesting. Definitely faster than just processing promises one-by-one.

[Cortado-J]

Thanks for great post @afif Sohaili.

I'm new to javascript but not new to programming.
I was interested in the "Complete async tasks in sequence" method using reduce as you describe.
In the spirit of "DRY", I immediately wanted to encapsulate it:

    function sequencer(array, waitForThis, andThen){
      return (async function() {
        await array.reduce(async (previousPromise, item) => {
          await previousPromise;
          const result = await waitForThis(item);
          andThen(result)
        }, Promise.resolve());
      })()
    }

which then gives us the re-usable "sequencer" which takes as parameters:

• An array of values which are passed to each task in the sequence.
• A function returning a promise to be applied to each of the above.
• A function to be executed for each result.

which means we can do this:

    const resolveInNSeconds = (n) => {
      return new Promise((resolve) => {
        setTimeout(() => resolve(n), n * 1000);
      })
    };

    const logResult = (result) => {
      console.log(result);
    }

    sequencer([2,3,5], resolveInNSeconds, logResult)

Does this look like sound javascript?

I can't help thinking the resolveInNSeconds function could be a bit simpler?

[Afif Sohaili]

Sorry I hadn't come and check this often enough and missed it. Yes, you can abstract those however you'd like to make it more readable, and functions are something you can toss around in JavaScript.

[Mark Voorberg]

Thanks for this. When you write the "explainer" article for number 3, please contrast it with an old-school for loop. I'm interested to see if your opinion changes!

[Afif Sohaili]

Hi, I just googled for it, and I was mindblown. Definitely will opt for that instead. I didn't know. 😁

This is why I love the comments section!

Edit: I just reread your comment, and 🤔 do you mean the new for await of that came with ES2018?

[Mark Voorberg]

I did not mean for-await-of, I mean:

  for (var i = 0; i < asyncFunctions.length; i++) {
    var result = await asyncFunctions[i]();
    console.log(result);
  }

Less code, less cognitive overhead!

jsfiddle.net/4umfreak/bkqcymuL/

[Afif Sohaili]

Yeah, that's definitely a more readable approach. I have to be honest I haven't used for loops in a while so it didn't really occur to me.

[Afif Sohaili]

@4umfreak updated the code to use plain ol' for-loops. Took me long enough.

[kor3k]

there is a nice lib for promises

bluebirdjs.com/docs/api-reference....

[Afif Sohaili]

Yep, definitely a great library for Promises. I just felt like most of the time what I need with Promises are already available in ES6+.

[Hector]

Awesome!! This has helped me a lot refactoring an endpoint in an API that acts as payments cart and was originally implemented using Promise.all. Thank you v.m