1 line of code: How to remove all duplicates from an Array

const removeDuplicates = arr => [...new Set(arr)]; 

Assumes that the given argument is an Array and removes duplicate entries, keep in mind that it works only for entries with primitive values (string, number, bigint, boolean, undefined, symbol, and null). Preserves the order of the entries and returns a copy of the array.

---

The repository & npm package

You can find the all the utility functions from this series at github.com/martinkr/onelinecode
The library is also published to npm as @onelinecode for your convenience.

The code and the npm package will be updated every time I publish a new article.

---

Follow me on Twitter: @martinkr and consider to buy me a coffee

Photo by zoo_monkey on Unsplash

---

Comments

[Michael Currin]

Worth explaining this separately when used with array instead of sets

const x = ["abc", "def"]
const y = [...x]
// ["abc", "def"]

And modifying x and y won't affect the other.

Then bringing in set

const x = ["abc", "def", "abc"]
const y = new Set(x)
// Set(["abc", "def"])
const z = [...y]
// ["abc", "def"]

And the the original post does that in one line in a function

[Michael Currin]

Also worth adding that you might want to stop once you have a set and not convert it back to an array.

Why?

Lookup time from a set is constant to check if a value is in a set while in an array of 1000 elements worst case you have to go through 1000 elements until you find the one at the end.

Secondly when you convert from array to set, the order will be lost. Sets are inherently unordered. So when you convert from set to array you will not get the order of the original array. So you might as well keep the set.

[Mihail Malo]

It's actually specced to preserve insertion order: developer.mozilla.org/en-US/docs/W...

(...spread operator calls y[Symbol.iterator], which is specced to be the same as what is returned by .values() on Set)

[Michael Currin]

Oh interesting, in Python it is unordered.

[Martin Krause]

Thank you for the explanation.

[Erik Waters]

I think they mean it won't behave as expected if the array contains similar objects that have different references

const originalObject = {}
const boundToOriginalObject = originalObject
const newObjectThatLooksLikeOriginal = {}
const arr = [originalObject, boundToOriginalObject, newObjectThatLooksLikeOriginal]
const dedupe = [ ...new Set(arr)]
dedupe // [{}, {}]
// deduping removes the duplicate reference but not similar objects
const mutation = {a: 1}
Object.assign(dedupe[1], mutation)
originalObject // {}
newObjectThatLooksLikeOriginal // { a: 1 }
// this proves which references remained in the deduped array prior to mutation

It might be helpful to clarify that dedupe will remove duplicate bindings but not similar objects with a different binding

[yossarian]

I think it worth saying that it works only on primitives

[Martin Krause]

Thank you for your contribution. I'll add it to the description.

[Michael Currin]

I just learned of an alternative for set to array

const myArr = Array.from(mySet1)

And if you want to iterate over a set without converting to an array.

for (const item of mySet1) { 
  console.log(item)
}

developer.mozilla.org/en-US/docs/W...

[Mihail Malo]

Array.from goes through iterator protocol for Set and Map, so it's exactly equivalent to the [... sugar.