1. Using Set (The Modern Way)
The Set object lets you store unique values of any type. It's the simplest and often most performant way to remove duplicates.
Removing Duplicates:
const numbers = [1, 2, 2, 3, 4, 4, 5, 1];
const uniqueNumbers = [...new Set(numbers)];
console.log(uniqueNumbers); // Output: [1, 2, 3, 4, 5]
Counting Duplicates:
To count the total number of duplicate occurrences removed, simply compare the original array's length with the unique array's length.
const originalLength = numbers.length; // 8
const uniqueLength = uniqueNumbers.length; // 5
const totalDuplicatesRemoved = originalLength - uniqueLength;
console.log(`Total duplicates removed: ${totalDuplicatesRemoved}`);
// Output: Total duplicates removed: 3 (representing the extra 2, 4, and 1)
To get a count of each element's frequency (which highlights duplicates):
const counts = {};
numbers.forEach(num => {
counts[num] = (counts[num] || 0) + 1;
});
console.log(counts);
// Output: { '1': 2, '2': 2, '3': 1, '4': 2, '5': 1 }
// Elements with count > 1 are duplicates.
2. Using filter() and indexOf()
This method leverages the filter() array method in combination with indexOf(). indexOf() returns the first index at which a given element can be found in the array. If an element's first occurrence is at its current index during iteration, it's considered unique.
Removing Duplicates:
const numbers = [1, 2, 2, 3, 4, 4, 5, 1];
const uniqueNumbersFilter = numbers.filter((item, index, self) => {
return self.indexOf(item) === index;
});
console.log(uniqueNumbersFilter); // Output: [1, 2, 3, 4, 5]
Counting Duplicates:
Similar to the Set approach, the total count of removed duplicates can be found by length difference. For individual frequencies, the object counting method is still effective.
const originalLengthFilter = numbers.length;
const uniqueLengthFilter = uniqueNumbersFilter.length;
const totalDuplicatesRemovedFilter = originalLengthFilter - uniqueLengthFilter;
console.log(`Total duplicates removed: ${totalDuplicatesRemovedFilter}`);
// Output: Total duplicates removed: 3
// Frequency counting remains the same:
const countsFilter = {};
numbers.forEach(num => {
countsFilter[num] = (countsFilter[num] || 0) + 1;
});
console.log(countsFilter);
// Output: { '1': 2, '2': 2, '3': 1, '4': 2, '5': 1 }
3. Using reduce()
The reduce() method can iterate over an array and build up a single result, which can be a new array of unique elements.
Removing Duplicates:
const numbers = [1, 2, 2, 3, 4, 4, 5, 1];
const uniqueNumbersReduce = numbers.reduce((accumulator, current) => {
if (!accumulator.includes(current)) { // Checking if element already exists in accumulator
accumulator.push(current);
}
return accumulator;
}, []);
console.log(uniqueNumbersReduce); // Output: [1, 2, 3, 4, 5]
Counting Duplicates:
The counting logic remains consistent across approaches.
const originalLengthReduce = numbers.length;
const uniqueLengthReduce = uniqueNumbersReduce.length;
const totalDuplicatesRemovedReduce = originalLengthReduce - uniqueLengthReduce;
console.log(`Total duplicates removed: ${totalDuplicatesRemovedReduce}`);
// Output: Total duplicates removed: 3
// Frequency counting:
const countsReduce = {};
numbers.forEach(num => {
countsReduce[num] = (countsReduce[num] || 0) + 1;
});
console.log(countsReduce);
// Output: { '1': 2, '2': 2, '3': 1, '4': 2, '5': 1 }
4. Using a for loop and an Object/Map (Manual Tracking)
This traditional approach involves iterating through the array and using an auxiliary object (or Map) to keep track of elements already encountered. This method is often preferred for performance in very large arrays or when direct frequency counting is needed.
Removing Duplicates:
const numbers = [1, 2, 2, 3, 4, 4, 5, 1];
const uniqueNumbersLoop = [];
const seen = {}; // Can also use new Map() for better key flexibility
for (let i = 0; i < numbers.length; i++) {
const item = numbers[i];
if (!seen[item]) { // Check if item is not in 'seen' object
uniqueNumbersLoop.push(item);
seen[item] = true; // Mark item as seen
}
}
console.log(uniqueNumbersLoop); // Output: [1, 2, 3, 4, 5]
Counting Duplicates:
This approach is particularly efficient if you want to count frequencies directly while removing duplicates in a single pass.
const numbers = [1, 2, 2, 3, 4, 4, 5, 1];
const uniqueNumbersLoopCount = [];
const countsLoop = {}; // This will store frequencies
for (let i = 0; i < numbers.length; i++) {
const item = numbers[i];
countsLoop[item] = (countsLoop[item] || 0) + 1; // Increment count for current item
// If it's the first time seeing this item, add it to the unique array
if (countsLoop[item] === 1) {
uniqueNumbersLoopCount.push(item);
}
}
console.log(uniqueNumbersLoopCount); // Output: [1, 2, 3, 4, 5]
console.log(countsLoop); // Output: { '1': 2, '2': 2, '3': 1, '4': 2, '5': 1 }
const totalDuplicatesRemovedLoop = numbers.length - uniqueNumbersLoopCount.length;
console.log(`Total duplicates removed: ${totalDuplicatesRemovedLoop}`);
// Output: Total duplicates removed: 3
Conclusion
JavaScript provides flexible ways to handle array duplicates. For most modern applications, the Set object offers the cleanest and most concise solution for simply removing duplicates. However, when it comes to counting occurrences of each element, iterating and using an object/Map to store frequencies is a robust approach, often integrated directly or as a secondary step after obtaining unique elements. Choose the method that best fits your specific needs, performance considerations, and coding style!
