How to Find Duplicates in an Array in PowerShell?

Do you need to find duplicate elements from an array in PowerShell? In this PowerShell tutorial, I will explain how to find duplicates in an array in PowerShell using various methods and examples.

To find duplicates in an array in PowerShell, you can use the Group-Object cmdlet and filter for groups with a count greater than one. For example, $duplicates = $array | Group-Object | Where-Object { $_.Count -gt 1 } | ForEach-Object { $_.Name } will return a list of duplicate values from the array $array

Find Duplicates in an Array in PowerShell

In PowerShell, you can create an array by assigning multiple values to a single variable, separated by commas.

$myArray = "apple", "banana", "cherry", "apple", "banana"

In the above example, $myArray is an array that contains five items, including some duplicates. Now, let us check various methods to find duplicate items from a PowerShell array using different methods.

Method 1: Using Group-Object

The Group-Object cmdlet is a straightforward way to group objects in PowerShell. When it comes to arrays, you can use it to group similar items and count them. To find duplicates, you can filter these groups to find any that have a count greater than one.

Here’s an example:

$myArray = "apple", "banana", "cherry", "apple", "banana"
$duplicates = $myArray | Group-Object | Where-Object { $_.Count -gt 1 }
$duplicates | Foreach-Object { $_.Name }

This script will output apple and banana, because these are the items that appear more than once in the array. You can see the output in the screenshot below:

Find Duplicates in an Array in PowerShell

Method 2: Using Select-Object

Another method involves using the Select-Object cmdlet with the -Unique switch, which filters out duplicate entries. This method is useful if you want to see the unique values of a PowerShell array. However, to find duplicates, you must compare the unique array with the original.

Here’s how you can do that:

$myArray = "apple", "banana", "cherry", "apple", "banana"
$unique = $myArray | Select-Object -Unique
$duplicates = $myArray | Where-Object { $unique -notcontains $_ }

In this script, $unique will contain all unique elements, and $duplicates will have the duplicates after comparing with the original array.

Method 3: Using Custom Comparison

You can also write custom script to find duplicates from a PowerShell array, you can write a script that iterates over the array and checks for duplicates by keeping track of items that have already been seen.

Here’s a custom script to find duplicates:

$myArray = "apple", "banana", "cherry", "apple", "banana"
$seen = @{}
$duplicates = @()

foreach ($item in $myArray) {
    if ($seen[$item]) {
        $duplicates += $item
    } else {
        $seen[$item] = $true
    }
}

$duplicates = $duplicates | Select-Object -Unique
$duplicates

This script uses a hashtable $seen to keep track of items that have been iterated over. If an item is already in the hashtable, it is added to the $duplicates array. Finally, we filter $duplicates to only unique values since the same duplicate could be added multiple times.

You can see the output in the screenshot below:

PowerShell Find Duplicates in an Array

Method 4: Using LINQ in PowerShell

For those who are a bit more advanced and are familiar with .NET, PowerShell can leverage LINQ (Language-Integrated Query) to find duplicates. This is a more complex method but can be very efficient for large datasets.

$myArray = "apple", "banana", "cherry", "apple", "banana"
$duplicates = [System.Linq.Enumerable]::ToLookup([object[]]$myArray, [Func[object, object]]{param($i) $i}) |
    Where-Object { $_.Count -gt 1 } |
    Foreach-Object { $_.Key }

This script converts the array into a lookup, where each item is a key, and filters out the keys with more than one occurrence, which indicates duplicates.

Conclusion

Finding duplicates in an array is easy and possible in various ways in PowerShell; you can use Group-Object, Select-Object, a custom comparison script, or even LINQ.

In this post, I have explained how to find duplicates in an array in PowerShell using various methods.

You may also like: