How to delete duplicate items in a SharePoint list with Power Automate

“The SharePoint list is a mess, there’re so many duplicate items that it would take hours to delete them, can I build a Power Automate flow to do it for me?”

---

Power Automate flow doesn’t have to be always a repeatable flow, started on regular basis or on specific event. It can be a flow that you’ll start manually only once or twice, just to do a specific task. Some task that has clear rules, but is very time consuming. For example, deletion of duplicate items from a SharePoint list.

You could do it manually – create a view, sort the items, and go through all of them and delete duplicates. But the manual approach has a few downsides: it can take some time, and it’s easy to make a mistake. You can miss one duplicate, or delete a unique item without even noticing. That’s when it makes sense to delegate the work to a flow – to do it fast and without any mistakes.

Get all the items

If you want to search for duplicates and delete them, you must find them first. Get all the items using the ‘Get items’ action on the list. This is also the place where you define which of the duplicates you want to keep. Is it the first item, or the last item? Use the Order By field to keep the desired items at the top, e.g. to keep the latest items:

Modified desc

Find the duplicate items

Since you want to keep only the unique items, there must be a column that should stay unique. ‘Select’ only this column, e.g. the Title, and in a following ‘Compose’ keep only the unique values. It’s the same approach as used to send only one email per user.

union(body('Select'),body('Select'))

The result of ‘Compose’ will be only the unique values. Go through the values one by one inside ‘Apply to each’, and for each of them find the duplicates using ‘Filter array’.

You’re searching for all the items where the column that should be unique contains the currently processed unique value.

Remove only the duplicates

Here comes the tricky part. The ‘Filter array’ gives you all the items with the specific unique value, now you want to delete all of them except one – the one that should stay. And since you sorted the items already in the ‘Get items’, it’s the first item it’ll find.

Use the skip(…) expression to skip the first item and loop through the rest in another ‘Apply to each’.

skip(body('Filter_array'),1)

Then just delete all the remaining items using their ID.

item()?['ID']

At the end of the flow you’ll have only the unique items in the list.

Summary

This article described a flow to delete duplicate SharePoint items using Power Automate, but there’s one more idea behind this post: flows are not limited to repeated tasks, even though that’s the main use case. You can build a simple flow also for one-time tasks, like removing duplicates or copying files, the flow doesn’t care.

If the task has clearly defined steps that you’d have to do over and over and over again, you might consider building a flow. But always consider the amount of work! You shouldn’t spend 1 hour building a flow to do a 30 minutes manual task.

---

Do you struggle with the various expressions, conditions, filters, or HTTP requests available in Power Automate?

I send one email per week with a summary of the new solutions, designed to help even non IT people to automate some of their repetitive tasks.

All subscribers have also access to resources like a SharePoint Filter Query cheat sheet or Date expressions cheat sheet.

First Name

Email address

Zero spam, unsubscribe anytime.

Leave this field empty if you’re human: