“The SharePoint list is a mess, there’re so many duplicate items that it would take hours to delete them, can I build a Power Automate flow to do it for me?”
Power Automate flow doesn’t have to be always a repeatable flow, started on regular basis or on specific event. It can be a flow that you’ll start manually only once or twice, just to do a specific task. Some task that has clear rules, but is very time consuming. For example, deletion of duplicate items from a SharePoint list.
You could do it manually – create a view, sort the items, and go through all of them and delete duplicates. But the manual approach has a few downsides: it can take some time, and it’s easy to make a mistake. You can miss one duplicate, or delete a unique item without even noticing. That’s when it makes sense to delegate the work to a flow – to do it fast and without any mistakes.
Get all the items
If you want to search for duplicates and delete them, you must find them first. Get all the items using the ‘Get items’ action on the list. This is also the place where you define which of the duplicates you want to keep. Is it the first item, or the last item? Use the Order By field to keep the desired items at the top, e.g. to keep the latest items:
Modified desc

Find the duplicate items
Since you want to keep only the unique items, there must be a column that should stay unique. ‘Select’ only this column, e.g. the Title, and in a following ‘Compose’ keep only the unique values. It’s the same approach as used to send only one email per user.
union(body('Select'),body('Select'))

The result of ‘Compose’ will be only the unique values. Go through the values one by one inside ‘Apply to each’, and for each of them find the duplicates using ‘Filter array’.

You’re searching for all the items where the column that should be unique contains the currently processed unique value.
Remove only the duplicates
Here comes the tricky part. The ‘Filter array’ gives you all the items with the specific unique value, now you want to delete all of them except one – the one that should stay. And since you sorted the items already in the ‘Get items’, it’s the first item it’ll find.
Use the skip(…) expression to skip the first item and loop through the rest in another ‘Apply to each’.
skip(body('Filter_array'),1)
Then just delete all the remaining items using their ID.
item()?['ID']

At the end of the flow you’ll have only the unique items in the list.
Summary
This article described a flow to delete duplicate SharePoint items using Power Automate, but there’s one more idea behind this post: flows are not limited to repeated tasks, even though that’s the main use case. You can build a simple flow also for one-time tasks, like removing duplicates or copying files, the flow doesn’t care.
If the task has clearly defined steps that you’d have to do over and over and over again, you might consider building a flow. But always consider the amount of work! You shouldn’t spend 1 hour building a flow to do a 30 minutes manual task.
Great flow, Tom, thanks for sharing!
Hi
This was exactly what I was looking for, however, I can’t seem to get the filter to output anything. The only difference in my flow is that my Select box has a key/value entry, so I’ve put ‘Title’ as text for the key and the column ‘Title’ in the value.
Can you help?
Thanks
Hello Matt,
use the button on the right side of the ‘Select’ action to hide the ‘Key’ column and keep only the one for the value, as used in the post.