Skip to content

Let's POWER Automate

From no-code to low-code

Menu
  • Expressions
  • Filters
  • General
  • Problems
  • Triggers
  • Application specific solutions
    • Excel
    • Forms
    • Planner
    • Outlook
    • SharePoint
    • Teams
  • Resources
Menu

How to delete duplicate items in a SharePoint list with Power Automate

Posted on January 9, 2022January 9, 2022 by Tom

“The SharePoint list is a mess, there’re so many duplicate items that it would take hours to delete them, can I build a Power Automate flow to do it for me?”


Power Automate flow doesn’t have to be always a repeatable flow, started on regular basis or on specific event. It can be a flow that you’ll start manually only once or twice, just to do a specific task. Some task that has clear rules, but is very time consuming. For example, deletion of duplicate items from a SharePoint list.

You could do it manually – create a view, sort the items, and go through all of them and delete duplicates. But the manual approach has a few downsides: it can take some time, and it’s easy to make a mistake. You can miss one duplicate, or delete a unique item without even noticing. That’s when it makes sense to delegate the work to a flow – to do it fast and without any mistakes.

Get all the items

If you want to search for duplicates and delete them, you must find them first. Get all the items using the ‘Get items’ action on the list. This is also the place where you define which of the duplicates you want to keep. Is it the first item, or the last item? Use the Order By field to keep the desired items at the top, e.g. to keep the latest items:

Modified desc

Find the duplicate items

Since you want to keep only the unique items, there must be a column that should stay unique. ‘Select’ only this column, e.g. the Title, and in a following ‘Compose’ keep only the unique values. It’s the same approach as used to send only one email per user.

union(body('Select'),body('Select'))
Power Automate delete duplicate SharePoint

The result of ‘Compose’ will be only the unique values. Go through the values one by one inside ‘Apply to each’, and for each of them find the duplicates using ‘Filter array’.

Power Automate delete duplicate SharePoint

You’re searching for all the items where the column that should be unique contains the currently processed unique value.

Remove only the duplicates

Here comes the tricky part. The ‘Filter array’ gives you all the items with the specific unique value, now you want to delete all of them except one – the one that should stay. And since you sorted the items already in the ‘Get items’, it’s the first item it’ll find.

Use the skip(…) expression to skip the first item and loop through the rest in another ‘Apply to each’.

skip(body('Filter_array'),1)

Then just delete all the remaining items using their ID.

item()?['ID']
Power Automate delete duplicate SharePoint

At the end of the flow you’ll have only the unique items in the list.

Summary

This article described a flow to delete duplicate SharePoint items using Power Automate, but there’s one more idea behind this post: flows are not limited to repeated tasks, even though that’s the main use case. You can build a simple flow also for one-time tasks, like removing duplicates or copying files, the flow doesn’t care.

If the task has clearly defined steps that you’d have to do over and over and over again, you might consider building a flow. But always consider the amount of work! You shouldn’t spend 1 hour building a flow to do a 30 minutes manual task.


Do you struggle with the various expressions, conditions, filters, or HTTP requests available in Power Automate?

I send one email per week with a summary of the new solutions, designed to help even non IT people to automate some of their repetitive tasks.

All subscribers have also access to resources like a SharePoint Filter Query cheat sheet or Date expressions cheat sheet.

Zero spam, unsubscribe anytime.

3 thoughts on “How to delete duplicate items in a SharePoint list with Power Automate”

  1. Leandro says:
    January 10, 2022 at 2:25 pm

    Great flow, Tom, thanks for sharing!

    Reply
  2. Matt Bishop says:
    January 13, 2022 at 4:48 pm

    Hi
    This was exactly what I was looking for, however, I can’t seem to get the filter to output anything. The only difference in my flow is that my Select box has a key/value entry, so I’ve put ‘Title’ as text for the key and the column ‘Title’ in the value.
    Can you help?
    Thanks

    Reply
    1. Tom says:
      January 16, 2022 at 10:50 am

      Hello Matt,
      use the button on the right side of the ‘Select’ action to hide the ‘Key’ column and keep only the one for the value, as used in the post.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

NOW AVAILABLE:

The Ultimate Power Automate expressions cheat sheet
Spend your time thinking about what the flow should do, not how to do it!

Do you struggle with the various expressions, conditions, filters, or HTTP requests available in Power Automate?

I send one email per week with a summary of the new solutions, designed to help even non IT people to automate some of their repetitive tasks.

All subscribers have also access to resources like a SharePoint Filter Query cheat sheet or Date expressions cheat sheet.

Zero spam, unsubscribe anytime.

Hello and welcome!

My name is Tom and I'm a business process automation consultant and Microsoft MVP living in the Czech Republic. I’ve been working with Microsoft technologies for almost 10 years, currently using mainly Power Automate, SharePoint, Teams, and the other M365 tools.

I believe that everyone can automate part of their work with the Power Automate platform. You can achieve a lot by "clicking" the flows in the designer, but you can achieve much more if you add a bit of coding knowledge. And that's what this blog is about.

To make the step from no-code Power Automate flows to low-code flows: using basic coding knowledge to build more complex yet more efficient flows to automate more of your daily tasks.

  • How to create a new SharePoint list column with Power Automate flowMay 22, 2022
  • How to combine expressions in your Power Automate flowsMay 18, 2022
  • Import Planner tasks with checklists into various buckets (Power Automate)May 15, 2022
  • How to get notified when Planner task was reassigned (Power Automate)May 11, 2022
  • How to extract value from XML using Power Automate flowMay 8, 2022

Power Automate blogs worth visiting

Damien Bird
Dennis (Expiscornovus)
Paul Murana

© 2022 Let's POWER Automate | Powered by Superbs Personal Blog theme