You’ve got a duplicate problem. Now what?

Working in data, you might imagine that I have strong feelings about this subject! So here it is.

Duplicates are the bed bugs of nonprofit databases everywhere.

They’re tough to spot when the problem is small. But as things get noticeably worse, you’ll look closely enough and start to see them everywhere – sucking the life out of everything good in your system!

To be serious though, duplicates are damaging to an org’s CRM strategy. Not only do they make it difficult to gather the right data, but they can cause staff to lose confidence in your system.

And when staff don’t have faith in your database, you either get more bad data going into it, or none at all.

We can’t have that! Here are 6 steps for wrangling those duplicates and getting your org back on track.

6 Steps for Getting A Handle on Those Duplicates

If ONLY our duplicates looked this cute.

1. Find the source.

First things first: figure out why your duplicates exist. Are they all echoes from a distant past, or are they still entering your system somehow?

Obviously, the type of system plays a role here. But generally speaking, duplicates exist for at least one of the below reasons:

  • Users are entering them manually
  • Users/admins are mass importing them accidentally
  • You have an automation that’s creating them…automatically

*Tip*: You need a strong understanding of 1) all the ways data enters your system, and 2) how your system does/doesn’t de-dupe. Consider setting up a call with your vendor to get clear on the tech.

2. Then, cut off your sources.

Next, start heading off new duplicates immediately. If it’s an automation giving you grief, pause it and get to fixing! If it’s your staff, sit down with your top offenders and politely remind them that they are driving you crazy must check the database before creating new records.

Then, before you start de-duping, make sure your interventions actually worked. Enter step 3.

3. Put systems in place to monitor your progress.

If you’re not familiar with the ins & outs of your system, this is when you give your rep or consultant a call. Make sure you’re leveraging any and all functionality to help you track new duplicates, to see if you still have a sourcing issue.

What does that look like exactly? Most likely it’s reports. If your org uses Salesforce though, you’re in luck – there are a few different options available here.

P.s. if you use a system that doesn‘t track duplicates, but charges for storage, you may want to consider a new platform. (This is my reality with Pardot, for anyone who’s familiar. -_-)

4. Turn your attention to those duplicates.

No matter how bad your duplicate problem is, it truly can’t get much worse (as long as you’ve done #’s 1-3). Remember that for this next part.

Before you dive into cleaning/merging mode, get a rough estimate of the dupes. You’ll want that end number in sight, since it will help you timeline this cleanup. You can do this using a duplicates report, or in Excel (by exporting your entire database and calculating using the Remove Duplicates feature).

5. Project-plan the process.

I don’t think there is an admin in existence who has time to tackle all that cleanup, all at once. So plan this out! Using that rough estimate from step 4, figure out how many duplicates you’d have to clean on a daily/weekly/monthly basis in order to finish the cleanup in a reasonable timeframe.

p.s. If you can delegate this to a direct report (or even an intern), go for it! While not the most fun task, it’s still a great way to give someone else more database experience.

6. Be vocal about this in your org.

Even if duplicates have caused staff to lose confidence in your database, it’s never too late to do some damage control and educating.

After all, preventing duplicates isn’t just on us admins. It’s the job of the entire organization: everyone and anyone even tangentially responsible for entering data into your system. So bring folks into the conversation, to assure them that you’re all headed in the right direction.

Share your thoughts!