De-duplication

Did you know?

Duplicate records can lead to an overestimation of contacts on a database. A de-dupe should be run at least once every 30 days for organisations with a high number of contacts.

What is de-duplication?

It is the process of removing any duplicate records, but there are two main approaches:

  • Removal of the duplicate(s) record
  • Merging of the duplicate(s) records.

Where two records are exactly the same it makes complete sense to remove one or the other. However, some duplicates should be merged because one may contain address data, the other a telephone number.

We use the best software in the market, Cygnus, it has been around for nearly 20 years. It allows us to do all sorts of de-dupes, including:

  • At individual level – this ensures only 1 individual record is kept per address.
  • At family level – this ensures only 1 family member is kept per address – the surname is used.
  • At address level – this ensures only 1 record per address is kept.

So, which level to use? It all depends. For a bank, individual, for a charity donation campaign, family perhaps, it all depends.

Why do I need de-duplication?

Firstly, you need to know how many unique contacts you have on your database. Using this information you can forecast with better accuracy. Moreover, if the true number of contacts is way below what you expected you can take action to acquire more contacts. Too many organisations prefer to think they have x contacts, instead of actually knowing.

It is also good practice to remove or merge duplicates. Ideally the database will be checking for duplicates at the point of entry, preventing them from being added and informing the database administrator.

shutterstock_594035996