IBRAVRA Media Network
Marketing • 12 min read

CRM Data Hygiene: Duplicate Detection, Merge Rules & the Cost of Dirty Records

Duplicates are a tax on every number downstream — inflated contact counts, split histories, and reports nobody trusts. This playbook detects, merges, and prevents duplicates from Salesforce and HubSpot's own documented limits.

By Enow A. Jovial • Published 2026-11-19

Every duplicate record in a CRM is a tax on every number downstream. A contact split across two records means a rep calls a prospect marketing already emailed, a report counts one customer as two, and a lead score is computed on half the history. The tax is quiet — nobody gets an error message — and it compounds with every import, form fill, and integration. CRM data hygiene stops the compounding: detect the duplicates, merge them under a rule you can defend, and prevent the next ones at the point of entry.

This playbook is for operators who own a CRM with at least 10,000 records — a RevOps lead, a sales operations manager, or a founder who has just found the same customer three times in a pipeline review. It is not for a fresh CRM with 200 contacts; at that size, manual cleanup is faster than any rule. The scope is the CRM itself: Salesforce and HubSpot, whose duplicate machinery is documented well enough to build on.

Duplicate management has three layers, in order: prevention at the point of entry, detection on a schedule, and merge under a written survivorship rule. Teams that start with merging and skip prevention re-merge the same people every quarter, because the entry points that created the duplicates are still open.

How duplicates get in: the four entry points and their gaps

Duplicates are not created by bad data, but by normal operations running without a uniqueness check. Each entry point has a different gap, and the gap is what prevention has to close.

Entry pointHow the duplicate formsThe gap in the native checkAuthority
Web formsA known contact submits with a new email or a typoForm matches on email only; a new email is a new contactHubSpot Deduplication docs
ImportsA list is loaded without Record ID or unique keysRows saved in the same import are not compared with each otherSalesforce Duplicate Rules docs
API and integrationsA sync app creates companies by domain-less payloadCompanies created through the API are not deduplicated by domainHubSpot Deduplication docs
Manual entryA rep creates a contact instead of searching firstDuplicate rules do not run on Quick CreateSalesforce Duplicate Rules docs

HubSpot's automatic deduplication is narrow by design: contacts by email address and companies by company domain name, and only for records created manually, through forms, or through imports. Companies created through the API — including installed third-party sync apps — are not deduplicated on domain at all. Deals and tickets have no automatic deduplication. The only reliable key for those objects on import is the Record ID property, or a custom property marked as requiring unique values. HubSpot allows up to ten such custom properties per object, applying to manual edits and imports — but not to forms, where the default email and domain checks are what you get.

Salesforce's gaps are documented the same way. Duplicate rules do not run on records created through Quick Create, on leads converted unless Apex lead conversion is enabled, on records restored from the Recycle Bin, on records merged manually, or on an edit that does not touch a field in the matching rule. Records saved together in one import are not compared with each other. A team importing a 5,000-row list and assuming the rules cleaned it has made the most common error in this playbook: the rules never ran.

A merge cannot be undone. HubSpot states it on every merge path: merged records cannot be reverted. Salesforce's merge is equally permanent. The only recovery is a pre-merge export — contacts and companies with Record ID, Email, Company domain name, owner, and create date — taken before the first merge runs. If the merge rule is wrong, the export is the business. Skip it and a bad survivorship decision deletes history you cannot reconstruct.

Detection: what the native tools actually see

Detection is a matching rule: which fields to compare, and whether each must match exactly or approximately. Salesforce splits the job in two — a matching rule defines how records are compared, a duplicate rule defines what happens on a match: block, alert, or log. HubSpot's duplicates manager compares property values daily, surfaces pairs with a similarity score, and lets you merge or reject them individually or in bulk.

The limits are the operating envelope — hard-coded, and Salesforce's help article says they cannot be increased by Support:

LimitSalesforceHubSpot
Active matching rules per object52 custom rules per object (up to 9 properties each)
Total matching rules100 total, 25 activeDefault rule plus custom rules
Active duplicate rules per object5, up to 3 matching rules eachN/A — merge criteria instead
Records merged in one action3, including the masterBulk merge by criteria (see below)
Pairs the tool will showDuplicate jobs cap: 1,000,000 itemsPro/Enterprise: 10,000; Data Hub Pro: 30,000; Enterprise: 100,000
EditionsMatching rules: Professional and aboveManage duplicates: Professional and above; bulk: Data Hub Pro and above

Three limits change the design. First, Salesforce's activation threshold: when you create or modify a matching rule, the platform runs it against existing data, and if the rule identifies potential duplicates exceeding approximately 2% of total records, it refuses to activate. Support can raise the threshold to approximately 5%, but cannot remove it. Second, HubSpot's custom-rule warning: if a rule matches more than 10% of all records and the average group size is greater than three, the duplicates table may not return complete results. Third, Salesforce duplicate jobs — the org-wide scans — are limited to Performance and Unlimited editions, cap at 1,000,000 items across all jobs, and cannot compare and merge records on custom objects.

HubSpot duplicates manager documentation showing Professional and Enterprise subscriptions display up to 10,000 duplicate pairs, Data Hub Professional up to 30,000, and Data Hub Enterprise up to 100,000
HubSpot duplicates manager documentation showing Professional and Enterprise subscriptions display up to 10,000 duplicate pairs, Data Hub Professional up to 30,000, and Data Hub Enterprise up to 100,000

HubSpot's default matching properties define what "similar" means out of the box. Contacts are compared on first name, last name, email address, IP country, phone number, zip code, and company name. Companies are compared on company domain name, company name, country or region, phone number, and industry.

The merge decision: survivorship is the whole job

Detection is mechanical. The merge decision is judgment, and it is the part of the playbook that destroys value when improvised. Every merge picks a surviving record — the master — and every other record's values either survive into it or are discarded. The rule that decides the master is survivorship, and it must be written down before the first merge, because two operators merging the same pair with different instincts will produce different CRMs.

HubSpot's bulk merge criteria are the menu to choose from: most recent engagement, oldest engagement, created first, created last, or most recently updated. Custom merge rules go further — up to five criteria per rule, checked in order, with a tie-breaker: a property condition (for example, email does not contain gmail.com), most associated objects, most activities logged, or highest fill rate. For contacts, most recent engagement usually wins — the record the customer interacted with carries the live email and consent. For companies, created first usually wins — the oldest record carries the original source and first-touch attribution. Write the choice in the rule; never merge by feel.

The cost of getting survivorship wrong is not theoretical. The merged record keeps the oldest create date, the furthest lifecycle stage, and the combined activities — but the primary record's original source and first-touch attribution dominate, and the secondary's are lost.

Founder & CEO Insight — Enow A. Jovial

"A CRM with duplicates is a company arguing with itself. Sales says the customer is new, marketing says the customer is engaged, and the report says both. The merge rule is the referee, and it has to be written before the argument starts — not during it."
— Enow A. Jovial, Founder & Chief Executive Officer

The math: price the cleanup before you start it

Duplicate rate=Duplicate recordsTotal records×100\text{Duplicate rate} = \frac{\text{Duplicate records}}{\text{Total records}} \times 100
Manual cleanup hours=Duplicate pairs×Minutes per pair60\text{Manual cleanup hours} = \frac{\text{Duplicate pairs} \times \text{Minutes per pair}}{60}

Take a stated scenario: a 120,000-contact database at an 8% duplicate rate — 9,600 duplicate records, roughly 4,800 pairs. At 2 minutes per pair, the manual cleanup is 160 hours: four full work-weeks. At a stated loaded cost of $45 an hour, that is $7,200 for one cleanup pass — and it buys nothing permanent if the entry points stay open.

The pair caps decide whether the native tool can hold the work. HubSpot Professional and Enterprise display up to 10,000 pairs; at 4,800 pairs the native tool covers the scenario. At 60,000 pairs it does not — Data Hub Enterprise caps at 100,000. Salesforce's duplicate-job cap is 1,000,000 items across all jobs; a job that would exceed it must be split, and old results deleted first.

Insycle prices its data-integrity platform by records, not seats: Starter at $100 a month, Growth at $150, Professional at $200, all billed annually with 20% off, unlimited users and operations on every plan, for databases up to 500,000 records. Its Merge Duplicates module matches on exact, similar, or fuzzy criteria and bulk-merges with master selection rules, field retention control, and a full preview before anything changes. The renewal price is recalculated on the record count — an incentive to keep the database clean, because removing redundant records lowers the next bill.

OptionWhat it costsWhat it coversAuthority
Manual cleanup$7,200 one-off (scenario: 4,800 pairs at $45/hr)Any platform, any objectDerived from the scenario above
HubSpot nativeIncluded in Professional and aboveContacts and companies, daily scan, 10,000 pairs shownHubSpot duplicates manager docs
Salesforce nativeIncluded in Professional and aboveAccounts, contacts, leads; jobs on Performance and aboveSalesforce duplicate management docs
Insycle Professional$200 a month, billed annuallyAll modules, fuzzy matching, bulk merge with previewInsycle Pricing

Run the scenario against your own counts with the break-even calculator: if the manual hours exceed the annual tool cost, the tool pays for itself in the first pass. Our email list hygiene playbook covers duplicates inside the email list itself, and the HubSpot vs Salesforce vs Pipedrive pricing playbook prices the CRM seats the duplicates inflate.

Prevention: close the entry points

Prevention has four moves, each mapped to the gap it closes. One more belongs in the tooling, not the process: export the duplicates table as a CSV on every review and keep it in the audit trail, so a disputed merge can be reconstructed from the file. Where the SaaS CRM exposes a REST API, pull the duplicate counts into the same dashboard as pipeline — a SQL query over the export gives the rate by owner, source, and object, and a JSON payload from the API gives the same numbers live.

  1. Unique-value properties — mark up to ten custom properties per object as requiring unique values in HubSpot, on your real business keys: the billing-system customer ID, the partner-supplied external ID. They apply to manual edits and imports, not forms, so they are the import gate.
  2. Record ID on every import — include the Record ID property in every import file.
  3. Duplicate rules at the door — activate the standard matching rules in Salesforce and set the duplicate rule to alert on create and block on edit for exact matches. Alert, never block, on records integrations and web forms create: a blocked form is a lost lead.
  4. API hygiene — dedupe lists before loading them, because same-batch imports are never compared, and give API-created companies a domain value at creation time, because the domain check never runs on them otherwise.
Run the duplicates manager on a fixed cadence — weekly under 50,000 records, daily above it — and record the duplicate rate where you record pipeline. A rate falling 1 point a month is prevention working. A rate holding flat while you merge is prevention missing.

Who should NOT use this playbook

Do not run a bulk merge on a database whose survivorship rule is not written down — bulk merging without a rule is data destruction at scale, and it is irreversible. Do not use Salesforce duplicate jobs on custom objects expecting a merge interface; the platform cannot compare and merge those records. And do not buy a dedup tool to fix a process problem: if the entry points are still open and nobody owns the weekly review, the tool cleans the database once and the duplicates return at the same rate. Fix the entry points first, then decide whether the tool earns its $100 to $200 a month.

The 10-step CRM hygiene rollout

  • [x] Step 1: Export everything — full export with Record ID, Email, Company domain, owner, create date.
  • [x] Step 2: Measure the duplicate rate — duplicates divided by total records, per object.
  • [x] Step 3: Write the survivorship rule — master-selection criteria, per object, in writing.
  • [x] Step 4: Activate the native rules — standard matching rules on, duplicate rules alerting on create.
  • [x] Step 5: Clean the worst segment first — the object with the highest duplicate rate, in a reviewable batch.
  • [x] Step 6: Merge in reviewed batches — never bulk-merge an unreviewed rule.
  • [x] Step 7: Close the entry points — unique-value properties, Record ID on imports, API domain values.
  • [x] Step 8: Set the cadence — weekly or daily review, duplicate rate recorded.
  • [x] Step 9: Decide on a tool — manual hours against Insycle's $100 to $200 a month.
  • [x] Step 10: Re-measure monthly — the duplicate rate is the metric; a falling rate is the proof.
  1. Export before anything — the export is the only undo.
  2. Measure the rate — per object, with the formula above.
  3. Write survivorship — one rule per object, agreed by sales and marketing.
  4. Activate prevention — rules on, entry points closed.
  5. Merge in batches — reviewed, criteria-driven, never by feel.
  6. Record the rate monthly — the number that proves the operation works.