Article

Independent Email Verifier Accuracy Benchmark: How to Test Any Tool in 2026

By Unlimited Verifier Team ·

Diagram of an independent email verifier accuracy benchmark showing labels checked against real send outcomes to measure hard-bounce leakage.

Summary

An independent email verifier accuracy benchmark is a vendor-neutral test that checks Valid, Risky and Invalid labels against real send outcomes. The metric that matters most is hard-bounce leakage, the share of Valid addresses that still bounce, which should stay under 1% for first sends.

What is an independent email verifier accuracy benchmark?

An independent email verifier accuracy benchmark is a vendor-neutral test that checks each Valid, Risky and Invalid label against what happened when mail was actually sent, and the number that matters most is hard-bounce leakage, which should stay under 1%. Unlimited Verifier states 99.5% accuracy for deep verification, but that figure is the vendor's claim until a test on a list you can audit confirms it.

For the wider picture, start with our guide to email verifier accuracy.

Quick answer

Bottom line: An independent benchmark is a labelled test on your own addresses, and it turns a vendor's headline into a number you can check.

Why do vendor accuracy numbers disagree so much?

Vendor numbers disagree because they measure different things on different samples, and few publish enough detail to reconcile them. A "99% accurate" claim can be true and still tell you little about your list.

Four differences explain most of the gap:

The structural limit matters most. RFC 5321 defines SMTP, including how servers respond to recipient addresses. Many domains accept mail at the handshake stage and only reject it later, or never. An outside verifier cannot fully confirm those mailboxes. That is why how accurate email verifiers are on catch-all domains is a different question from accuracy on standard mailboxes. For the broader question of how far to trust headline figures, see how accurate email verification really is.

Bottom line: A single accuracy figure only has meaning once you know its definition, its ground truth and its sample mix.

Source of an accuracy figure What it usually measures What is usually missing
Vendor marketing claim A headline rate, often on Valid/Invalid only Sample, ground truth, Risky handling
Vendor test on its own sample Performance on a curated list Independent sample, blinding
Third-party comparison Results across a shared file Often a single domain mix, sometimes stale
Your own labelled benchmark Performance on your data and domain mix Requires effort and clean ground truth

Only the last row answers the question you actually have. For a closer look at reading these claims, see are vendor accuracy claims for email verifiers trustworthy. For why two popular tools can disagree on one address, see why ZeroBounce and NeverBounce disagree on the same email.

Bottom line: Trust a figure only when you can see its definition, ground truth and sample mix.

How do you build a benchmark that holds up?

Build the benchmark in seven steps. Each step closes a gap that makes many published comparisons unreliable.

  1. Define ground truth from three sources, and note each one's noise.
    • Hard bounces. A bounce with enhanced status code 5.1.1 is a strong signal that the mailbox is bad. RFC 3463 defines these enhanced status codes.
    • Recent engagement. An open or click is strong evidence that a mailbox exists. Silence is not evidence that it doesn't.
    • Seeded mailboxes you control. These give clean answers for a few hundred addresses, which is useful for calibration.
  2. Stratify the sample. Split it by provider (Gmail, Microsoft, Yahoo, other consumer, corporate) and by flag (catch-all, disposable, role-based). Accuracy varies by group, and an average hides the weak spots. Email verifier accuracy on Gmail vs Outlook addresses explains why these groups differ.
  3. Size the sample. To estimate a rate near 95% within ±1 percentage point at 95% confidence, you need about 1,825 labelled addresses in the class you are measuring. The formula is 1.96² × 0.95 × 0.05 ÷ 0.01², which gives 1,825. Rare classes, such as bad addresses in a mostly clean list, need larger samples.
  4. Blind the run. The verifier should receive only the email address. Labels, provider tags and bounce history stay in your private file.
  5. Score with a confusion matrix. Count every combination of ground truth and verifier label. Every metric below comes from that matrix, not from a single percentage.
  6. Report the mix. State the share of catch-all, disposable and role addresses. A result without its mix cannot be compared with anything.
  7. Repeat on a schedule. Re-run before a purchase, after a major import, and at least quarterly. Fold it into your email list maintenance best practices routine.

For a step-by-step version of this process, see how to test an email verifier's accuracy yourself. For the measurement logic behind it, see how email verifiers measure accuracy.

Bottom line: Ground truth, stratification and blinding decide whether a benchmark is useful. Sample size matters, but less than these three.

Which metrics should a benchmark report?

A useful benchmark reports five metrics. Each answers a different operational question, and the first two predict bounces.

In this article, a false Valid is a bad address labelled Valid, and a false Invalid is a good address labelled Invalid. The false positives vs false negatives article explains why the same terms get used in opposite ways across the industry, so always define yours.

Metric Formula What it tells you Our recommended threshold
Valid precision Good among Valid labels ÷ all Valid labels How often a Valid label is safe to send Above 99% before first sends
Hard-bounce leakage 1 − Valid precision Bounces you will still take after cleaning Under 1%; re-test the tool above 2%
Invalid recall Bad caught as Invalid ÷ all bad How much dead weight the tool removes Above 90% if you plan to purge Invalid automatically
False Invalid rate Good labelled Invalid ÷ all good Good contacts you would wrongly delete Under 2%
Risky rate Risky labels ÷ all labels Workload and sending restrictions Under 10% is workable for most lists

These thresholds are our recommendations for planning, not published rules. Set your own tolerance based on your sender reputation and what a bounce spike would cost you.

Two of these numbers do the most work. Valid precision and its inverse, leakage, predict the bounces you will take. The other three tell you how much cleanup the tool creates. For how an unknown or Risky share should look, see what is a good unknown rate for an email verifier. For why a Valid result does not guarantee delivery, see does a valid result mean the email will be delivered.

Bottom line: Valid precision and hard-bounce leakage predict bounces. The other metrics measure the cost of the cleanup.

How do the major verifiers compare on documented features?

Verifiers differ less in headline accuracy than in how they categorise results, and that categorisation decides what you can benchmark. The table below is based on each vendor's public documentation as we understand it at the time of writing, not on our own testing. Check current docs before you run anything.

Tool Result categories in public docs Distinctive documented feature What to benchmark first
ZeroBounce Separate statuses for catch-all, spamtrap, abuse and do-not-mail Spamtrap and abuse statuses sit alongside validity Spamtrap handling, which is hard to test directly
NeverBounce Valid, invalid, disposable, catchall and unknown A distinct unknown result The share and handling of unknown results
Kickbox Result labels paired with a Sendex score on a 0–1 scale A continuous score as well as a label Where you set the score cut-off
Unlimited Verifier Valid, Risky and Invalid; Risky groups catch-all, disposable and role-based addresses Deep verification, with the company's stated 99.5% figure; engine source is on GitHub Whether the Risky share fits your sending plan

Each tool carries its own trade-off to test:

Reading the open-source engine tells you which checks run. It does not tell you how often those checks are right. Only a labelled benchmark does that. For a shortlist framed by use case, see our best email verification tools overview, or compare our shortlist of verifiers directly. For buying criteria, see how to choose an email verifier.

Bottom line: Compare tools on how they categorise results, then benchmark each category. The headline accuracy figure comes last.

What accuracy should you expect from verifiers in 2026?

No universal accuracy figure exists, and any article that offers one should be read with suspicion. Accuracy depends on your domain mix, the age of your list and the share of catch-all addresses.

Three points frame realistic expectations:

Our recommendation is to target under 1% hard bounces on first sends, and to require a verifier to show leakage under 2% on your own labelled sample before you trust its Valid results. We have not published a 2026 ranking of verifiers, because any ranking built from one vendor's test set would need your own labelled data to be meaningful.

Bottom line: Expect results to vary with your list. Set thresholds for your own sends, and benchmark against them.

What does a 4,500-address benchmark look like in numbers?

This example is hypothetical. The counts are invented to show the arithmetic, and they do not describe any real verifier's performance.

Setup: You build a sample of 4,500 addresses from past campaigns. Ground truth comes from bounce logs and recent engagement:

You upload the blinded file, and the verifier returns:

When you join the labels to ground truth, the confusion matrix looks like this:

Label \ Truth Good Bad Total
Valid 3,540 160 3,700
Invalid 50 600 650
Risky 10 140 150
Total 3,600 900 4,500

Metrics:

Interpretation: A vendor could report "92% accurate" or "95.7% precise on Valid" from the same data, and both would be true. The figure that matters for sending is the 4.3% leakage, because those 160 addresses are the bounces you would take. Invalid recall of 66.7% also means a third of bad addresses are still in your list, so you would need a second pass on Risky and Valid.

Sampling error: With 3,700 Valid labels, the 95% confidence interval around 95.7% is about ±0.7 points. The result is precise enough to act on.

Plan limits: Unlimited Verifier's Free plan allows 25 lists a month of up to 150 emails each, which is 25 × 150 = 3,750 addresses in total. That does not cover 4,500. Free also keeps results for only 24 hours. The Pro plan at $29 a month (or $25 billed annually) allows lists of up to 25,000 emails and 25,000 deep verifications a month, so the full sample fits in one list. At $29 for one month, that is about $6.44 per 1,000 addresses checked, and the effective cost falls if you use more of the allowance. For how pricing models compare, see email verification pricing.

Bottom line: Headline accuracy can look excellent while leakage through Valid remains the number that costs you bounces. Score both.

How do you run the benchmark in Unlimited Verifier?

These steps run a benchmark on up to 25,000 addresses per list on Pro, or 150 per list on Free. They assume you have already built your labelled file.

  1. Build two files. The first is your labelled file, with columns for email, ground truth, provider and domain type. Keep it private. The second is a blinded file that contains only the email column. Upload only the blinded file.
  2. Create an account. Sign up and choose the plan that fits your sample. A 4,500-address sample needs Pro, because Free caps each list at 150 emails.
  3. Rehearse on Free if you like. A 150-address test list uses 150 of Free's 750 monthly deep verifications, so you can practise the full workflow before paying.
  4. Upload the blinded file. Go to Upload and verify a list. The tool accepts CSV or Excel files up to 10MB and detects the email column automatically.
  5. Run verification at full depth. Standard verification is unlimited on every plan. For a fair comparison, run deep verification on every row so each tool is tested at the same depth. Pro's 25,000 monthly deep verifications covers a 4,500-address sample.
  6. Export the results. Download them as CSV, Excel or JSON. JSON suits scripts, and CSV suits spreadsheets. On Free, export right away, because results are kept for 24 hours. On Pro, they are kept for 30 days.
  7. Join and score. Match the results to your labelled file by email address. In a spreadsheet, put the verifier label in column D and ground truth in column C. Then =COUNTIFS(D:D,"Valid",C:C,"bad") counts false Valid labels, and you can build the full confusion matrix from there.
  8. Check disputed rows one at a time. For addresses that look wrong, use the free email checker to spot-check a single address. For catch-all rows, read about catch-all email verification before deciding how to score them.
  9. Automate repeat tests. The email verification API handles single-email checks and bulk batches of up to 10,000 emails per request. API access is part of Pro, which includes 100,000 API emails a month. For how this fits into automated pipelines, see email verification API and automation and real-time email validation.

Because there are no native ESP, CRM or Zapier integrations, the practical loop is to export a CSV, verify it, and re-import the results, or call the API from your own scripts.

Bottom line: Blind the file, run deep verification on every row, export before any retention window closes, and then score the confusion matrix.

What are the common mistakes in accuracy benchmarks?

Most failed benchmarks fail in predictable ways. Watch for these:

Bottom line: Most accuracy benchmarks fail on sample design and scoring, not on the tool itself.

How we compared (and limitations)

This article is based on public documentation, not on tests we ran ourselves. We did not benchmark any verifier, and we have not reproduced any published 2026 results.

Run the benchmark on your own labelled sample before you trust any headline figure, including ours.

Benchmark steps

  1. Build a labelled sample of about 4,500 addresses that mirrors your real list mix.
  2. Run each address through the verifier and record Valid, Risky or Invalid.
  3. Send mail to Valid and Risky addresses and record permanent bounces.
  4. Calculate hard-bounce leakage: bounced Valid addresses divided by all Valid addresses.
  5. Compare leakage to the 1% first-send threshold and re-test any tool above 2%.
  6. Repeat the test before buying and at least quarterly.

Frequently asked questions

What is a good email verifier accuracy rate?

There is no single industry standard, so judge a verifier by hard-bounce leakage rather than one headline number. Our recommended planning threshold is leakage under 1% on Valid addresses for first sends, with re-testing if a tool exceeds 2%. Always check the definition and sample behind any accuracy claim.

How many emails do I need to benchmark a verifier?

A sample of about 4,500 labelled addresses usually yields roughly 3,700 Valid labels, which is enough to estimate precision to about plus or minus 0.7 points at 95% confidence. Use a mix that reflects your real list, including catch-all and role-based addresses.

What is hard-bounce leakage?

Hard-bounce leakage is the share of addresses a verifier labelled Valid that still produce a permanent bounce when mail is sent. It is the metric most closely tied to your bounce rate and sender reputation, so it matters more than a blended accuracy score.

Why should Risky be a separate category?

Risky addresses, such as catch-all, disposable and role-based ones, carry different delivery risks. Folding them into Valid or Invalid hides those differences and makes the score harder to act on. Reporting Risky separately lets you choose how to treat each segment.

Why do vendors report different accuracy numbers?

Vendors define accuracy differently, use different ground truth such as bounces, seeded mailboxes or engagement, and test on different domain mixes. Timing also matters because mailboxes can close after verification. Without the definition and sample, two claims cannot be compared fairly.

How often should I re-run an email verifier benchmark?

Re-run a benchmark before buying a tool and at least quarterly after that. Lists age and mailboxes close, and verifier results can change over time, so a test from last year may no longer reflect current performance.

Can an email verifier confirm every mailbox exists?

No. Many domains, especially catch-all domains, accept mail during the SMTP handshake and reject later or never. An outside verifier cannot fully confirm those mailboxes, so Risky labels on these domains reflect uncertainty rather than a definitive answer.

Should I trust a vendor's 99% accuracy claim?

Treat a vendor's accuracy claim as unverified until you test it on a list you can audit. Ask what counts as correct, how ground truth was measured, and what share of the sample was catch-all. A labelled test on your own addresses is the only way to confirm the figure.

Cut Bounces and Protect Your Sender Reputation With Unlimited Email Verification

Every invalid address you send to hurts your sender reputation and your inbox placement. Verifying your list before each send means fewer bounces, healthier deliverability, and a predictable cost. With Unlimited Verifier, standard verification is unlimited on every plan, with no per-email fees.

Unlimited Verifier checks syntax, domain, MX records and the SMTP mailbox. It also detects catch-all domains and flags disposable and role-based addresses. Export a CSV from your email platform or CRM, upload and verify the list, then re-import the clean results. You can also automate checks with the email verification API.

Free Pro
Price $0 $29/month ($25/month billed annually)
Lists 25 a month, up to 150 emails each Unlimited, up to 25,000 emails each
Deep verifications 750 a month 25,000 a month
API access No Yes (100,000 emails a month)
Result retention 24 hours 30 days

Why teams choose Unlimited Verifier

Unlimited Verifier states 99.5% accuracy for deep verification.

Create a free account and verify your next list before you hit send.

Want to test a single address first? Try the free email checker.