Article
Independent Email Verifier Accuracy Benchmark: How to Test Any Tool in 2026
By Unlimited Verifier Team ·

Summary
An independent email verifier accuracy benchmark is a vendor-neutral test that checks Valid, Risky and Invalid labels against real send outcomes. The metric that matters most is hard-bounce leakage, the share of Valid addresses that still bounce, which should stay under 1% for first sends.What is an independent email verifier accuracy benchmark?
An independent email verifier accuracy benchmark is a vendor-neutral test that checks each Valid, Risky and Invalid label against what happened when mail was actually sent, and the number that matters most is hard-bounce leakage, which should stay under 1%. Unlimited Verifier states 99.5% accuracy for deep verification, but that figure is the vendor's claim until a test on a list you can audit confirms it.
For the wider picture, start with our guide to email verifier accuracy.
Quick answer
- Measure the number that drives bounces. Hard-bounce leakage, the share of Valid addresses that still bounce, predicts your bounces. A single "accuracy" figure blends that with errors that cost you much less.
- Report Risky as its own category. Catch-all, disposable and role-based addresses carry different risks. Folding them into one score hides them.
- Benchmark on about 4,500 labelled addresses. That sample usually yields roughly 3,700 Valid labels, enough to pin a 95% precision to about ±0.7 points at 95% confidence.
- Our recommended thresholds: keep leakage under 1% for first sends, and re-test any tool that exceeds 2%. These are planning figures we recommend, not an industry standard.
- Re-run before you buy and at least quarterly. Lists age, and verifier results change over time.
Bottom line: An independent benchmark is a labelled test on your own addresses, and it turns a vendor's headline into a number you can check.
Why do vendor accuracy numbers disagree so much?
Vendor numbers disagree because they measure different things on different samples, and few publish enough detail to reconcile them. A "99% accurate" claim can be true and still tell you little about your list.
Four differences explain most of the gap:
- Definition. One vendor may score only Valid and Invalid labels. Another may count Risky as a wrong answer. In the worked example below, the same 4,500 labels produce 95.7% precision on Valid labels and 92.0% of labels that are unambiguously right. Both figures are correct, and they describe different things.
- Ground truth. Some tests use bounce outcomes, others use seeded mailboxes, and others use engagement. Each has blind spots.
- Domain mix. A list heavy in corporate catch-all domains will score lower for any tool than a list of consumer mailboxes.
- Timing. An address verified today may bounce in three weeks because the mailbox was closed in the meantime. That measures list decay, not verifier quality.
The structural limit matters most. RFC 5321 defines SMTP, including how servers respond to recipient addresses. Many domains accept mail at the handshake stage and only reject it later, or never. An outside verifier cannot fully confirm those mailboxes. That is why how accurate email verifiers are on catch-all domains is a different question from accuracy on standard mailboxes. For the broader question of how far to trust headline figures, see how accurate email verification really is.
Bottom line: A single accuracy figure only has meaning once you know its definition, its ground truth and its sample mix.
| Source of an accuracy figure | What it usually measures | What is usually missing |
|---|---|---|
| Vendor marketing claim | A headline rate, often on Valid/Invalid only | Sample, ground truth, Risky handling |
| Vendor test on its own sample | Performance on a curated list | Independent sample, blinding |
| Third-party comparison | Results across a shared file | Often a single domain mix, sometimes stale |
| Your own labelled benchmark | Performance on your data and domain mix | Requires effort and clean ground truth |
Only the last row answers the question you actually have. For a closer look at reading these claims, see are vendor accuracy claims for email verifiers trustworthy. For why two popular tools can disagree on one address, see why ZeroBounce and NeverBounce disagree on the same email.
Bottom line: Trust a figure only when you can see its definition, ground truth and sample mix.
How do you build a benchmark that holds up?
Build the benchmark in seven steps. Each step closes a gap that makes many published comparisons unreliable.
- Define ground truth from three sources, and note each one's noise.
- Hard bounces. A bounce with enhanced status code 5.1.1 is a strong signal that the mailbox is bad. RFC 3463 defines these enhanced status codes.
- Recent engagement. An open or click is strong evidence that a mailbox exists. Silence is not evidence that it doesn't.
- Seeded mailboxes you control. These give clean answers for a few hundred addresses, which is useful for calibration.
- Stratify the sample. Split it by provider (Gmail, Microsoft, Yahoo, other consumer, corporate) and by flag (catch-all, disposable, role-based). Accuracy varies by group, and an average hides the weak spots. Email verifier accuracy on Gmail vs Outlook addresses explains why these groups differ.
- Size the sample. To estimate a rate near 95% within ±1 percentage point at 95% confidence, you need about 1,825 labelled addresses in the class you are measuring. The formula is 1.96² × 0.95 × 0.05 ÷ 0.01², which gives 1,825. Rare classes, such as bad addresses in a mostly clean list, need larger samples.
- Blind the run. The verifier should receive only the email address. Labels, provider tags and bounce history stay in your private file.
- Score with a confusion matrix. Count every combination of ground truth and verifier label. Every metric below comes from that matrix, not from a single percentage.
- Report the mix. State the share of catch-all, disposable and role addresses. A result without its mix cannot be compared with anything.
- Repeat on a schedule. Re-run before a purchase, after a major import, and at least quarterly. Fold it into your email list maintenance best practices routine.
For a step-by-step version of this process, see how to test an email verifier's accuracy yourself. For the measurement logic behind it, see how email verifiers measure accuracy.
Bottom line: Ground truth, stratification and blinding decide whether a benchmark is useful. Sample size matters, but less than these three.
Which metrics should a benchmark report?
A useful benchmark reports five metrics. Each answers a different operational question, and the first two predict bounces.
In this article, a false Valid is a bad address labelled Valid, and a false Invalid is a good address labelled Invalid. The false positives vs false negatives article explains why the same terms get used in opposite ways across the industry, so always define yours.
| Metric | Formula | What it tells you | Our recommended threshold |
|---|---|---|---|
| Valid precision | Good among Valid labels ÷ all Valid labels | How often a Valid label is safe to send | Above 99% before first sends |
| Hard-bounce leakage | 1 − Valid precision | Bounces you will still take after cleaning | Under 1%; re-test the tool above 2% |
| Invalid recall | Bad caught as Invalid ÷ all bad | How much dead weight the tool removes | Above 90% if you plan to purge Invalid automatically |
| False Invalid rate | Good labelled Invalid ÷ all good | Good contacts you would wrongly delete | Under 2% |
| Risky rate | Risky labels ÷ all labels | Workload and sending restrictions | Under 10% is workable for most lists |
These thresholds are our recommendations for planning, not published rules. Set your own tolerance based on your sender reputation and what a bounce spike would cost you.
Two of these numbers do the most work. Valid precision and its inverse, leakage, predict the bounces you will take. The other three tell you how much cleanup the tool creates. For how an unknown or Risky share should look, see what is a good unknown rate for an email verifier. For why a Valid result does not guarantee delivery, see does a valid result mean the email will be delivered.
Bottom line: Valid precision and hard-bounce leakage predict bounces. The other metrics measure the cost of the cleanup.
How do the major verifiers compare on documented features?
Verifiers differ less in headline accuracy than in how they categorise results, and that categorisation decides what you can benchmark. The table below is based on each vendor's public documentation as we understand it at the time of writing, not on our own testing. Check current docs before you run anything.
| Tool | Result categories in public docs | Distinctive documented feature | What to benchmark first |
|---|---|---|---|
| ZeroBounce | Separate statuses for catch-all, spamtrap, abuse and do-not-mail | Spamtrap and abuse statuses sit alongside validity | Spamtrap handling, which is hard to test directly |
| NeverBounce | Valid, invalid, disposable, catchall and unknown | A distinct unknown result | The share and handling of unknown results |
| Kickbox | Result labels paired with a Sendex score on a 0–1 scale | A continuous score as well as a label | Where you set the score cut-off |
| Unlimited Verifier | Valid, Risky and Invalid; Risky groups catch-all, disposable and role-based addresses | Deep verification, with the company's stated 99.5% figure; engine source is on GitHub | Whether the Risky share fits your sending plan |
Each tool carries its own trade-off to test:
- ZeroBounce's richer status list means you score each status separately. A single "valid" accuracy figure would hide spamtrap errors, which are among the costliest to get wrong.
- NeverBounce's unknown result is its own outcome. A high unknown share hands the decision back to you, so benchmark unknowns as a category rather than ignoring them.
- Kickbox's Sendex score is continuous. Benchmarking it means choosing a cut-off, and that cut-off is the real decision.
- Unlimited Verifier's Risky bucket mixes catch-all, disposable and role addresses, which carry different risks. Split Risky by flag in your benchmark, or one figure will hide very different outcomes.
Reading the open-source engine tells you which checks run. It does not tell you how often those checks are right. Only a labelled benchmark does that. For a shortlist framed by use case, see our best email verification tools overview, or compare our shortlist of verifiers directly. For buying criteria, see how to choose an email verifier.
Bottom line: Compare tools on how they categorise results, then benchmark each category. The headline accuracy figure comes last.
What accuracy should you expect from verifiers in 2026?
No universal accuracy figure exists, and any article that offers one should be read with suspicion. Accuracy depends on your domain mix, the age of your list and the share of catch-all addresses.
Three points frame realistic expectations:
- Consumer-heavy lists usually verify more cleanly than B2B lists, where corporate catch-all domains are common. That is why email verification for ecommerce and SaaS needs a different benchmark from a pure consumer list.
- Old lists decay. Addresses that were valid when collected can be closed since, so a benchmark on a two-year-old list measures decay as much as verifier quality.
- Sources disagree on the thresholds that matter. Google's sender guidelines describe keeping user-reported spam rates under 0.1% and never letting them reach 0.3%. Amazon SES's documentation describes account review at a 5% bounce rate and a pause at 10%. Other ESPs act at lower levels without publishing figures. Where sources differ, plan to the stricter one.
Our recommendation is to target under 1% hard bounces on first sends, and to require a verifier to show leakage under 2% on your own labelled sample before you trust its Valid results. We have not published a 2026 ranking of verifiers, because any ranking built from one vendor's test set would need your own labelled data to be meaningful.
Bottom line: Expect results to vary with your list. Set thresholds for your own sends, and benchmark against them.
What does a 4,500-address benchmark look like in numbers?
This example is hypothetical. The counts are invented to show the arithmetic, and they do not describe any real verifier's performance.
Setup: You build a sample of 4,500 addresses from past campaigns. Ground truth comes from bounce logs and recent engagement:
- Truly good: 3,600
- Truly bad: 900
You upload the blinded file, and the verifier returns:
- Valid: 3,700
- Invalid: 650
- Risky: 150
When you join the labels to ground truth, the confusion matrix looks like this:
| Label \ Truth | Good | Bad | Total |
|---|---|---|---|
| Valid | 3,540 | 160 | 3,700 |
| Invalid | 50 | 600 | 650 |
| Risky | 10 | 140 | 150 |
| Total | 3,600 | 900 | 4,500 |
Metrics:
- Valid precision: 3,540 ÷ 3,700 = 95.7%
- Hard-bounce leakage: 160 ÷ 3,700 = 4.3%, well above our 2% re-test line
- Invalid recall: 600 ÷ 900 = 66.7%. The remaining bad addresses sit in Risky (140, or 15.6%) or Valid (160, or 17.8%)
- False Invalid rate: 50 ÷ 3,600 = 1.4%
- Risky rate: 150 ÷ 4,500 = 3.3%
- Unambiguously right labels: (3,540 + 600) ÷ 4,500 = 92.0%, counting Risky as neither right nor wrong
Interpretation: A vendor could report "92% accurate" or "95.7% precise on Valid" from the same data, and both would be true. The figure that matters for sending is the 4.3% leakage, because those 160 addresses are the bounces you would take. Invalid recall of 66.7% also means a third of bad addresses are still in your list, so you would need a second pass on Risky and Valid.
Sampling error: With 3,700 Valid labels, the 95% confidence interval around 95.7% is about ±0.7 points. The result is precise enough to act on.
Plan limits: Unlimited Verifier's Free plan allows 25 lists a month of up to 150 emails each, which is 25 × 150 = 3,750 addresses in total. That does not cover 4,500. Free also keeps results for only 24 hours. The Pro plan at $29 a month (or $25 billed annually) allows lists of up to 25,000 emails and 25,000 deep verifications a month, so the full sample fits in one list. At $29 for one month, that is about $6.44 per 1,000 addresses checked, and the effective cost falls if you use more of the allowance. For how pricing models compare, see email verification pricing.
Bottom line: Headline accuracy can look excellent while leakage through Valid remains the number that costs you bounces. Score both.
How do you run the benchmark in Unlimited Verifier?
These steps run a benchmark on up to 25,000 addresses per list on Pro, or 150 per list on Free. They assume you have already built your labelled file.
- Build two files. The first is your labelled file, with columns for email, ground truth, provider and domain type. Keep it private. The second is a blinded file that contains only the email column. Upload only the blinded file.
- Create an account. Sign up and choose the plan that fits your sample. A 4,500-address sample needs Pro, because Free caps each list at 150 emails.
- Rehearse on Free if you like. A 150-address test list uses 150 of Free's 750 monthly deep verifications, so you can practise the full workflow before paying.
- Upload the blinded file. Go to Upload and verify a list. The tool accepts CSV or Excel files up to 10MB and detects the email column automatically.
- Run verification at full depth. Standard verification is unlimited on every plan. For a fair comparison, run deep verification on every row so each tool is tested at the same depth. Pro's 25,000 monthly deep verifications covers a 4,500-address sample.
- Export the results. Download them as CSV, Excel or JSON. JSON suits scripts, and CSV suits spreadsheets. On Free, export right away, because results are kept for 24 hours. On Pro, they are kept for 30 days.
- Join and score. Match the results to your labelled file by email address. In a spreadsheet, put the verifier label in column D and ground truth in column C. Then
=COUNTIFS(D:D,"Valid",C:C,"bad")counts false Valid labels, and you can build the full confusion matrix from there. - Check disputed rows one at a time. For addresses that look wrong, use the free email checker to spot-check a single address. For catch-all rows, read about catch-all email verification before deciding how to score them.
- Automate repeat tests. The email verification API handles single-email checks and bulk batches of up to 10,000 emails per request. API access is part of Pro, which includes 100,000 API emails a month. For how this fits into automated pipelines, see email verification API and automation and real-time email validation.
Because there are no native ESP, CRM or Zapier integrations, the practical loop is to export a CSV, verify it, and re-import the results, or call the API from your own scripts.
Bottom line: Blind the file, run deep verification on every row, export before any retention window closes, and then score the confusion matrix.
What are the common mistakes in accuracy benchmarks?
Most failed benchmarks fail in predictable ways. Watch for these:
- Testing on a list you have already cleaned. A pre-cleaned list tends to flatter every tool. Use raw data, including the messy parts.
- Letting the vendor pick the sample. Vendor-chosen samples tend to lean toward that vendor's strengths. Choose your own.
- Scoring Valid and Invalid only. Ignoring Risky hides the unknown share, which is often where the cleanup work lands.
- Treating silence as proof on catch-all domains. Most catch-all servers accept nearly everything at the handshake, so a lack of bounce proves little.
- Using too few addresses per class. Under a few hundred labelled addresses in any class, a handful of results can swing the metric.
- Mixing time periods. Bounce logs from three years ago, compared with a verification run today, measure list decay rather than verifier quality.
- Losing results to a retention window. On Free, results expire after 24 hours. Export before that.
- Relying on one number. Report the matrix, the mix and the thresholds together.
- Pricing a benchmark per email alone. A low per-email price matters less than how many passes you need. Standard verification is unlimited on every Unlimited Verifier plan, which makes repeat benchmarks cheaper to run.
Bottom line: Most accuracy benchmarks fail on sample design and scoring, not on the tool itself.
How we compared (and limitations)
This article is based on public documentation, not on tests we ran ourselves. We did not benchmark any verifier, and we have not reproduced any published 2026 results.
- Vendor feature descriptions come from each vendor's public documentation as we understand it at the time of writing. These may change, so check current docs before testing.
- Unlimited Verifier's 99.5% deep-verification accuracy is the company's own stated figure. It is not an independent benchmark, and the method above is how you can test it.
- Thresholds such as 1% and 2% leakage are our recommendations for planning, not published industry standards.
- The worked example is hypothetical. It exists to show the arithmetic, not any real verifier's performance.
- Standards and platform references cover SMTP behaviour (RFC 5321), enhanced status codes (RFC 3463), bulk-sender expectations from Google and bounce-rate thresholds from AWS SES's documentation. M3AAWG publishes sender best practices that are useful background for list hygiene.
Run the benchmark on your own labelled sample before you trust any headline figure, including ours.
Benchmark steps
- Build a labelled sample of about 4,500 addresses that mirrors your real list mix.
- Run each address through the verifier and record Valid, Risky or Invalid.
- Send mail to Valid and Risky addresses and record permanent bounces.
- Calculate hard-bounce leakage: bounced Valid addresses divided by all Valid addresses.
- Compare leakage to the 1% first-send threshold and re-test any tool above 2%.
- Repeat the test before buying and at least quarterly.
Frequently asked questions
What is a good email verifier accuracy rate?
There is no single industry standard, so judge a verifier by hard-bounce leakage rather than one headline number. Our recommended planning threshold is leakage under 1% on Valid addresses for first sends, with re-testing if a tool exceeds 2%. Always check the definition and sample behind any accuracy claim.
How many emails do I need to benchmark a verifier?
A sample of about 4,500 labelled addresses usually yields roughly 3,700 Valid labels, which is enough to estimate precision to about plus or minus 0.7 points at 95% confidence. Use a mix that reflects your real list, including catch-all and role-based addresses.
What is hard-bounce leakage?
Hard-bounce leakage is the share of addresses a verifier labelled Valid that still produce a permanent bounce when mail is sent. It is the metric most closely tied to your bounce rate and sender reputation, so it matters more than a blended accuracy score.
Why should Risky be a separate category?
Risky addresses, such as catch-all, disposable and role-based ones, carry different delivery risks. Folding them into Valid or Invalid hides those differences and makes the score harder to act on. Reporting Risky separately lets you choose how to treat each segment.
Why do vendors report different accuracy numbers?
Vendors define accuracy differently, use different ground truth such as bounces, seeded mailboxes or engagement, and test on different domain mixes. Timing also matters because mailboxes can close after verification. Without the definition and sample, two claims cannot be compared fairly.
How often should I re-run an email verifier benchmark?
Re-run a benchmark before buying a tool and at least quarterly after that. Lists age and mailboxes close, and verifier results can change over time, so a test from last year may no longer reflect current performance.
Can an email verifier confirm every mailbox exists?
No. Many domains, especially catch-all domains, accept mail during the SMTP handshake and reject later or never. An outside verifier cannot fully confirm those mailboxes, so Risky labels on these domains reflect uncertainty rather than a definitive answer.
Should I trust a vendor's 99% accuracy claim?
Treat a vendor's accuracy claim as unverified until you test it on a list you can audit. Ask what counts as correct, how ground truth was measured, and what share of the sample was catch-all. A labelled test on your own addresses is the only way to confirm the figure.
Cut Bounces and Protect Your Sender Reputation With Unlimited Email Verification
Every invalid address you send to hurts your sender reputation and your inbox placement. Verifying your list before each send means fewer bounces, healthier deliverability, and a predictable cost. With Unlimited Verifier, standard verification is unlimited on every plan, with no per-email fees.
Unlimited Verifier checks syntax, domain, MX records and the SMTP mailbox. It also detects catch-all domains and flags disposable and role-based addresses. Export a CSV from your email platform or CRM, upload and verify the list, then re-import the clean results. You can also automate checks with the email verification API.
| Free | Pro | |
|---|---|---|
| Price | $0 | $29/month ($25/month billed annually) |
| Lists | 25 a month, up to 150 emails each | Unlimited, up to 25,000 emails each |
| Deep verifications | 750 a month | 25,000 a month |
| API access | No | Yes (100,000 emails a month) |
| Result retention | 24 hours | 30 days |
Why teams choose Unlimited Verifier
- Unlimited standard verification on all plans, including Free, with no per-email fees.
- Catch-all detection so you know which domains accept every address.
- Clear Valid / Risky / Invalid results, exportable as CSV, Excel or JSON.
- An open-source verification engine you can inspect on GitHub.
Unlimited Verifier states 99.5% accuracy for deep verification.
Create a free account and verify your next list before you hit send.
Want to test a single address first? Try the free email checker.