Article

How Email Verifiers Measure Accuracy (and What Their Numbers Hide)

By Unlimited Verifier Team ·

Diagram showing how an email verifier's labels are compared against ground truth bounce outcomes to measure email verifier accuracy.

Summary

Email verifiers measure accuracy by comparing their Valid, Risky and Invalid labels against ground truth, addresses whose real delivery outcome is already known. Accuracy alone hides two costly errors: false valids that bounce and false invalids that discard working contacts.

Email verifiers measure accuracy by checking their Valid, Risky and Invalid labels against ground truth, addresses whose real delivery outcome is already known, and reporting the share labeled correctly: 97 correct calls out of every 100 means 97% accuracy. A headline figure such as the 99.5% that Unlimited Verifier states for deep verification only means something once you know the sample, the ground truth and the kinds of error behind it. For the wider picture, see our pillar on email verifier accuracy.

Quick answer

What does "accuracy" mean for an email verifier?

For an email verifier, accuracy is the number of addresses labeled correctly divided by the total number labeled. That single ratio conceals the two mistakes that cost you money. A false valid puts a bad address into your sending segment and produces a bounce. A false invalid removes a working contact you could have mailed.

Verifier label Address really delivers Address really bounces
Valid Correct False valid: a bounce that cleaning should have caught
Invalid False invalid: a working contact you suppressed Correct
Risky Held back, not scored right or wrong Held back, not scored right or wrong

Vendors score Risky differently. Risky usually covers catch-all, disposable and role-based addresses that can't be confirmed. Some vendors count Risky as wrong, some exclude it, and some count it as right because it isn't Valid. That single choice can move a headline figure noticeably, so ask how it was handled before you compare numbers. Our recommendation is to report Risky as its own share of the list and exclude it from the accuracy ratio, so the rule is visible rather than buried.

The term "false positive" is used inconsistently in this field, so this article uses the explicit labels "false valid" and "false invalid." The false positive guide explains the naming conventions. Teams working under email verification compliance and hygiene requirements need both error types measured, not just one headline score.

Bottom line: Accuracy counts only the right calls. You need to know which wrong calls the tool made.

What can an SMTP mailbox check actually prove?

Many verifiers base their Valid or Invalid call on an SMTP probe. The tool connects to the recipient domain's mail server, introduces itself, names a sender and asks whether the recipient is accepted, without sending a message. The server's reply is the evidence. RFC 5321 defines the basic reply codes: 250 means the recipient is accepted, 550 means the mailbox is unavailable, and 4yz codes are temporary failures. RFC 3463 adds enhanced status codes, such as 5.1.1 for a bad destination mailbox.

That evidence has four limits:

So an SMTP check shows that a server would accept mail for an address at that moment. It doesn't show that a person reads the mail or that it reaches the inbox. The delivery question is separate from the accuracy question.

Bottom line: A 250 reply is strong evidence about the server and weaker evidence about the person behind the address.

How do verifiers get ground truth for an address?

Ground truth is the outcome a verifier is trying to predict. Every accuracy claim rests on how that outcome was observed, and each source has a blind spot.

Ground-truth source What it can prove Blind spot Effort for you
Vendor-controlled test mailboxes Whether a probe matches a known mailbox state Seed mailboxes rarely match your list's age, domains or mix None, and usually not auditable
Real send outcomes (hard bounces) Whether mail was refused on delivery Catch-all and accepted-then-dropped mail look deliverable Medium: needs your ESP's bounce logs
Bounce codes (for example, 5.1.1 vs 5.7.x) Whether a rejection was about a missing mailbox or a policy block Some rejections are policy blocks, not dead addresses Low
Recent engagement (clicks) That an address was live recently Silence doesn't prove an address is dead Low
Hand review of a sample Judgment on ambiguous cases Slow, subjective and small High

The most useful ground truth for a sender is your own send results, read through bounce codes. A 5.1.1 rejection means the mailbox doesn't exist. A 5.7.x code usually means a policy or reputation block, which may not reflect a dead address at all. Treating the two the same inflates your false-valid count and makes a verifier look worse than it is. Recent clicks are a useful second check: an address that was clicked in the last 90 days was very likely live when it was clicked, so Invalid addresses with recent clicks are evidence of false invalids. Clicks can't reveal every false invalid, though, because a live reader who never clicks looks identical to a dead address.

Bottom line: Judge a verifier's accuracy against outcomes you can observe, and read the bounce code, not just the bounce count.

Why does a single accuracy percentage mislead?

Accuracy depends on the base rate, the share of your list that is genuinely deliverable. Consider an illustrative 100,000-row list in which 88,000 addresses are deliverable and 12,000 are not. A tool that labeled every row Valid would be right 88% of the time without checking a single address.

Now compare two hypothetical verifiers with near-identical headline scores:

The scores are nearly identical, but the outcomes differ. Tool X's Valid segment holds 89,900 addresses, of which 2,400 (about 2.7%) will bounce. Tool Y's Valid segment holds 85,600 addresses, of which 300 (about 0.4%) will bounce, but it discards 2,700 real contacts, which is 3.1% of the deliverable list. If bounce reputation is your main concern, Tool Y is safer. If reach matters more, Tool X keeps more of your audience. The headline score doesn't tell you which trade-off you're making.

Bottom line: Two tools with the same accuracy can send you very different risks, so measure the error mix.

Which metrics should you compare instead?

Six metrics give you a usable picture. The first two matter most for sending, and the last two show how much of the headline figure is real.

Metric Plain-English definition Why it matters Watch for
Valid precision Share of Valid-labeled addresses that actually deliver Predicts bounces from the segment you'll mail Needs real send outcomes
False-invalid rate Share of Invalid-labeled addresses that were live Measures contacts you lose Usually estimated with engagement evidence
Invalid recall Share of all undeliverable addresses caught as Invalid Measures bad data left in your Valid segment Requires outcomes for the whole list
Risky share Risky ÷ total Sets how much of the list you hold back A high share means less mailable data
Unknown rate No definitive answer ÷ total Shows how much accuracy rests on decided addresses Often excluded from the headline
Repeatability Same result when re-tested days apart Detects unstable probes Requires two test runs

The unknown rate changes the denominator. Suppose a tool reports 98% accuracy but returns 5% unknowns and excludes them. Even if every unknown were wrong, the honest floor across the whole list would be 0.98 × 0.95 = 93.1%. Ask for the figure with unknowns included. As a working threshold, we treat an unknown rate above 5% as a reason to ask questions before trusting a result, and the unknown rate guide covers how to judge that share in more depth.

Bottom line: Valid precision and false-invalid rate tell you about sending and lost reach. Unknown rate and repeatability tell you whether the headline can be trusted.

How many addresses do you need to trust an accuracy number?

Accuracy measured on a sample carries sampling error. At 95% confidence, the margin of error for a proportion is ±1.96 × √(p(1−p)/n), where p is the observed accuracy and n is the sample size. The results:

Our recommendation is a stratified sample of at least 1,500 rows. That is enough to separate 99% from 99.5% in the main population, and it is the smallest sample that keeps a serious comparison honest.

The last example matters most. Overall samples can look precise while the subgroups that carry the most risk, such as catch-all domains, remain unreliable. The margins also assume a random or stratified sample. A sample that over-represents clean corporate addresses will flatter the result however large it is.

Bottom line: A few hundred addresses can detect large gaps, but you need thousands to separate 99% from 99.5%, and subgroups need their own samples.

Why do catch-all and unknown results drag accuracy down?

Two categories sit outside the clean right-or-wrong framing, and both distort the headline.

Catch-all domains accept every address at the SMTP stage, so the probe can't separate a real mailbox from a fake one. Accuracy on these addresses only shows up through real send outcomes, and even those are noisy. Catch-alls are common in B2B lists, so they matter more there than in consumer data. For detail, see how accurate verifiers are on catch-all domains. Unlimited Verifier flags catch-all domains so you can see which addresses sit behind them, and the catch-all email verification page explains how that handling works.

Unknown results are addresses where the tool got no definitive answer, often because of timeouts or greylisting. How a tool counts them changes the score. Excluding them inflates accuracy, and including them as errors lowers it. Report both figures, and treat the gap between them as the uncertainty you're carrying.

Bottom line: Ask for accuracy on catch-all and unknown addresses separately, and include unknowns in the denominator.

Why do two verifiers disagree on the same address?

Disagreement is normal, and it usually comes from one of five causes:

  1. Different probes. Timing, sending IP reputation and retry logic affect what the server reveals. A probe may get a clearer answer from one network than another.
  2. Different catch-all rules. Tools handle accept-all servers differently, so the same address can be Valid in one and Risky in another.
  3. Different label rules. Thresholds for Risky, and handling of role and disposable addresses, vary.
  4. Different ground truth. Each vendor calibrates on its own data and definitions.
  5. Changing data. Mailboxes are created and closed constantly, and verifiers update their logic.

The practical move is to run both tools on the same list and focus on the disagreement set. If two tools agree on 92% of 10,000 addresses, the remaining 800 addresses decide the comparison, and those are the ones to test with real send outcomes. Consensus is useful too. Addresses that two independent tools both mark Invalid are safe to suppress, and addresses where they split are where a small test send earns its keep. For a deeper look at one common pair, read why ZeroBounce and NeverBounce can disagree on the same email. When you're shortlisting best email verification tools, use the same disagreement test rather than the vendor's headline.

Bottom line: Disagreements show you where the tools' accuracy actually differs, so test those addresses first.

How do you test a verifier's accuracy on your own list?

Use this seven-step framework. Each step produces evidence you can show a colleague or a vendor.

  1. Set your rules first. Write down how Risky and unknown addresses will be scored, and which ground truth you'll use: real bounce outcomes, bounce codes and recent clicks.
  2. Build a stratified sample of 1,500 rows, split across the domain types you actually mail. Include corporate domains, Gmail, Outlook and Microsoft domains, Yahoo, catch-all domains, and disposable or role-based addresses. For ecommerce and SaaS lists, the email verification guide for ecommerce and SaaS covers which strata matter most. Add a last-engaged date to every row. The Gmail vs Outlook accuracy comparison shows why those two providers are often tested separately.
  3. Freeze and export the sample on one day, so every tool tests identical data.
  4. Verify with the tool under test using the check type you plan to rely on, and keep the original fields alongside the labels.
  5. Collect ground truth. Send only to the Valid group, most engaged first, and read bounce codes after the bounce report has settled over several days. Check Invalid addresses for recent clicks.
  6. Calculate the six metrics from the table above, with margins at 95% confidence.
  7. Re-test a subset of disagreements after a few days to measure repeatability.

Bottom line: A test is only as good as its sample and its ground truth, so design both before you run anything.

How do you run an accuracy test in Unlimited Verifier?

Follow these steps to run the framework above on a 1,500-row sample.

  1. Export your sample from your ESP as a CSV. Include the email column and a last-engaged date.
  2. Sign up for an account. The Free plan covers 25 lists a month of up to 150 emails each, with 750 deep verifications a month, but it has no API access and keeps results for only 24 hours. A 1,500-row test needs more deep verifications than Free includes, so the Pro plan is the practical choice at $29 a month, or $25 a month billed annually. Pro handles lists of up to 25,000 emails and includes 25,000 deep verifications a month. For how plans compare, see email verification pricing.
  3. Upload and verify a list. CSV and Excel files up to 10MB are accepted, and the email column is detected automatically.
  4. Run the deep check. It covers syntax, domain, MX records and the SMTP mailbox, and it flags catch-all, disposable and role-based addresses.
  5. Count the three buckets. Note the Valid, Risky and Invalid totals, and record catch-all results separately so they don't blur into Risky.
  6. Export the results as CSV, Excel or JSON before the results window closes. Pro results are kept 30 days and Free results only 24 hours, so save the file on the day you run the test.
  7. Send to the Valid segment only, most-engaged first, and collect bounce codes over several days. Import the Invalid addresses as suppressions.
  8. Re-check inconsistent addresses one at a time with the free email checker, particularly any row that looks wrong against its engagement history.
  9. Script the test later. Once the method is proven, the same checks can run inside your email verification API and automation setup, using the Email verification API. Pro includes API access for 100,000 emails a month, and each bulk request accepts up to 10,000 emails. For signup forms, see real-time email validation.

Bottom line: The whole test takes one Pro month and a few days of observation, and it produces numbers you can defend.

What does an accuracy audit look like with real numbers?

The numbers below are illustrative. The plan limits are real.

Setup: A 60,000-contact list. You draw a stratified sample of 1,000 addresses. On Pro, deep verification uses 1,000 of the 25,000 monthly deep allowance, leaving 24,000.

Verification results (hypothetical): 790 Valid, 150 Risky, 60 Invalid, and no unknowns in this run.

Ground truth:

Scaling to the full 60,000:

Cost: One Pro month at $29, plus the time to export, send and count bounces.

Bottom line: The sample tells you the direction and rough size of the error. Size the sample to the decision you need to make.

How should you read a vendor's accuracy claim?

Use the questions below before you accept any headline number.

Question to ask Good answer Red flag
What is the ground truth? A named method, such as send outcomes or test mailboxes, with dates "Proprietary dataset" with no description
How are Risky and unknown addresses counted? Reported separately, with the rule stated Silently excluded from the denominator
What was the sample size and mix? A number, a domain breakdown and a test date Not disclosed
Which check type was measured? Deep and standard results reported separately One blended figure
Has anyone else tested it? An independent test with a published method Self-reported only
Can I reproduce it on my list? A free trial or sample export you can test Nothing to test before paying

Unlimited Verifier states 99.5% accuracy for deep verification. That is the company's own figure. This article hasn't seen the methodology behind it, so treat it as a claim to test with the framework above. For context on how to weigh such claims, read whether vendor accuracy claims are trustworthy, whether 99% accuracy is real and how accurate email verification really is. For a third-party reference point across tools, the independent accuracy benchmark is worth reading alongside your own test.

Bottom line: A credible accuracy claim names its ground truth, its sample and its handling of Risky addresses. If it doesn't, it's a marketing figure.

What are the common mistakes when measuring verifier accuracy?

Once you know your accuracy, set a schedule for upkeep using email list maintenance best practices, and use how to choose an email verifier to decide which tool fits your volume.

Bottom line: Most measurement errors come from skipping a step, not from the maths.

How we compared (and limitations)

This article is based on public documentation: RFC 5321 for SMTP reply codes, RFC 3463 for enhanced status codes, and Google's email sender guidelines for what bulk senders are expected to maintain.

We have not run controlled accuracy tests of any vendor, and we have not benchmarked any verifier. The worked examples are arithmetic on hypothetical numbers, and the margins of error are standard calculations, not measured results. The 99.5% deep-verification figure is Unlimited Verifier's stated number and hasn't been independently verified here. Plan details are as published and may change, and the thresholds in this article should be rechecked against current documentation before you rely on them.

Base-rate worked example

List of 10,000 addresses, 88% truly deliverable (8,800 good, 1,200 bad).

Verifier behavior Correct calls Accuracy
Labels everything Valid 8,800 88%
Catches 1,100 bad, misses 100 9,700 97%
Catches 1,100 bad, flags 300 good Invalid 9,500 95%

The first row scores 88% with zero verification effort, which is why accuracy must be read alongside false-valid and false-invalid rates.

Frequently asked questions

What is ground truth in email verification?

Ground truth is the known real-world delivery outcome of an address. It usually comes from actual send results such as bounce codes, controlled test mailboxes or recent engagement. Accuracy is only meaningful when measured against this independent reference, not against the vendor's own labels.

What is a false valid?

A false valid is an address the verifier labeled Valid that then bounces when you send to it. It is the error that damages sender reputation most directly, because it puts a bad address into your sending segment. Measure it as a share of all addresses labeled Valid.

What is a false invalid?

A false invalid is an address labeled Invalid that would actually have delivered. It causes lost reach because you suppress working contacts. Measure it as a share of all addresses that truly deliver, using a reliable ground truth sample.

Why can a 99% accuracy score be misleading?

Accuracy depends on the base rate of deliverable addresses in the list. If 98% of a list is deliverable, labeling everything Valid already scores 98%. A high headline figure therefore says little without the sample composition and the breakdown of error types.

How big a sample do I need to test email verifier accuracy?

A random sample of 1,000 addresses gives roughly plus or minus 0.6 percentage points around a 99% result, based on standard binomial margin math. Smaller subgroups, such as catch-all domains, need their own samples of similar size to produce reliable estimates.

Should Risky addresses count against accuracy?

Risky results cannot be confirmed as deliverable or undeliverable, so they are best reported as a separate share of the list. Excluding them from the accuracy ratio while disclosing the Risky percentage keeps the rule visible. Ask vendors how they score Risky before comparing headline numbers.

How do I test an email verifier on my own list?

Draw a stratified random sample across domains and risk types, verify it, then send to the addresses and record bounce codes. Calculate valid precision, false-invalid rate and unknown rate separately. Hard bounces are the most reliable signal of a real undeliverable address.

Do SMTP checks prove an email address exists?

Not reliably. An SMTP probe reads the server's response to a recipient query, but many servers accept all addresses or hide mailbox status. Treat SMTP results as evidence to weigh, and confirm with send outcomes where accuracy matters.

What is an acceptable accuracy for an email verifier?

There is no universal threshold, because acceptable performance depends on your list, risk tolerance and use case. Most teams focus less on the headline percentage and more on the false-valid rate, since that error directly affects bounce rates and sender reputation.

Cut Bounces and Protect Your Sender Reputation With Unlimited Email Verification

Every invalid address you send to hurts your sender reputation and your inbox placement. Verifying your list before each send means fewer bounces, healthier deliverability, and a predictable cost. With Unlimited Verifier, standard verification is unlimited on every plan, with no per-email fees.

Unlimited Verifier checks syntax, domain, MX records and the SMTP mailbox. It also detects catch-all domains and flags disposable and role-based addresses. Export a CSV from your email platform or CRM, upload and verify the list, then re-import the clean results. You can also automate checks with the email verification API.

Free Pro
Price $0 $29/month ($25/month billed annually)
Lists 25 a month, up to 150 emails each Unlimited, up to 25,000 emails each
Deep verifications 750 a month 25,000 a month
API access No Yes (100,000 emails a month)
Result retention 24 hours 30 days

Why teams choose Unlimited Verifier

Unlimited Verifier states 99.5% accuracy for deep verification.

Create a free account and verify your next list before you hit send.

Want to test a single address first? Try the free email checker.