Article
How Email Verifiers Measure Accuracy (and What Their Numbers Hide)
By Unlimited Verifier Team ·

Summary
Email verifiers measure accuracy by comparing their Valid, Risky and Invalid labels against ground truth, addresses whose real delivery outcome is already known. Accuracy alone hides two costly errors: false valids that bounce and false invalids that discard working contacts.Email verifiers measure accuracy by checking their Valid, Risky and Invalid labels against ground truth, addresses whose real delivery outcome is already known, and reporting the share labeled correctly: 97 correct calls out of every 100 means 97% accuracy. A headline figure such as the 99.5% that Unlimited Verifier states for deep verification only means something once you know the sample, the ground truth and the kinds of error behind it. For the wider picture, see our pillar on email verifier accuracy.
Quick answer
- Accuracy is correct labels divided by all labels. It hides the two errors that matter: a false valid (labeled Valid, then bounces) and a false invalid (labeled Invalid, though it would have delivered).
- Ground truth comes from real send outcomes, bounce codes, controlled test mailboxes or recent engagement. A dataset the vendor alone controls is hard to audit.
- Base rates decide how impressive accuracy looks. On a list that is 88% deliverable, labeling every address Valid already scores 88%.
- A random sample of 1,000 addresses gives roughly ±0.6 points of margin around a 99% result. Subgroups such as catch-all domains need their own, larger samples.
- Test on your own list: stratify the sample, record bounce codes from a real send, and report valid precision, false-invalid rate and unknown rate separately.
What does "accuracy" mean for an email verifier?
For an email verifier, accuracy is the number of addresses labeled correctly divided by the total number labeled. That single ratio conceals the two mistakes that cost you money. A false valid puts a bad address into your sending segment and produces a bounce. A false invalid removes a working contact you could have mailed.
| Verifier label | Address really delivers | Address really bounces |
|---|---|---|
| Valid | Correct | False valid: a bounce that cleaning should have caught |
| Invalid | False invalid: a working contact you suppressed | Correct |
| Risky | Held back, not scored right or wrong | Held back, not scored right or wrong |
Vendors score Risky differently. Risky usually covers catch-all, disposable and role-based addresses that can't be confirmed. Some vendors count Risky as wrong, some exclude it, and some count it as right because it isn't Valid. That single choice can move a headline figure noticeably, so ask how it was handled before you compare numbers. Our recommendation is to report Risky as its own share of the list and exclude it from the accuracy ratio, so the rule is visible rather than buried.
The term "false positive" is used inconsistently in this field, so this article uses the explicit labels "false valid" and "false invalid." The false positive guide explains the naming conventions. Teams working under email verification compliance and hygiene requirements need both error types measured, not just one headline score.
Bottom line: Accuracy counts only the right calls. You need to know which wrong calls the tool made.
What can an SMTP mailbox check actually prove?
Many verifiers base their Valid or Invalid call on an SMTP probe. The tool connects to the recipient domain's mail server, introduces itself, names a sender and asks whether the recipient is accepted, without sending a message. The server's reply is the evidence. RFC 5321 defines the basic reply codes: 250 means the recipient is accepted, 550 means the mailbox is unavailable, and 4yz codes are temporary failures. RFC 3463 adds enhanced status codes, such as 5.1.1 for a bad destination mailbox.
That evidence has four limits:
- Catch-all servers accept any recipient, so a 250 reply confirms the server, not the mailbox. This is the most common surprise, and it's why catch-all detection matters as much as the probe itself.
- Greylisting and throttling return temporary codes, so a live mailbox can look unknown on the first attempt.
- Blocked port 25 and rate limits on networks and mail providers mean some probes never get a clean answer.
- Timing matters. A reply reflects the server's state at the moment of the check, and mailboxes are closed and created constantly.
So an SMTP check shows that a server would accept mail for an address at that moment. It doesn't show that a person reads the mail or that it reaches the inbox. The delivery question is separate from the accuracy question.
Bottom line: A 250 reply is strong evidence about the server and weaker evidence about the person behind the address.
How do verifiers get ground truth for an address?
Ground truth is the outcome a verifier is trying to predict. Every accuracy claim rests on how that outcome was observed, and each source has a blind spot.
| Ground-truth source | What it can prove | Blind spot | Effort for you |
|---|---|---|---|
| Vendor-controlled test mailboxes | Whether a probe matches a known mailbox state | Seed mailboxes rarely match your list's age, domains or mix | None, and usually not auditable |
| Real send outcomes (hard bounces) | Whether mail was refused on delivery | Catch-all and accepted-then-dropped mail look deliverable | Medium: needs your ESP's bounce logs |
| Bounce codes (for example, 5.1.1 vs 5.7.x) | Whether a rejection was about a missing mailbox or a policy block | Some rejections are policy blocks, not dead addresses | Low |
| Recent engagement (clicks) | That an address was live recently | Silence doesn't prove an address is dead | Low |
| Hand review of a sample | Judgment on ambiguous cases | Slow, subjective and small | High |
The most useful ground truth for a sender is your own send results, read through bounce codes. A 5.1.1 rejection means the mailbox doesn't exist. A 5.7.x code usually means a policy or reputation block, which may not reflect a dead address at all. Treating the two the same inflates your false-valid count and makes a verifier look worse than it is. Recent clicks are a useful second check: an address that was clicked in the last 90 days was very likely live when it was clicked, so Invalid addresses with recent clicks are evidence of false invalids. Clicks can't reveal every false invalid, though, because a live reader who never clicks looks identical to a dead address.
Bottom line: Judge a verifier's accuracy against outcomes you can observe, and read the bounce code, not just the bounce count.
Why does a single accuracy percentage mislead?
Accuracy depends on the base rate, the share of your list that is genuinely deliverable. Consider an illustrative 100,000-row list in which 88,000 addresses are deliverable and 12,000 are not. A tool that labeled every row Valid would be right 88% of the time without checking a single address.
Now compare two hypothetical verifiers with near-identical headline scores:
- Tool X misses 2,400 undeliverable addresses (labeled Valid) and wrongly flags 500 good ones as Invalid. Correct labels: 9,600 + 87,500 = 97,100, so 97.1% accuracy.
- Tool Y misses 300 undeliverable addresses and wrongly flags 2,700 good ones. Correct labels: 11,700 + 85,300 = 97,000, so 97.0% accuracy.
The scores are nearly identical, but the outcomes differ. Tool X's Valid segment holds 89,900 addresses, of which 2,400 (about 2.7%) will bounce. Tool Y's Valid segment holds 85,600 addresses, of which 300 (about 0.4%) will bounce, but it discards 2,700 real contacts, which is 3.1% of the deliverable list. If bounce reputation is your main concern, Tool Y is safer. If reach matters more, Tool X keeps more of your audience. The headline score doesn't tell you which trade-off you're making.
Bottom line: Two tools with the same accuracy can send you very different risks, so measure the error mix.
Which metrics should you compare instead?
Six metrics give you a usable picture. The first two matter most for sending, and the last two show how much of the headline figure is real.
| Metric | Plain-English definition | Why it matters | Watch for |
|---|---|---|---|
| Valid precision | Share of Valid-labeled addresses that actually deliver | Predicts bounces from the segment you'll mail | Needs real send outcomes |
| False-invalid rate | Share of Invalid-labeled addresses that were live | Measures contacts you lose | Usually estimated with engagement evidence |
| Invalid recall | Share of all undeliverable addresses caught as Invalid | Measures bad data left in your Valid segment | Requires outcomes for the whole list |
| Risky share | Risky ÷ total | Sets how much of the list you hold back | A high share means less mailable data |
| Unknown rate | No definitive answer ÷ total | Shows how much accuracy rests on decided addresses | Often excluded from the headline |
| Repeatability | Same result when re-tested days apart | Detects unstable probes | Requires two test runs |
The unknown rate changes the denominator. Suppose a tool reports 98% accuracy but returns 5% unknowns and excludes them. Even if every unknown were wrong, the honest floor across the whole list would be 0.98 × 0.95 = 93.1%. Ask for the figure with unknowns included. As a working threshold, we treat an unknown rate above 5% as a reason to ask questions before trusting a result, and the unknown rate guide covers how to judge that share in more depth.
Bottom line: Valid precision and false-invalid rate tell you about sending and lost reach. Unknown rate and repeatability tell you whether the headline can be trusted.
How many addresses do you need to trust an accuracy number?
Accuracy measured on a sample carries sampling error. At 95% confidence, the margin of error for a proportion is ±1.96 × √(p(1−p)/n), where p is the observed accuracy and n is the sample size. The results:
- 1,000 addresses at 99% accuracy: ±0.6 points.
- 1,000 addresses at 99.5% accuracy: ±0.4 points.
- 500 addresses at 99% accuracy: ±0.9 points.
- Pinning 99% within ±1 point needs about 381 addresses. Within ±0.5 points needs about 1,522.
- A subgroup of 200 catch-all addresses at 80% accuracy carries a margin of ±5.5 points.
Our recommendation is a stratified sample of at least 1,500 rows. That is enough to separate 99% from 99.5% in the main population, and it is the smallest sample that keeps a serious comparison honest.
The last example matters most. Overall samples can look precise while the subgroups that carry the most risk, such as catch-all domains, remain unreliable. The margins also assume a random or stratified sample. A sample that over-represents clean corporate addresses will flatter the result however large it is.
Bottom line: A few hundred addresses can detect large gaps, but you need thousands to separate 99% from 99.5%, and subgroups need their own samples.
Why do catch-all and unknown results drag accuracy down?
Two categories sit outside the clean right-or-wrong framing, and both distort the headline.
Catch-all domains accept every address at the SMTP stage, so the probe can't separate a real mailbox from a fake one. Accuracy on these addresses only shows up through real send outcomes, and even those are noisy. Catch-alls are common in B2B lists, so they matter more there than in consumer data. For detail, see how accurate verifiers are on catch-all domains. Unlimited Verifier flags catch-all domains so you can see which addresses sit behind them, and the catch-all email verification page explains how that handling works.
Unknown results are addresses where the tool got no definitive answer, often because of timeouts or greylisting. How a tool counts them changes the score. Excluding them inflates accuracy, and including them as errors lowers it. Report both figures, and treat the gap between them as the uncertainty you're carrying.
Bottom line: Ask for accuracy on catch-all and unknown addresses separately, and include unknowns in the denominator.
Why do two verifiers disagree on the same address?
Disagreement is normal, and it usually comes from one of five causes:
- Different probes. Timing, sending IP reputation and retry logic affect what the server reveals. A probe may get a clearer answer from one network than another.
- Different catch-all rules. Tools handle accept-all servers differently, so the same address can be Valid in one and Risky in another.
- Different label rules. Thresholds for Risky, and handling of role and disposable addresses, vary.
- Different ground truth. Each vendor calibrates on its own data and definitions.
- Changing data. Mailboxes are created and closed constantly, and verifiers update their logic.
The practical move is to run both tools on the same list and focus on the disagreement set. If two tools agree on 92% of 10,000 addresses, the remaining 800 addresses decide the comparison, and those are the ones to test with real send outcomes. Consensus is useful too. Addresses that two independent tools both mark Invalid are safe to suppress, and addresses where they split are where a small test send earns its keep. For a deeper look at one common pair, read why ZeroBounce and NeverBounce can disagree on the same email. When you're shortlisting best email verification tools, use the same disagreement test rather than the vendor's headline.
Bottom line: Disagreements show you where the tools' accuracy actually differs, so test those addresses first.
How do you test a verifier's accuracy on your own list?
Use this seven-step framework. Each step produces evidence you can show a colleague or a vendor.
- Set your rules first. Write down how Risky and unknown addresses will be scored, and which ground truth you'll use: real bounce outcomes, bounce codes and recent clicks.
- Build a stratified sample of 1,500 rows, split across the domain types you actually mail. Include corporate domains, Gmail, Outlook and Microsoft domains, Yahoo, catch-all domains, and disposable or role-based addresses. For ecommerce and SaaS lists, the email verification guide for ecommerce and SaaS covers which strata matter most. Add a last-engaged date to every row. The Gmail vs Outlook accuracy comparison shows why those two providers are often tested separately.
- Freeze and export the sample on one day, so every tool tests identical data.
- Verify with the tool under test using the check type you plan to rely on, and keep the original fields alongside the labels.
- Collect ground truth. Send only to the Valid group, most engaged first, and read bounce codes after the bounce report has settled over several days. Check Invalid addresses for recent clicks.
- Calculate the six metrics from the table above, with margins at 95% confidence.
- Re-test a subset of disagreements after a few days to measure repeatability.
Bottom line: A test is only as good as its sample and its ground truth, so design both before you run anything.
How do you run an accuracy test in Unlimited Verifier?
Follow these steps to run the framework above on a 1,500-row sample.
- Export your sample from your ESP as a CSV. Include the email column and a last-engaged date.
- Sign up for an account. The Free plan covers 25 lists a month of up to 150 emails each, with 750 deep verifications a month, but it has no API access and keeps results for only 24 hours. A 1,500-row test needs more deep verifications than Free includes, so the Pro plan is the practical choice at $29 a month, or $25 a month billed annually. Pro handles lists of up to 25,000 emails and includes 25,000 deep verifications a month. For how plans compare, see email verification pricing.
- Upload and verify a list. CSV and Excel files up to 10MB are accepted, and the email column is detected automatically.
- Run the deep check. It covers syntax, domain, MX records and the SMTP mailbox, and it flags catch-all, disposable and role-based addresses.
- Count the three buckets. Note the Valid, Risky and Invalid totals, and record catch-all results separately so they don't blur into Risky.
- Export the results as CSV, Excel or JSON before the results window closes. Pro results are kept 30 days and Free results only 24 hours, so save the file on the day you run the test.
- Send to the Valid segment only, most-engaged first, and collect bounce codes over several days. Import the Invalid addresses as suppressions.
- Re-check inconsistent addresses one at a time with the free email checker, particularly any row that looks wrong against its engagement history.
- Script the test later. Once the method is proven, the same checks can run inside your email verification API and automation setup, using the Email verification API. Pro includes API access for 100,000 emails a month, and each bulk request accepts up to 10,000 emails. For signup forms, see real-time email validation.
Bottom line: The whole test takes one Pro month and a few days of observation, and it produces numbers you can defend.
What does an accuracy audit look like with real numbers?
The numbers below are illustrative. The plan limits are real.
Setup: A 60,000-contact list. You draw a stratified sample of 1,000 addresses. On Pro, deep verification uses 1,000 of the 25,000 monthly deep allowance, leaving 24,000.
Verification results (hypothetical): 790 Valid, 150 Risky, 60 Invalid, and no unknowns in this run.
Ground truth:
- You send to the 790 Valid addresses. After the bounce report settles, 7 hard bounces appear.
- Valid precision = 783 ÷ 790 = 99.1%. The false-valid rate is 0.9%.
- The margin at 95% confidence is ±0.7 points, so the true precision sits between about 98.4% and 99.8%. In false-valid terms, that is between 0.2% and 1.6%.
- Of the 60 Invalid addresses, 4 were clicked in the last 90 days. That proves at least 4 false invalids, or 6.7% of the Invalid bucket, as a lower bound.
- The 150 Risky addresses (15%) are held back and not scored.
Scaling to the full 60,000:
- Expected Valid segment: 60,000 × 79% = 47,400.
- Expected bounces at a 0.9% false-valid rate: about 425 (47,400 × 0.009).
- At the edges of the margin, the bounce count could range from roughly 95 to 760 (0.2% and 1.6% of 47,400). That range is too wide to plan a send precisely.
- Raising the sample to 1,500 narrows the margin to about ±0.5 points, which is the right step before you commit to a volume plan.
Cost: One Pro month at $29, plus the time to export, send and count bounces.
Bottom line: The sample tells you the direction and rough size of the error. Size the sample to the decision you need to make.
How should you read a vendor's accuracy claim?
Use the questions below before you accept any headline number.
| Question to ask | Good answer | Red flag |
|---|---|---|
| What is the ground truth? | A named method, such as send outcomes or test mailboxes, with dates | "Proprietary dataset" with no description |
| How are Risky and unknown addresses counted? | Reported separately, with the rule stated | Silently excluded from the denominator |
| What was the sample size and mix? | A number, a domain breakdown and a test date | Not disclosed |
| Which check type was measured? | Deep and standard results reported separately | One blended figure |
| Has anyone else tested it? | An independent test with a published method | Self-reported only |
| Can I reproduce it on my list? | A free trial or sample export you can test | Nothing to test before paying |
Unlimited Verifier states 99.5% accuracy for deep verification. That is the company's own figure. This article hasn't seen the methodology behind it, so treat it as a claim to test with the framework above. For context on how to weigh such claims, read whether vendor accuracy claims are trustworthy, whether 99% accuracy is real and how accurate email verification really is. For a third-party reference point across tools, the independent accuracy benchmark is worth reading alongside your own test.
Bottom line: A credible accuracy claim names its ground truth, its sample and its handling of Risky addresses. If it doesn't, it's a marketing figure.
What are the common mistakes when measuring verifier accuracy?
- Using overall accuracy as the only metric. It hides the error mix, and on a mostly deliverable list it flatters weak tools.
- Testing on a pre-cleaned list. If the sample excludes the messy, older rows, the false-valid rate will look artificially low.
- Dropping Risky and unknown addresses silently. Report them separately, every time.
- Counting bounces too early. Bounce reports can lag, so wait several days before scoring.
- Re-verifying with the same tool and calling it validation. The same tool shares its own blind spots, so you need outside ground truth.
- Comparing deep and standard checks as if they were equal. Compare like with like.
- Forgetting the results window. Free results are kept for 24 hours, so export before they expire.
- Treating Valid as inbox placement. Valid means the server accepted the address at the time of the check. It doesn't guarantee delivery to the inbox.
Once you know your accuracy, set a schedule for upkeep using email list maintenance best practices, and use how to choose an email verifier to decide which tool fits your volume.
Bottom line: Most measurement errors come from skipping a step, not from the maths.
How we compared (and limitations)
This article is based on public documentation: RFC 5321 for SMTP reply codes, RFC 3463 for enhanced status codes, and Google's email sender guidelines for what bulk senders are expected to maintain.
We have not run controlled accuracy tests of any vendor, and we have not benchmarked any verifier. The worked examples are arithmetic on hypothetical numbers, and the margins of error are standard calculations, not measured results. The 99.5% deep-verification figure is Unlimited Verifier's stated number and hasn't been independently verified here. Plan details are as published and may change, and the thresholds in this article should be rechecked against current documentation before you rely on them.
Base-rate worked example
List of 10,000 addresses, 88% truly deliverable (8,800 good, 1,200 bad).
| Verifier behavior | Correct calls | Accuracy |
|---|---|---|
| Labels everything Valid | 8,800 | 88% |
| Catches 1,100 bad, misses 100 | 9,700 | 97% |
| Catches 1,100 bad, flags 300 good Invalid | 9,500 | 95% |
The first row scores 88% with zero verification effort, which is why accuracy must be read alongside false-valid and false-invalid rates.
Frequently asked questions
What is ground truth in email verification?
Ground truth is the known real-world delivery outcome of an address. It usually comes from actual send results such as bounce codes, controlled test mailboxes or recent engagement. Accuracy is only meaningful when measured against this independent reference, not against the vendor's own labels.
What is a false valid?
A false valid is an address the verifier labeled Valid that then bounces when you send to it. It is the error that damages sender reputation most directly, because it puts a bad address into your sending segment. Measure it as a share of all addresses labeled Valid.
What is a false invalid?
A false invalid is an address labeled Invalid that would actually have delivered. It causes lost reach because you suppress working contacts. Measure it as a share of all addresses that truly deliver, using a reliable ground truth sample.
Why can a 99% accuracy score be misleading?
Accuracy depends on the base rate of deliverable addresses in the list. If 98% of a list is deliverable, labeling everything Valid already scores 98%. A high headline figure therefore says little without the sample composition and the breakdown of error types.
How big a sample do I need to test email verifier accuracy?
A random sample of 1,000 addresses gives roughly plus or minus 0.6 percentage points around a 99% result, based on standard binomial margin math. Smaller subgroups, such as catch-all domains, need their own samples of similar size to produce reliable estimates.
Should Risky addresses count against accuracy?
Risky results cannot be confirmed as deliverable or undeliverable, so they are best reported as a separate share of the list. Excluding them from the accuracy ratio while disclosing the Risky percentage keeps the rule visible. Ask vendors how they score Risky before comparing headline numbers.
How do I test an email verifier on my own list?
Draw a stratified random sample across domains and risk types, verify it, then send to the addresses and record bounce codes. Calculate valid precision, false-invalid rate and unknown rate separately. Hard bounces are the most reliable signal of a real undeliverable address.
Do SMTP checks prove an email address exists?
Not reliably. An SMTP probe reads the server's response to a recipient query, but many servers accept all addresses or hide mailbox status. Treat SMTP results as evidence to weigh, and confirm with send outcomes where accuracy matters.
What is an acceptable accuracy for an email verifier?
There is no universal threshold, because acceptable performance depends on your list, risk tolerance and use case. Most teams focus less on the headline percentage and more on the false-valid rate, since that error directly affects bounce rates and sender reputation.
Cut Bounces and Protect Your Sender Reputation With Unlimited Email Verification
Every invalid address you send to hurts your sender reputation and your inbox placement. Verifying your list before each send means fewer bounces, healthier deliverability, and a predictable cost. With Unlimited Verifier, standard verification is unlimited on every plan, with no per-email fees.
Unlimited Verifier checks syntax, domain, MX records and the SMTP mailbox. It also detects catch-all domains and flags disposable and role-based addresses. Export a CSV from your email platform or CRM, upload and verify the list, then re-import the clean results. You can also automate checks with the email verification API.
| Free | Pro | |
|---|---|---|
| Price | $0 | $29/month ($25/month billed annually) |
| Lists | 25 a month, up to 150 emails each | Unlimited, up to 25,000 emails each |
| Deep verifications | 750 a month | 25,000 a month |
| API access | No | Yes (100,000 emails a month) |
| Result retention | 24 hours | 30 days |
Why teams choose Unlimited Verifier
- Unlimited standard verification on all plans, including Free, with no per-email fees.
- Catch-all detection so you know which domains accept every address.
- Clear Valid / Risky / Invalid results, exportable as CSV, Excel or JSON.
- An open-source verification engine you can inspect on GitHub.
Unlimited Verifier states 99.5% accuracy for deep verification.
Create a free account and verify your next list before you hit send.
Want to test a single address first? Try the free email checker.