Article

How to Test an Email Verifier's Accuracy Yourself: A Step-by-Step Guide

By Unlimited Verifier Team ·

Flow diagram showing a 2,000-address email verifier test: label ground truth, run each tool, then score false positives and false negatives separately.

Summary

To test an email verifier's accuracy yourself, build a labeled sample of about 2,000 addresses, including at least 750 known-bad ones, run the same file through each tool, and score false positives and false negatives separately. This takes about an afternoon and measures your own data rather than a vendor's headline figure.

To test an email verifier's accuracy yourself, build a labeled sample of roughly 2,000 addresses, at least 750 of them known-bad, run the same file through each tool, and score false positives and false negatives separately instead of trusting a single headline percentage. The whole test takes about an afternoon and measures the addresses in your data, not someone else's.

Our email verifier accuracy pillar covers the wider picture. This page is the hands-on test, from ground truth to scoring, with a worked example you can copy.

Quick answer

Why can't you trust a single vendor accuracy number?

A single accuracy percentage only means something if you know the sample behind it, and vendors rarely publish theirs. Accuracy also hides the cost of each error type.

Take a list where two-thirds of addresses are good. A tool that marks everything Valid scores 66.7% on that list while passing every bad address straight through. The share of good addresses in your data sets a floor that a lazy tool can clear. What matters is the error rate inside each class.

Vendors describe their methods differently, and the same address can get different labels from different tools. Why ZeroBounce and NeverBounce disagree on the same email explains the mechanics. Unlimited Verifier states 99.5% accuracy for deep verification. That is the company's own figure, not an independent benchmark, so treat it as a claim to check against your data. For how to read claims like this, see are vendor accuracy claims for email verifiers trustworthy. For the broader question of how accurate verification is in general, see how accurate email verification really is.

If you are shortlisting best email verification tools, run this test on at least two of them with the same file. Comparing their marketing pages tells you very little.

Bottom line: A headline accuracy figure can hide a high false-positive rate, so measure each error type within its own class.

What ground truth do you need before you test?

Ground truth is a set of addresses whose real status you know with high confidence. Three sources cover most needs, and each has a blind spot you should write down before you start.

Ground-truth source Used as Strength Blind spot
Hard bounces with SMTP 5.1.1 in the last 90 days, from non-catch-all domains Known-bad The receiving server said the mailbox does not exist Occasionally a temporary fault gets logged as a permanent failure
Clicks or replies in the last 90 days, with no bounce and a documented source Known-good A real person interacted with it Proves the address worked then, not on test day
Mailboxes you control on a domain that rejects unknown users Controlled good and bad You know every answer Small, and not representative of real list diversity

Why these sources. Code 5.1.1 means "bad destination mailbox address" in the enhanced status codes defined in RFC 3463, which extend the base SMTP protocol in RFC 5321. A recent 5.1.1 is a strong signal of a dead address. Use a 90-day window because recent bounces describe current mailboxes, while older ones may refer to addresses that have since been reissued.

For known-good, avoid opens. Apple's Mail Privacy Protection, introduced in 2021, loads remote content for many Apple Mail users, which inflates open counts. Clicks and replies are a cleaner signal.

Archive your bounce history first. Many ESPs suppress hard-bounced addresses automatically, and the evidence can disappear from the platform. Export the bounce log with timestamps and SMTP codes before you touch anything.

Document consent. Engagement is only half the picture. Verification tells you whether a mailbox exists, while consent tells you whether you may email it. The FTC's CAN-SPAM compliance guide sets baseline obligations for commercial email in the US. For B2B lists common in email verification for ecommerce and SaaS programs, record the source of each contact, since that is the first thing you will need if the list is ever reviewed. For how verification and consent fit together, see email verification compliance and hygiene.

No bounce history? Build a seed set. Use a domain you control, turn off catch-all on its mail server, and create a few dozen real mailboxes for known-good. Generate random local parts that don't exist for known-bad. Keep seed mailboxes on domains you are authorized to use, and never send mail to real people to establish ground truth. A seed set is small and unrepresentative, so use it as a sanity check alongside your main sample, not as the main score.

Bottom line: Known-bad comes from recent 5.1.1 bounces and known-good from recent clicks or replies. Archive both before your ESP suppresses anything.

How many addresses should the test sample include?

Use the standard sample-size formula for a proportion, n = 1.96² × p(1−p) ÷ e², where p is the error rate you expect and e is the margin of error you can accept. The answer depends heavily on p. Most sample-size calculators default to p = 0.5, which gives large numbers that don't fit verification, where error rates are small.

Margin of error (95% confidence) n if the true rate is about 50% n if the true rate is about 2%
±2 points 2,401 189
±1 point 9,604 753
±0.5 points 38,416 3,011

Because the rates you care about sit near 2%, the right column applies. Suppose a tool marks 24 of 600 known-bad addresses as Valid. That is 4.0%, with a 95% interval of about ±1.6 points (2.4% to 5.6%). The interval sits entirely above 2%, so the leak is real.

Now suppose a tool marks 12 of 600 as Valid. That is 2.0%, but the interval is about ±1.1 points (0.9% to 3.1%), which straddles your ceiling. You cannot show it passes, and 600 bad addresses is not enough to say so. These intervals use the normal approximation, which is adequate at these counts. For very small numbers of errors, a Wilson interval is more accurate, and many statistics tools compute it.

The practical rule: Plan for about 750 known-bad addresses to estimate a 2% rate to ±1 point, plus about 1,200 known-good so the other metrics are stable. That brings the sample to roughly 2,000. If a tool's interval straddles the ceiling, label the result inconclusive and retest with more addresses rather than declaring a pass.

Bottom line: Size each class to the rate you are testing against. Near a 2% ceiling, that means about 750 known-bad addresses, not a generic 385.

Which metrics reveal a verifier's real accuracy?

Put every result in a confusion matrix with two axes: your truth label and the tool's output. Five metrics do most of the work, and the table below shows how to compute each. For a deeper look at how verifiers arrive at these outputs, see how email verifiers measure accuracy.

Metric Formula What it tells you Suggested target
False-positive rate Known-bad marked Valid ÷ all known-bad Dead addresses you would mail as if valid At or below 2%
Bad caught Known-bad marked Invalid ÷ all known-bad Coverage of dead addresses Compare across tools
Valid precision Known-good marked Valid ÷ all known addresses marked Valid Whether a Valid label can be trusted At or above 97%
False-negative rate Known-good marked Invalid ÷ all known-good Reach lost to over-filtering Track alongside false positives
Good held back Known-good marked Risky or Invalid ÷ all known-good Real contacts you hold back Track

The false-positive rate deserves the most weight. A false positive is an address the tool calls Valid that then bounces, and repeated bounces are what push mailbox providers and ESPs to act. A false negative costs you reach, but the loss is recoverable, because a wrongly filtered address can be re-checked later. For the trade-off in depth, see false positives vs false negatives in email verification.

Track the unknown rate too. A tool that returns many inconclusive answers pushes the work back onto you. What is a good unknown rate for an email verifier covers that threshold.

Bottom line: Score the false-positive rate first, then check how many good addresses each tool holds back.

What steps make the test fair?

Design the test before you upload anything. Most unfair comparisons come from decisions made after the first results arrive.

  1. Define the decision. Are you choosing a tool, or checking whether your current tool is safe for Valid-only sends? The answer sets how strict the ceiling needs to be. If you are choosing, how to choose an email verifier helps you set the criteria first.
  2. Archive bounce logs and engagement data before any suppression runs.
  3. Build a master sheet with columns for ID, email, truth label (Good, Bad or Catch-all), source, provider group and date. Keep it offline.
  4. Stratify by provider. Group addresses into Gmail, Microsoft (Outlook, Hotmail, Live), Yahoo, other corporate domains and catch-all domains. Errors can cluster by provider, and a blended score hides that. For the pattern to watch, see email verifier accuracy on Gmail vs Outlook addresses.
  5. Strip the labels from the upload file. Upload only an ID and an email column, shuffle the row order, and use a neutral filename. The labels are for scoring, not for the tool. The bias you must guard against is your own expectation, which creeps in when you judge ambiguous results by hand.
  6. Fix the mode. If a tool offers more than one verification mode, as Unlimited Verifier does with standard and deep verification, test the mode you will run in production.
  7. Run every tool on the same file on the same day. Mailbox behavior and greylisting change over time, so same-day runs keep the comparison fair.
  8. Write the mapping before you look at results. Map each tool's output to Valid, Risky and Invalid using the rules in the mapping section below.
  9. Score the confusion matrix and read the disagreements by hand, grouped by provider.
  10. Retest a 10% subset after 7–14 days. Count how many addresses changed category. That change rate is your consistency score.

Bottom line: Strip the labels, fix the mode, run everything on the same day, and write the scoring rules before you see a result.

How do you run the test in a spreadsheet?

This tutorial needs a spreadsheet and a browser. You do not need to write code.

  1. Build your master sheet with the columns from step 3 above.
  2. Create the upload file. Copy only the ID and email columns from the master sheet, shuffle the rows, and save as CSV.
  3. Create a free account. The Free plan allows lists of up to 150 emails, with 25 lists a month. Split the 2,000-row file into 14 lists (2,000 ÷ 150 = 13.3, rounded up), keeping the ID column in every file. Note the budget: 14 lists for one tool leaves 11 for the month, so a second tool's 14 lists won't fit in the same month on Free.
  4. Upload and verify a list. CSV and Excel files up to 10MB are accepted, and the email column is detected automatically.
  5. Choose the mode you'll use in production. Standard verification is unlimited on every plan, including Free.
  6. Export the results as CSV immediately. Free-plan results are kept for only 24 hours, so don't leave the export for tomorrow.
  7. Join the results to your master sheet. If the export drops the ID column, match on the email address. In a helper column, use =LOWER(TRIM(B2)) on both sheets, then bring the truth label across with INDEX/MATCH or VLOOKUP.
  8. Build the confusion counts with formulas such as =COUNTIFS(Truth,"Bad",Result,"Valid"). Repeat for each cell in the matrix.
  9. Filter the disagreements. Pull every row where Truth is Bad and Result is Valid. Read them, note the provider, and look for a pattern.
  10. Repeat for each other tool with the same input file and the same mode.

If you want to script the retest, the email verification API accepts single checks and batches of up to 10,000 emails per request. API access is included on the Pro plan (100,000 emails a month), not on Free. You can create a key on the Email verification API page. For a single disputed address, the free email checker gives a quick second look.

Bottom line: A spreadsheet, a few COUNTIFS formulas and one exported CSV per tool are enough to produce a defensible score.

How do you score results that don't fit Valid, Risky and Invalid?

Not every tool returns three buckets, and the edge cases decide your score. Write the mapping down before you score.

Result type Score as Reasoning
Syntax error, no MX record, or mailbox rejected (5.1.1) Invalid Mail cannot be delivered
Mailbox explicitly accepted Valid The strongest positive signal available
Catch-all (domain accepts any address) Reported separately The tool cannot confirm the individual mailbox
Unknown, timeout or no conclusive answer Risky No evidence of validity
Disposable Risky, or Invalid if your policy bans them Deliverable, but low value
Role-based (info@, sales@) Risky Often deliverable, with low engagement and higher complaint risk

Catch-all handling has the biggest effect on the final scores. Count catch-alls as Valid and you inflate Valid precision. Count them as Invalid and you penalize tools that are honest about uncertainty. Reporting them separately is the defensible choice. How accurate email verifiers are on catch-all domains goes deeper, and the catch-all email verification page explains how Unlimited Verifier flags these domains.

Bottom line: Write the mapping down before scoring, and keep catch-alls out of both the Valid and Invalid counts.

What does a real scoring run look like?

The numbers below are hypothetical. The arithmetic and plan limits are real.

Setup: 1,200 known-good, 600 known-bad and 200 catch-all addresses.

Results for Tool A:

Calculations:

Interpretation: The 4.0% false-positive rate exceeds the 2% ceiling, and its 95% interval (2.4% to 5.6%) sits entirely above 2%, so the failure is a real signal. Next, group the 24 false positives by provider. If most come from one provider, the problem is provider-specific, not general.

Run Tool B through the same steps and compare the same numbers. A tool with a lower false-positive rate but a higher false-negative rate is making a trade-off, not winning outright.

Retest: Take 10% of the known-good and known-bad addresses (180 rows out of 1,800) and rerun them after 14 days. If 12 change category, the change rate is 12 ÷ 180 = 6.7%. That drift is worth investigating, because an address can get a different answer on different days.

Cost arithmetic:

Bottom line: A score is only as trustworthy as the numbers behind it. A 2,000-address test with a retest fits comfortably in one month of Pro, for about $29.

What mistakes skew the results?

Bottom line: Most bad test results come from design choices made before the upload, not from the tools themselves.

How we compared (and limitations)

This guide draws on public documentation: the enhanced status codes in RFC 3463, the base SMTP specification in RFC 5321, Google's email sender guidelines, the FTC's CAN-SPAM guide, and the Amazon SES documentation. Amazon SES describes account review at a 5% bounce rate. That is a campaign-level bounce figure, not a verifier error rate, so it does not set our target. Where thresholds conflict, we recommend the stricter 2% for false positives. We have not run controlled tests against any vendor's accuracy claims, and we have not verified any tool's internal methods.

The worked example uses hypothetical results. The sample-size and confidence-interval formulas are standard statistics. The 2% ceiling, the 90-day windows, the 97% precision target and the 10% retest share are our recommendations, not figures from a regulator or standards body. Unlimited Verifier's 99.5% deep-verification accuracy is the company's own stated figure. Plan limits are those published for the product.

For a broader independent view, see the independent email verifier accuracy benchmark for 2026. Once you've chosen a tool, build a list maintenance schedule, and consider real-time validation at signup so new addresses are checked before they enter your database.

Bottom line: Treat this test as a method you can repeat on your own data, and let your numbers, not a vendor's headline, make the decision.

Test framework steps

  1. Build ground truth: hard bounces (SMTP 5.1.1, last 90 days, non-catch-all) as known-bad; recent replies or clicks as known-good.
  2. Exclude catch-all domains from both groups.
  3. Assemble about 2,000 addresses, at least 750 known-bad.
  4. Set your false-positive ceiling (e.g., 2% of known-bad) before testing.
  5. Run the same file through each verifier.
  6. Export and save results the same day.
  7. Calculate false positives = known-bad marked Valid / total known-bad.
  8. Calculate false negatives = known-good marked Invalid / total known-good.
  9. Compare tools on each error rate separately.

Frequently asked questions

How many emails do I need to test an email verifier?

Aim for about 2,000 labeled addresses, with at least 750 known-bad. That sample size lets you estimate a roughly 2% false-positive rate to within about one percentage point. Smaller samples give wide error bars, so a few hundred addresses will not separate two close tools reliably.

What is a false positive in email verification?

A false positive is a bad address that the verifier marks as Valid. It is the costly error for sender reputation because it sends mail to nonexistent or risky mailboxes. Measure it as a share of your known-bad addresses.

What is a false negative in email verification?

A false negative is a good address that the verifier marks as Invalid. It costs reach because you drop real contacts. Measure it as a share of your known-good addresses, separately from false positives.

What is ground truth for an email verifier test?

Ground truth is a set of addresses whose real status you know with high confidence. Good sources include hard bounces with SMTP code 5.1.1 from non-catch-all domains and recent replies or clicks. Each source has blind spots, so note them before testing.

Should catch-all domains be included in the test?

Usually no. Catch-all domains accept mail for any address, so you cannot confirm whether a mailbox exists. Leaving them out of both known-good and known-bad groups keeps the scoring fair.

Why not just use the vendor's accuracy percentage?

Vendors rarely publish their sample or method, and a single percentage can hide high false-positive rates. If most of your list is good, a tool that marks everything Valid still scores well. Testing each error class on your own data gives a clearer result.

How long does it take to test an email verifier?

The test takes roughly an afternoon once you have the labeled sample. Most of the time goes into building ground truth and exporting results. Save the CSV output the same day, since free-plan results may be kept for only 24 hours.

Can I compare two email verifiers with the same file?

Yes, and you should. Run the identical labeled file through each tool and compare false positive and false negative rates side by side. Using the same input removes differences caused by list composition.

What false-positive rate is acceptable?

This is a judgment call. One reasonable internal ceiling is no more than 2% of known-bad addresses marked Valid, but this is a recommendation rather than an industry standard. Set your threshold before running the test so results are not interpreted to fit a preferred tool.

Cut Bounces and Protect Your Sender Reputation With Unlimited Email Verification

Every invalid address you send to hurts your sender reputation and your inbox placement. Verifying your list before each send means fewer bounces, healthier deliverability, and a predictable cost. With Unlimited Verifier, standard verification is unlimited on every plan, with no per-email fees.

Unlimited Verifier checks syntax, domain, MX records and the SMTP mailbox. It also detects catch-all domains and flags disposable and role-based addresses. Export a CSV from your email platform or CRM, upload and verify the list, then re-import the clean results. You can also automate checks with the email verification API.

Free Pro
Price $0 $29/month ($25/month billed annually)
Lists 25 a month, up to 150 emails each Unlimited, up to 25,000 emails each
Deep verifications 750 a month 25,000 a month
API access No Yes (100,000 emails a month)
Result retention 24 hours 30 days

Why teams choose Unlimited Verifier

Unlimited Verifier states 99.5% accuracy for deep verification.

Create a free account and verify your next list before you hit send.

Want to test a single address first? Try the free email checker.