Test any OCR tool on your own documents in 15 minutes

Why not just trust a published accuracy number?

Because nobody else’s test used your pages. OCR results swing with the script, the print size, how the page was captured and how it is laid out. A number measured on clean typed English tells you little about faded receipts, phone photos of a textbook, or an Urdu newspaper. Most published figures are also scored by the vendor that sells the tool.

The good news is that a fair test of your own is quick. You do not need software or statistics. You need a handful of pages that look like your real work, and the patience to type out a few lines of each by hand.

How do I run the test?

1. Pick five pages like yours. Three typical pages, one bad photo (dim, tilted or blurred), and one with your hardest feature: handwriting, a table, small print or a non-Latin script such as Arabic or Hindi.
2. Make an answer key. On each page, type out one paragraph exactly as printed, plus every name, date and amount on the page. This is the correct text you will compare against.
3. Run each tool on the same files. Do not crop, straighten or edit the pages for one tool and not another. Save each tool’s output as plain text.
4. Count the errors that matter. For each page, count wrong or missing names, dates and amounts. Then count other wrong words in the paragraph.
5. For bank statements, add the balance check. Opening balance plus money in minus money out should equal the closing balance (see the 2-minute check).
6. Pick the tool with the fewest errors that matter, not the one with the nicest claims. Test again when your documents change.

What does a scorecard look like?

Keep it to one row per page and two columns of counts per tool. Write it on paper or in a spreadsheet. The numbers below are made up to show the layout.

How should I read the results?

Look at key errors first. A tool that misreads one amount in five pages is worse for an accounts clerk than a tool that drops ten commas, even though the second tool has more errors in total. Then look at where the errors happen: if one tool fails only on your bad photo, better photos may fix it; if it fails on every Urdu page, it does not read Urdu well enough for you.

Two tools within one or two errors of each other are a tie on five pages. Break the tie on what else you need: the output you get back (plain text, Word, a searchable PDF), how files are handled, and the price for your volume.

What should I keep in mind when testing?

• Same files for every tool. Re-saving or compressing a photo for one tool changes the test.
• Watch for paraphrasing. Chat assistants can return text that reads well but is not what the page says. Compare word for word.
• Check accents and special letters (é, ñ, ü) and right-to-left order if you work in Arabic, Urdu, Farsi or Hebrew.
• Note the reading order. On two-column pages, text from both columns can be interleaved even when every word is right.

Technical appendix: character error rate

If you need a single number for a larger test, researchers use character error rate (CER): the number of character insertions, deletions and substitutions needed to turn the tool’s output into the reference text, divided by the length of the reference. Word error rate is the same idea on whole words.

Normalise both texts the same way before scoring: one Unicode form, collapsed whitespace and the same punctuation rules. Score each script or document type separately, so an easy group cannot hide a weak one, and report how many pages each figure covers. Even then, the key-error count above is what tells you whether a tool is safe for names, dates and money.