On-device OCR for a niche use case: what PGNify teaches about handwriting recognition
A handwritten chess scoresheet looks like this: pencil columns of moves, handwriting that varies from one player to another, sometimes a crossed-out entry. Betting on perfect handwriting recognition to digitize that is unrealistic, handwriting remains a poorly solved problem even with the best models. PGNify starts from a different premise: the OCR doesn't need to be perfect, it just needs to be quickly correctable.
The real lever isn't OCR accuracy#
On-device OCR (ML Kit) pre-fills a suggestion for each move, with no network connection and without sending the photo to a third-party service. That suggestion can be wrong: ambiguous handwriting, a digit confused with a letter, it happens constantly. What makes correction fast isn't the quality of that suggestion, it's what happens right after.
The legality engine does the work OCR can't#
After each OCR suggestion, a real chess engine computes the list of legal moves in the current position. That list is almost always short, somewhere between two and about thirty moves depending on how open the position is, never the full space of algebraic notation. Correcting a misread move then becomes a choice within a short list, one or two taps, rather than retyping notation on a keyboard.
OCR suggests: "Cf3" (ambiguous, illegible)
Engine computes legal moves: [Nf3, Nc3, e4, d4, ...]
User picks from the list, in 1 tap
That's a difference in kind, not just in speed: OCR alone asks the user to verify a string of characters against their memory of the game. The legality engine asks them to recognize a move within a short list, a much faster perceptual task than a proofreading task.
The illegible move: accepting failure rather than hiding it#
A move sometimes stays completely illegible, neither the OCR nor the player's memory can recover it with certainty. PGNify lets you explicitly skip that move, and the game stays exportable up to that breaking point. This design choice matters more than it looks: forcing an answer where there isn't one would produce a silently wrong PGN, which is worse than an incomplete PGN that's honest about its gaps.
What this generalizes beyond chess#
The lesson goes beyond recognizing chess moves: whenever a domain has a constraint structure that drastically narrows the space of possible answers at any given moment (legal moves, here), raw recognition accuracy matters less than the quality of the correction mechanism surrounding it. An 80%-accurate OCR paired with a list of three correct choices beats a 95%-accurate OCR paired with a free-text field to correct by hand. The problem to solve was therefore never "how to improve the OCR," it was "how to make an OCR mistake as cheap as possible to correct."
