OCR and document parsing
OCR turns an image of text into text; document parsing goes further and rebuilds tables, formulas, charts and reading order, usually as Markdown.
Classic OCR finds characters in an image and returns the text. Modern open OCR models are small vision-language models that read a whole page and write it back as Markdown, HTML or JSON, keeping headings, tables and formulas in place. Several also return where each block sits on the page, extract fields from forms or translate text inside images.
Most of these models have between 0.8 and 5 billion parameters, so they run on one consumer GPU, and they are often used to prepare PDFs for RAG. Benchmark scores are reported by the developers, often on different versions of the same benchmark, so test a model on your own scans before choosing, especially handwriting, photographed pages and less common languages.
Related terms
- Vision-language model (VLM) A vision-language model reads images, and often video, together with text, so you can ask questions about a picture, a screenshot or a document page.
- Retrieval-augmented generation (RAG) Retrieval-augmented generation lets a model answer from your own documents by retrieving the relevant passages and adding them to the prompt.
- OmniDocBench OmniDocBench is a benchmark for document parsing: it scores how well a model turns PDF pages of many types into text, tables and formulas.
- Benchmark A benchmark is a standardized test used to compare models, such as a set of maths, coding or knowledge questions; scores are useful but easy to over-read.