One of our clients handles thousands of long Hebrew documents, each fifty to sixty pages. Only a few specific details matter in each document, but they are scattered throughout the pages. The challenge was not understanding one document. It was doing so reliably across a very large number of them.
The obvious solution, and why it did not work
The first approach almost everyone tries today is to send the entire document to a language model and ask it to extract the details. It works on one short document. With a sixty-page document multiplied by thousands of files, it fails in two places:
- Accuracy. The more text sent to the model, the more the relevant detail is buried in noise. The model begins to miss information and sometimes invents details that do not appear in the document. With long files, hallucinations move from a theoretical concern to an everyday problem.
- Cost. Language model pricing is based on the amount of text processed. Sending sixty pages to retrieve half a page of information, thousands of times, is simply expensive. Each request pays for fifty-nine pages of noise.
The conclusion was clear. The model did not need the whole document. It needed only the relevant section. To provide that section, we first had to extract the Hebrew text correctly from the PDF.
The hard part: extracting Hebrew from a PDF without breaking it
Hebrew PDF files are a problem in their own right. A PDF does not store text like an ordinary document. It mainly stores where each character is drawn on the page. Standard extraction returns characters in the order produced by the PDF engine, which is almost never the order a person reads them. In Hebrew, the result becomes reversed and scrambled, while numbers and dates move in the wrong direction.
We built a pipeline that corrects this precisely:
- Read by physical position. Instead of relying on the order produced by the file, we used a Python library to read the exact horizontal position of every character on the page and reorder each line as it actually appears. Word boundaries are calculated from the spaces between characters.
- Apply a bidirectional algorithm. Each line passes through a version of the Unicode bidirectional algorithm. It restores Hebrew to right-to-left order while leaving numbers, dates, and English words left to right in the correct place. Telephone numbers and amounts remain readable, and brackets do not reverse.
- Extract only the relevant sections. Once the text is clean, we extract only the passages required. Sixty pages become half a focused page.
Only that half page, rather than the complete document, continues to processing or the language model.
The results
- Accuracy improved. When the model receives only the relevant passage, it stops missing and inventing details because there is no surrounding noise in which to get lost.
- Cost fell sharply. Instead of paying to process sixty pages for every document, the client pays only for what is needed.
- The process scales. What worked for one document now works across thousands with the same reliability.
What can we learn from this story?
Sending a complete document to a language model and asking it to work things out is tempting, but for long files it costs more and reduces accuracy. Precise, focused extraction before the model is not decoration. It is what makes AI processing of long Hebrew documents reliable and possible to budget in advance.
Questions we get asked
Why is a pipeline necessary? Do today’s models not understand long documents? They do, up to a point. As a document gets longer, accuracy falls and cost rises. Focused extraction addresses both problems at once.
What about scanned documents rather than text-based PDFs? The pipeline runs optical character recognition on scanned files. Accuracy depends on scan quality, so any passage detected with low confidence is marked for review.
Does this only work for one type of document? We built it for a specific case, but the same principle, precise extraction of text and sections from a long Hebrew document, applies to contracts, insurance policies, legal documents, and any long file where only a small amount of information is needed.
The client’s identifying details and document type were removed at their request. The scale described is based on the actual work.
