← All posts

Deskew before OCR, or after?

You have a folder of scans that all lean slightly left, and a recogniser lined up to make them searchable. Two jobs, and nothing tells you which one goes first.

So you searched, and the answer came back in pieces. An SDK help page lists deskew as a pre-processing step without saying why. A forum thread says the engine handles tilt itself. The OCRmyPDF documentation has a pipeline diagram that answers the question if you already know enough to read it.

The order does matter, it matters more than most cleanup advice, and the reason is arithmetic you can do on the back of an envelope.

The short answer

Straighten first, recognise second. Every time, with one exception below.

Both halves of that are load-bearing. Straightening before recognition makes the recognition measurably better. Straightening after recognition destroys the text layer you just made, because a corrected page is redrawn as a picture — which pages lose it and which keep it has its own answer. Do it in the wrong order and you pay twice for the same work.

What two degrees does to a line of type

The interesting part is why a tilt you can barely see costs an engine anything at all. The engine is not reading the page the way you are.

Recognition starts by finding the lines. The page is binarised, the ink is grouped into connected blobs — roughly, individual letters — and the blobs are then grouped into text lines by looking at which ones sit at the same height as each other. Only once there are lines does anything try to identify a character, because a character is classified in the context of the baseline and x-height of the line it belongs to.

Now do the arithmetic. A line of 11pt type on A4 is about 160 mm long, and the baselines are about 4.5 mm apart. Tilt the page by 2° and the far end of a line sits 160 × tan 2° = 5.6 mm lower than the near end. That is more than the gap between one line and the next.

So the last word of line one is now vertically below the first word of line two. Blobs that belong to different lines occupy the same band of the page, and the grouping step has to guess. It merges two lines into one, or splits one into two, or assigns a descender to the wrong row. Everything downstream inherits that mistake: word segmentation, the dictionary check, the reading order, and on a two-column page the column detection as well.

The crossover is at about 1.6° for ordinary body text — the tilt at which the drop across a line equals the line spacing. Below that the grouping mostly survives and accuracy sags gently. Above it the failure is sudden, which is why people report OCR that was fine on one batch and garbage on the next scanned by the same machine on the same day.

Small print makes it worse in both directions: the lines are closer together, so a smaller angle breaks them.

What the engine fixes by itself, and what it leaves you

The forum answer — "the engine handles skew" — is half true, and the half that is false is the half that matters.

Tesseract fits a baseline to each text line after grouping, and that baseline can be sloped, so modest tilt is tolerated during classification. What it does not do is fix the grouping that happened before, and it does not straighten your page. The image you handed it is the image that goes into the output PDF. Recognise a crooked scan and you get a crooked scan with a text layer on it, at whatever accuracy the grouping managed.

Commercial engines and the OCRmyPDF wrapper do more: they estimate a page angle and rotate the image before recognition. That is the exception to the rule above — if your recogniser deskews, let it, and do not deskew first. Resampling a bitmap blurs it slightly, and doing it twice blurs it twice for no gain.

Our Deskew PDF is worth running first when the thing doing the recognition will not do it: a bureau that takes what you send, a phone app, a scanner's bundled software, an engine you are driving through someone else's interface. It is also worth it when you want to see the measured angle per page before anything irreversible happens, because it reports the tilt it found on each page and you can sanity-check that against what you can see.

The order that works

  1. Find out which pages are actually scans. Recognition run over a page that already has good text makes that page worse, and cleanup run over a page of real text throws its text away for nothing. OCR PDF reads every page and hands back three lists — real text, picture, blank. Working from that page list is the whole trick. Read the note below about what this tool cannot do.
  2. Rotate the sideways pages, before anything else. A page fed in the short way is not crooked, it is turned, and Rotate PDF costs you nothing to fix — it changes a number in the page rather than redrawing it. Do it first, because both the tilt measurement and the recogniser's line-finding assume text runs left to right. On a sideways page the lines run down the sheet, the measurement finds nothing to measure, and the engine reads nothing at all.
  3. Crop the junk off the edges. The black band the scanner lid leaves, the strip of the next sheet that crept in, punch holes down the margin. All of it is ink as far as the tilt measurement is concerned, and all of it is a text block as far as layout analysis is concerned. Crop PDF takes millimetres off any side. On this route the trim ends up permanent rather than merely hidden, because the straightening step redraws each corrected page from what is on show, which for once is what you want. If the band is what brought you here, it is the lid and not damage.
  4. Straighten, at the resolution the recogniser needs. Set the page range to the scanned pages from step 1, leave the angle range at 10 degrees, and change the resolution — see the warning below.
  5. Recognise, in the tool named further down, over the same page list.
  6. Compress last, if the file needs it at all.

The cleanup steps that do nothing

Two pieces of standard advice are worth skipping, because both cost something and neither buys accuracy.

Converting to greyscale first. Every engine binarises internally — Otsu's method or an adaptive variant — so it reduces your page to black and white whatever you hand it. Doing a conversion yourself first changes nothing about that decision. Our Greyscale PDF tool would actively hurt here, because it renders every page at 150 dpi, which is below the floor in the warning above. Skip it. Straightening already renders in grey.

Compressing first. This is the expensive mistake. JPEG encoding at ordinary quality puts ringing around high-contrast edges, and a page of text is nothing but high-contrast edges — the halo is faint, but the binarisation threshold does not know it is a halo, so thin strokes fatten, counters fill in and adjacent letters fuse. Downsampling compounds it. Run Compress PDF at the end instead, once the text layer exists and the pixels no longer have to be read by a machine. The straightened pages will have grown; what the redraw costs in bytes, and how much of it comes back, sits with the redraw itself.

Where the recognition actually happens

Worth stating plainly: nothing on this site adds a text layer. Our OCR tool reports which pages need recognition and hands your file back byte for byte, because the Tesseract engine is not installed here. The preparation is real; the recognition has to happen elsewhere, and the free tools are better than the paid websites anyway.

Checking that it worked

Run the page check again on the finished file. Every page that was in the picture list should now be in the text list, and if some are not, the recognition silently skipped them.

Then read a page. PDF to text pulls the words out as plain text, and a page whose lines were grouped wrongly is obvious on sight — words from two lines interleaved, or a column read straight across. Our PDF inspector does the quick version in the browser without uploading anything, giving a character count per page; a scanned A4 page of prose should come back somewhere between 1,500 and 3,000 characters, and a page reporting 300 was recognised badly rather than sparsely.

If the count is low across the board, go back to the original scan rather than to the straightened copy, raise the resolution, and try again. That is the reason to keep it.

Tools this post uses: