IP Library Granted Patent US 11,704,476
Granted Patent B2
US 11,704,476 · App. 17/372,655 · Granted Jul 18, 2023

Text line normalization systems and methods

Inventors: Florian Kuhlmann (Berlin, DE); Michael Kieweg (Berlin, DE); Saurabh Shekhar Verma (Berlin, DE)
Assignee: LEVERTON HOLDING LLC
G06F40/109G06V10/242G06V10/243G06V10/507G06V10/82G06V30/158
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,704,476
App. No.
17/372,655
Granted
Jul 18, 2023
Kind
B2
Abstract

A method for estimating text heights of text line images includes estimating a text height with a sequence recognizer. The method further includes normalizing a vertical dimension and/or position of text within a text line image based on the text height. The method may also further include calculating a feature of the text line image. In some examples, the sequence recognizer estimates the text height with a machine learning model.

Claims (72)

1. A computer-implemented method comprising:

(a) receiving a first text line image associated with a first line of text contained within a document image;

(b) calculating a first feature of the first text line image,

wherein the first feature is chosen from the group consisting of (i) for each of a plurality of vertical positions in the first text line image, a sum of pixels at the respective vertical position across multiple horizontal positions within the first text line image, (ii) a gradient of the sums of pixels for the first text line image, and (iii) a gray value distribution at a plurality of vertical positions of the first text line image;

(c) estimating a first text height of the first text line image with a sequence recognizer,

wherein the sequence recognizer estimates the first text height using a machine learning model and the first feature; and

(d) normalizing the first text line image based on the estimated first text height.

2. The method of claim 1 , further comprising:

calculating a second feature of a second text line image associated with a second line of text contained within the document image,

wherein the sequence recognizer estimates the first text height using the first feature and the second feature.

3. The method of claim 2 , wherein the second feature includes a second text height of the second text line image.

4. The method of claim 1 , wherein the first feature includes the gradient of the sums of pixels across the plurality of vertical positions.

5. The method of claim 1 , further comprising:

calculating a third feature of a plurality of text line images associated with a plurality of lines of text contained within the document image,

wherein the sequence recognizer estimates the first text height using the third feature.

6. The method of claim 5 , wherein the third feature is a text height of a majority of the plurality of text line images.

7. The method of claim 1 , wherein the sequence recognizer estimates the first text height with a machine learning model selected from the group consisting of a recurrent neural network, a convolutional neural network, a conditional random field model, a Markov model, and combinations thereof.

8. The method of claim 7 , further comprising:

receiving (i) a training text line image associated with a training line of text and (ii) a labeled text height indicating a desired text height corresponding to the training text line image;

estimating a training text height of the training text line image;

comparing the training text height with the labeled text height to create a training output comparison that identifies one or more errors in the training text height; and

updating one or more parameters of the machine learning model based on the training output comparison.

9. The method of claim 1 , further comprising:

estimating one or both of an upper bound and a lower bound of the first text height,

wherein estimating one or both of the upper bound and the lower bound further comprises predicting a probability that each of a plurality of vertical positions of the first text line image is the upper bound and/or the lower bound.

10. The method of claim 1 , wherein the first text height includes one or more height measures chosen from the group consisting of a lowercase text height measure and a capital text height measure, and combinations thereof,

wherein the lowercase text height measure is a measure of a height of one or more lowercase letters in the text, and

wherein the capital text height measure is a measure of a height of one or more capital letters in the text.

11. The method of claim 10 , further comprising:

estimating a first height measure of the first text line image;

determining that the first height measure incorrectly measures the first text height of the first text line image;

estimating a second height measure of the first text line image; and

calculating a third height measure based on the second height measure and a height measure conversion ratio.

12. The method of claim 11 , wherein determining that the first height measure incorrectly measures the first text height of the first text line image further comprises:

performing optical character recognition on the first text line image after the first text line image is normalized based on the first height measure; and

measuring that a confidence measure of the optical character recognition is below a confidence threshold, and

wherein the first height measure is a lowercase text height measure, the second height measure is a capital text height measure, and the third text height measure is a lowercase text height measure.

13. The method of claim 1 , further comprising repeating (a)-(d) on a plurality of text line images associated with the document image.

14. The method of claim 1 , wherein the first feature includes the gray value distribution at the plurality of vertical positions.

15. The method of claim 1 , wherein the first feature includes the sums of pixels at the respective vertical positions.

16. A system comprising:

a processor; and

a memory storing instructions which, when executed by the processor, cause the processor to:

(a) receive a first text line image associated with a first line of text contained within a document image;

(b) calculate a first feature of the first text line image,

wherein the first feature is chosen from the group consisting of (i) for each of a plurality of vertical positions in the first text line image, a sum of pixels at the respective vertical position across multiple horizontal positions within the first text line image, (ii) a gradient of the sums of pixels for the first text line image across the plurality of vertical positions, and (iii) a gray value distribution at the plurality of vertical positions of the first text line image;

(c) estimate a first text height of the first text line image with a sequence recognizer,

wherein the sequence recognizer estimates the first text height using the first feature and a machine learning model; and

(d) normalize the first text line image based on the first text height.

17. The system of claim 16 , wherein the memory contains further instruction which, when executed by the processor, cause the processor to:

calculate a second feature of a second text line image associated with a second line of text contained within the document image,

wherein the sequence recognizer estimates the first text height using the first feature and the second feature.

18. The system of claim 16 , wherein the memory contains further instruction which, when executed by the processor, cause the processor to:

calculate a third feature of a plurality of text line images associated with a plurality of lines of text contained within the document image,

wherein the sequence recognizer estimates the first text height using the third feature.

19. The system of claim 16 , wherein the memory contains further instruction which, when executed by the processor, cause the processor to:

receive (i) a training text line image associated with a training line of text and (ii) a labeled text height indicating a desired text height corresponding to the training text line image;

estimate a training text height of the training text line image with the sequence recognizer;

compare the training text height with the labeled text height to create a training output comparison that identifies one or more errors in the training text height; and

update one or more parameters of the machine learning model based on the training output comparison.

20. The system of claim 16 , wherein the memory contains further instruction which, when executed by the processor, cause the processor to:

estimate a first height measure of the first text line image;

determine that the first height measure incorrectly measures the first text height of the first text line image;

estimate a second height measure of the first text line image; and

calculate a third height measure based on the second height measure and a height measure conversion ratio.

21. A tangible computer readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to:

(a) receive a first text line image associated with a first line of text contained within a document image;

(b) calculate a first feature of the first text line image,

wherein the first feature is chosen from the group consisting of (i) for each of a plurality of vertical positions in the first text line image, a sum of pixels at the respective vertical position across multiple horizontal positions within the first text line image, (ii) a gradient of the sums of pixels for the first text line image across the plurality of vertical positions, and (iii) a gray value distribution at the plurality of vertical positions of the first text line image;

(c) estimate a first text height of the first text line image with a sequence recognizer,

wherein the sequence recognizer estimates the first text height using the first feature and a machine learning model; and

(d) normalize the first text line image based on the first text height.

Assignments (3)
SECURITY INTEREST Recorded Oct 2, 2025
From: LEVERTON HOLDING, LLC
To: GOLUB CAPITAL MARKETS LLC, AS COLLATERAL AGENT
Reel/Frame 072447/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2023
From: LEVERTON GMBH
To: LEVERTON HOLDING LLC
Reel/Frame 063077/0732 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2023
From: KUHLMANN, FLORIAN; KIEWEG, MICHAL; VERMA, SAURABH SHEKHAR
To: LEVERTON GMBH
Reel/Frame 063150/0306 →
Continuity (3)
Continuation 16513094 · Jul 16, 2019
Provisional Application 62700427 · Jul 19, 2018
Related Publication 20210334573A1 · Oct 28, 2021