IP Library Granted Patent US 10,372,981
Granted Patent B1
US 10,372,981 · App. 15/272,744 · Granted Aug 6, 2019

Fast identification of text intensive pages from photographs

Inventors: Alexander Pashintsev (Cupertino, CA); Boris Gorbatov (Sunnyvale, CA); Eugene Livshitz (San Mateo, CA); Vitaly Glazkov (Moscow, RU)
Assignee: EVERNOTE CORPORATION
G06K9/00456G06K9/00463G06K9/52G06T3/40G06T7/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,372,981
App. No.
15/272,744
Granted
Aug 6, 2019
Kind
B1
Abstract

Determining if a document is a text page includes partitioning the document into a plurality of cells, scaling each of the cells to a standardized number of pixels to provide a corresponding snippet for each of the cells, using a classifier to examine the snippets to determine which of the cells are classified as text and which of the cells are not classified as text, determining a volume of text for the document based on a total amount of text in the document corresponding to a sum of an amount of text in each of the cells classified as text, and determining that the document is a text page in response to the total amount exceeding a pre-determined threshold. In response to the total amount being less than the pre-determined threshold, cells not classified as text may be examined further. The classifier may be provided by training a neural net.

Claims (49)

1. A method of determining if a document is a text page, comprising:

partitioning the document into a first plurality of cells;

scaling each respective cell of the first plurality of cells to a standardized number of pixels to provide a first set of snippets, wherein the first set of snippets correspond to the first plurality of cells;

using a classifier to examine the first set of snippets to determine which of the first plurality of cells are classified as text and which of the cells are not classified as text;

determining a volume of text for the document based on a determined total amount of text in the document corresponding to a sum of an amount of text in each of the first plurality of cells classified as text;

determining whether the determined total amount of text in the document meets a pre-determined threshold;

in response to determining that the determined total amount of text in the document does not meet the pre-determined threshold and that criteria for a next partition level are met, for respective cells that are not classified as text:

a) applying further partitioning to the respective cells that are not classified as text to generate a respective further plurality of cells;

b) scaling each of the further partitioned cells to provide a respective further set of snippets that correspond to the further partitioned cells;

c) using the classifier to examine the respective further set of snippets to determine which of the respective further partitioned cells are classified as text and which of the respective further partitioned cells are not classified as text;

d) augmenting the volume of text for the document based on an updated determined total amount of text in the document corresponding to a sum of the amount of text in each of the cells of the plurality of cells and the respective further partitioned cells classified as text;

in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for a next partition level are met, repeating a)-d);

and

determining that the document is a text page in response to the determined total amount of text in the document exceeding the pre-determined threshold.

2. A method, according to claim 1 , wherein the classifier examines the respective further set of snippets in a random order.

3. A method, according to claim 1 , wherein the classifier examines the respective further set of snippets in an order that prioritizes snippets that are adjacent to snippets previously classified as text.

4. A method, according to claim 1 , further comprising:

in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for the next partition level are not met, determining that the document is a text page in response to cells that are classified as text having a satisfactory geometry.

5. A method, according to claim 4 , wherein at least some of the cells corresponding to snippets that are classified as text are aligned to form at least one text line and wherein the at least one text line is one of: horizontal or vertical.

6. A method, according to claim 1 , wherein the snippets that are not classified as text are classified as images.

7. A method, according to claim 1 , wherein the snippets that are not classified as text are classified as one of: images and unknown.

8. A method, according to claim 1 , wherein partitioning the document into the first plurality of cells includes partitioning the document into six cells.

9. A method, according to claim 1 , wherein the document is captured using a smartphone.

10. A method, according to claim 1 , wherein the classifier is provided by training a neural net using a plurality of image documents and a plurality of text pages having various formats, layouts, text sizes, ranges of word, line and paragraph spacing.

11. A method, according to claim 1 , including determining that the document is not a text page in response to determining that the augmented volume of text for the document does not meet the pre-determined threshold and that criteria for a next partition level are not met.

12. A non-transitory computer readable medium containing software that determines if a document is a text page, the software comprising:

executable code that partitions the document into a first plurality of cells;

executable code that scales each respective cell of the first plurality of cells to a standardized number of pixels to provide a first set of snippets, wherein the first set of snippets correspond to the first plurality of cells;

executable code that uses a classifier to examine the first set of snippets to determine which of the first plurality of cells are classified as text and which of the cells are not classified as text;

executable code that determines a volume of text for the document based on a determined total amount of text in the document corresponding to a sum of an amount of text in each of the first plurality of cells classified as text;

executable code that determines whether the determined total amount of text in the document meets a pre-determined threshold;

executable code that, in response to determining that the determined total amount of text in the document does not meet the pre-determined threshold and that criteria for a next partition level are met, for respective cells that are not classified as text:

a) applies further partitioning to the respective cells that are not classified as text to generate a respective further plurality of cells;

b) scales each of the further partitioned cells to provide a respective further set of snippets that correspond to the further partitioned cells;

c) uses the classifier to examine the respective further set of snippets to determine which of the respective further partitioned cells are classified as text and which of the respective further partitioned cells are not classified as text;

d) augments the volume of text for the document based on an updated determined total amount of text in the document corresponding to a sum of the amount of text in each of the cells of the plurality of cells and the respective further partitioned cells classified as text

executable code that, in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for a next partition level are met, repeats a)-d);

and

executable code that determines that the document is a text page in response to the total amount exceeding a pre-determined threshold.

13. A non-transitory computer readable medium, according to claim 12 , wherein the classifier examines the respective further set of snippets in a random order.

14. A non-transitory computer readable medium, according to claim 12 , wherein the classifier examines the respective further set of snippets in an order that prioritizes snippets that are adjacent to snippets previously classified as text.

15. A non-transitory computer readable medium, according to claim 12 , further comprising:

executable code that, in response to determining that the augmented volume of text fort the document does not meet the pre-determined threshold and that criteria for the next partition level are not met, determines that the document is a text page in response to cells that are classified as text having a satisfactory geometry.

16. A non-transitory computer readable medium, according to claim 15 , wherein at least some of the cells corresponding to snippets that are classified as text are aligned to form at least one text line and wherein the at least one text line is one of: horizontal or vertical.

17. A non-transitory computer readable medium, according to claim 12 , wherein the snippets that are not classified as text are classified as images.

18. A non-transitory computer readable medium, according to claim 12 , wherein the snippets that are not classified as text are classified as one of: images and unknown.

19. A non-transitory computer readable medium, according to claim 12 , wherein partitioning the document into the first plurality of cells includes partitioning the document into six cells.

20. A non-transitory computer readable medium, according to claim 12 , wherein the document is captured using a smartphone.

21. A non-transitory computer readable medium, according to claim 12 , wherein the classifier is provided by training a neural net using a plurality of image documents and a plurality of text pages having various formats, layouts, text sizes, ranges of word, line and paragraph spacing.

Assignments (8)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 12, 2024
From: EVERNOTE CORPORATION
To: BENDING SPOONS S.P.A.
Reel/Frame 066288/0195 →
RELEASE OF SECURITY INTEREST Recorded Mar 17, 2023
From: MUFG BANK, LTD.
To: EVERNOTE CORPORATION
Reel/Frame 063116/0260 →
RELEASE OF SECURITY INTEREST Recorded Oct 8, 2021
From: EAST WEST BANK
To: EVERNOTE CORPORATION
Reel/Frame 057852/0078 →
SECURITY INTEREST Recorded Oct 6, 2021
From: EVERNOTE CORPORATION
To: MUFG UNION BANK, N.A.
Reel/Frame 057722/0876 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT TERMINATION AT R/F 048513/ 0003 Recorded Oct 22, 2020
From: HERCULES CAPITAL, INC.
To: EVERNOTE CORPORATION
Reel/Frame 054178/0892 →
SECURITY INTEREST Recorded Oct 19, 2020
From: EVERNOTE CORPORATION
To: EAST WEST BANK
Reel/Frame 054113/0876 →
SECURITY INTEREST Recorded Mar 5, 2019
From: EVERNOTE CORPORATION; EVERNOTE GMBH
To: HERCULES CAPITAL, INC., AS AGENT
Reel/Frame 048513/0003 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2017
From: PASHINTSEV, ALEXANDER; GORBATOV, BORIS; LIVSHITZ, EUGENE; GLAZKOV, VITALY
To: EVERNOTE CORPORATION
Reel/Frame 041169/0192 →
Continuity (1)
Continuation 62222368 · Sep 23, 2015
Cited By (1)
US 12,682,674