IP Library Granted Patent US 11,972,626
Granted Patent B2
US 11,972,626 · App. 17/133,794 · Granted Apr 30, 2024

Extracting multiple documents from single image

Inventors: Ivan Zagaynov (Dolgoprudniy, RU); Aleksandra Stepina (Dubna, RU)
Assignee: ABBYY Development Inc.
G06V30/414G06F18/214G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,972,626
App. No.
17/133,794
Granted
Apr 30, 2024
Kind
B2
Abstract

System and method for document image detection, comprising: producing, using a neural network, a superpixel segmentation map of an input image; generating a superpixel binary mask by associating each superpixel of the superpixel segmentation map with a class of a predetermined set of classes; identifying one or more connected components in the superpixel binary mask; for each connected component of the superpixel binary mask, identifying a corresponding minimum bounding polygon; creating one or more image dividing lines based on the minimum bounding polygons; and defining boundaries of one or more objects of interest based on at least a subset of the image dividing lines.

Claims (53)

1. A computer-implemented method for document image detection, comprising:

producing, using a neural network, a superpixel segmentation map of an input image, wherein each superpixel of the superpixel segmentation map is associated with a list of probability characteristics, such that each probability characteristic of the list of probability characteristics represents a probability of the superpixel belonging to a visual object found in the input image, wherein the visual object is identified by an index of the probability characteristic in the list of probability characteristics;

generating a superpixel binary mask by associating, based on a corresponding list of probability characteristics, each superpixel of the superpixel segmentation map with a class of a predetermined set of classes;

identifying one or more connected components in the superpixel binary mask;

for each connected component of the superpixel binary mask, identifying a corresponding minimum bounding polygon, wherein identifying the minimum bounding polygon further comprises:

responsive to determining that a first number of pixels in a first line of the superpixel binary mask exceeds, by at least a predetermined threshold, a second number of pixels in a second line of the superpixel binary mask which is adjacent to the first line of the superpixel binary mask, utilizing the second line as a candidate boundary of the minimum bounding polygon, wherein the first line is provided by one of: a row of the superpixel binary mask or a column of the superpixel binary mask, and

computing a value of a quality metric for a set of regions of interest that are defined using a plurality of candidate lines comprising the second line of the superpixel binary mask;

creating one or more image dividing lines based on the minimum bounding polygons; and

defining boundaries of one or more objects of interest based on at least a subset of the image dividing lines.

2. The method of claim 1 , wherein the neural network comprises:

a downscale block;

a context block; and

a final classification block.

3. The method of claim 2 , wherein the neural network further comprises a rectifier activation function.

4. The method of claim 1 , further comprising:

cropping each region of interest of one or more regions of interest to produce a corresponding document image.

5. The method of claim 4 , further comprising:

determining whether two or more regions of interest belong to a single multi-part document.

6. The method of claim 1 , wherein the neural network is trained using augmented images.

7. The method of claim 1 , wherein generating the plurality of candidate lines for the minimum bounding polygon further comprises:

utilizing, as a candidate boundary of the minimum bounding polygon, a line traversing a center of the superpixel binary mask.

8. The method of claim 1 , wherein computing a value of a quality metric for the set of regions of interest further comprises:

applying, to the set of regions of interest, a trainable classifier.

9. A system, comprising:

a memory;

a processor, coupled to the memory, the processor configured to:

produce, using a neural network, a superpixel segmentation map of an input image, wherein each superpixel of the superpixel segmentation map is associated with a list of probability characteristics, such that each probability characteristic of the list of probability characteristics represents a probability of the superpixel belonging to a visual object found in the input image, wherein the visual object is identified by an index of the probability characteristic in the list of probability characteristics;

generate a superpixel binary mask by associating, based on a corresponding list of probability characteristics, each superpixel of the superpixel segmentation map with a class of a predetermined set of classes;

identify one or more connected components in the superpixel binary mask, wherein identifying the minimum bounding polygon further comprises:

responsive to determining that a first number of pixels in a first line of the superpixel binary mask exceeds, by at least a predetermined threshold, a second number of pixels in a second line of the superpixel binary mask which is adjacent to the first line of the superpixel binary mask, utilizing the second line as a candidate boundary of the minimum bounding polygon, wherein the first line is provided by one of: a row of the superpixel binary mask or a column of the superpixel binary mask, and

computing a value of a quality metric for a set of regions of interest that are defined using a plurality of candidate lines comprising the second line of the superpixel binary mask;

for each connected component of the superpixel binary mask, identifying a corresponding minimum bounding polygon;

create one or more image dividing lines based on the minimum bounding polygons; and

define boundaries of one or more objects of interest based on at least a subset of the image dividing lines.

10. The system of claim 9 , wherein the neural network comprises:

a downscale block;

a context block; and

a final classification block.

11. The system of claim 10 , wherein the neural network further comprises a rectifier activation function.

12. The system of claim 9 , further comprising:

cropping each region of interest of one or more regions of interest to produce a corresponding document image.

13. The system of claim 12 , further comprising:

determining whether two or more regions of interest belong to a single multi-part document.

14. The system of claim 9 , wherein the neural network is trained using augmented images.

15. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to:

produce, using a neural network, a superpixel segmentation map of an input image, wherein each superpixel of the superpixel segmentation map is associated with a list of probability characteristics, such that each probability characteristic of the list of probability characteristics represents a probability of the superpixel belonging to a visual object found in the input image, wherein the visual object is identified by an index of the probability characteristic in the list of probability characteristics;

generate a superpixel binary mask by associating, based on a corresponding list of probability characteristics, each superpixel of the superpixel segmentation map with a class of a predetermined set of classes;

identify one or more connected components in the superpixel binary mask wherein identifying the minimum bounding polygon further comprises:

responsive to determining that a first number of pixels in a first line of the superpixel binary mask exceeds, by at least a predetermined threshold, a second number of pixels in a second line of the superpixel binary mask which is adjacent to the first line of the superpixel binary mask, utilizing the second line as a candidate boundary of the minimum bounding polygon, wherein the first line is provided by one of: a row of the superpixel binary mask or a column of the superpixel binary mask, and

computing a value of a quality metric for a set of regions of interest that are defined using a plurality of candidate lines comprising the second line of the superpixel binary mask;

for each connected component of the superpixel binary mask, identifying a corresponding minimum bounding polygon;

create one or more image dividing lines based on the minimum bounding polygons; and

define boundaries of one or more objects of interest based on at least a subset of the image dividing lines.

Assignments (3)
SECURITY INTEREST Recorded Aug 14, 2023
From: ABBYY INC.; ABBYY USA SOFTWARE HOUSE INC.; ABBYY DEVELOPMENT INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 064730/0964 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 25, 2022
From: ABBYY PRODUCTION LLC
To: ABBYY DEVELOPMENT INC.
Reel/Frame 059249/0873 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2021
From: ZAGAYNOV, IVAN; STEPINA, ALEKSANDRA
To: ABBYY PRODUCTION LLC
Reel/Frame 055109/0202 →
Priority Claims (1)
RU 2020142364 · Dec 22, 2020 · national
Continuity (1)
Related Publication 20220198187A1 · Jun 23, 2022
Cited By (2)
US 12,330,662 US 12,387,518