IP Library Granted Patent US 11,087,407
Granted Patent B2
US 11,087,407 · App. 16/824,518 · Granted Aug 10, 2021

Systems and methods for mobile image capture and processing

Inventors: Anthony Macciola (Irvine, CA); Jan Willers Amtrup (Silver Spring, MD); Alexander Shustorovich (Pittsford, NY); Christopher W. Thrasher (Rochester, NY)
Assignee: KOFAX, INC.
G06Q40/08G06F3/005G06F17/40G06K9/00456G06K9/00463G06K9/3233G06Q50/18G06T1/0007G06T3/0006G06T3/0056G06T5/00G06T5/003G06T7/00G06T7/0002G06T7/12G06T7/13G06T7/143G06T7/194G06T11/20H04N1/00116H04N1/00251H04N1/00403H04N1/387H04N1/3872H04N1/3878H04N1/40G06K9/38G06K2009/363G06T5/002G06T2207/10024G06T2207/20021G06T2207/20164G06T2207/30004G06T2207/30176H04N2201/0081H04N2201/0096
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,087,407
App. No.
16/824,518
Granted
Aug 10, 2021
Kind
B2
Abstract

In several embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. The techniques include capturing image data depicting a document; defining a plurality of candidate edge points within the image data; and defining four sides of a tetragon based on at least some of the plurality of candidate edge points; wherein each side of the tetragon corresponds to a different side of the document; wherein an area of the tetragon comprises at least a threshold percentage of a total area of the digital image; and wherein the tetragon bounds the digital representation of the document.

Claims (76)

1. A computer program product, comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

capturing one or more of image data depicting a digital representation of a document and audio data relating to the digital representation of the document;

defining a plurality of candidate edge points within the image data, wherein defining the plurality of candidate edge points comprises adjusting for illumination drift inherent to the image based at least in part on calculating statistics descriptive of background for each of a plurality of distinct portions of the image; and

defining four sides of a tetragon based on at least some of the plurality of candidate edge points;

wherein each side of the tetragon corresponds to a different side of the document;

wherein an area of the tetragon comprises at least a threshold percentage of a total area of the digital image; and

wherein the tetragon bounds the digital representation of the document.

2. The computer program product as recited in claim 1 , wherein defining the plurality of candidate edge points comprises, for each of a plurality of portions of the image data:

calculating one or more statistics corresponding to the portion;

estimating one or more distributions of statistics corresponding to the portion;

determining whether a statistically significant difference exists between one or more of the statistics calculated for the portion and the distribution of statistics estimated for the portion; and

designating a point corresponding to the statistically significant difference as one of the candidate edge points in response to determining the statistically significant difference exists.

3. The computer program product as recited in claim 2 , wherein the statistics calculated for each portion comprise:

a minimum brightness value;

a maximum brightness value; and/or

a brightness value range; and

wherein the distributions of statistics estimated for each portion comprise:

a distribution of color channel value minima;

a distribution of color channel value maxima; and/or

a distribution of color channel value ranges.

4. The computer program product as recited in claim 2 , wherein each portion is defined along a path proceeding from an outer region of the image toward a center of the image; and

wherein the path proceeds along rows and/or columns of the image.

5. The computer program product as recited in claim 2 , wherein at least one portion comprises a corner region of the image;

wherein each portion is characterized by a substantially rectangular shape; and

wherein each small analysis window is characterized by a substantially rectangular shape.

6. The computer program product as recited in claim 2 , wherein each portion is characterized by a single center pixel, and

wherein the center pixel is designated as one of the candidate edge points upon determining the statistically significant difference exists.

7. The computer program product as recited in claim 1 , further comprising program instructions executable by the processor to cause the processor to bypass one or more variations in a texture of a background of the image in response to determining a statistically significant difference exists between one or more statistics representative of a given portion of the image and a corresponding distribution of statistics representative of the given portion of the image.

8. The computer program product as recited in claim 1 , wherein each side of the tetragon is characterized by a polynomial equation, and

wherein the defining comprises determining one or more coefficients for each polynomial equation.

9. The computer program product as recited in claim 1 , wherein a corner of the tetragon comprises an intersection of two adjacent sides of the tetragon,

wherein the two adjacent sides are selected from:

one substantially straight line and one substantially curved line;

one substantially straight line and one substantially parabolic curve; and

two substantially parabolic curves.

10. The computer program product as recited in claim 1 , further comprising program instructions executable by the processor to cause the processor to define one or more tetragon corners, the defining comprising:

solving one or more of:

a first degree polynomial equation;

a second degree polynomial equation;

a third degree polynomial equation; and

a fourth degree polynomial equation; and

calculating one or more intersections between two adjacent sides of the tetragon; and

designating an appropriate intersection from the one or more calculated intersections.

11. The computer program product as recited in claim 1 , wherein a first line connects a calculated top left corner of the tetragon to a calculated bottom right corner of the tetragon,

wherein a second line connects a calculated top right corner of the tetragon and a calculated bottom left corner of the tetragon, and

wherein the first line and the second line intersect inside the tetragon.

12. The computer program product as recited in claim 1 , further comprising program instructions executable by the processor to cause the processor to:

determine whether the tetragon satisfies one or more quality control metrics; and

reject the tetragon upon determining the tetragon does not satisfy one or more of the quality control metrics;

wherein the quality control metrics comprise:

a Least Mean Squares (LMS) support metric;

a minimum corner-to-edge pixel distance metric; and/or

a tetragon diagonal intersection location metric.

13. The computer program product as recited in claim 1 , wherein defining the plurality of candidate edge points within the image data is based on detecting one or more image characteristics indicative of a transition from a background of the image to a foreground of the image rather than detecting one or more edges of the digital representation of the document.

14. The computer program product as recited in claim 13 , wherein the one or more image characteristics indicative of the transition from the background of the image to the foreground of the image comprise a flattening of brightness levels within a region of the image.

15. The computer program product as recited in claim 1 , wherein defining the plurality of candidate edge points is performed without respect to: edge thickness; and edge direction.

16. A system, comprising a processor and logic executable by the processor to cause the processor to perform operations comprising:

capturing image data depicting a digital representation of a document;

defining a plurality of candidate edge points within the image data, wherein defining the plurality of candidate edge points comprises adjusting for illumination drift inherent to the image based at least in part on calculating statistics descriptive of background for each of a plurality of distinct portions of the image; and

defining four sides of a tetragon based on at least some of the plurality of candidate edge points;

wherein each side of the tetragon corresponds to a different side of the document;

wherein an area of the tetragon comprises at least a threshold percentage of a total area of the digital image; and

wherein the tetragon bounds the digital representation of the document.

17. The system as recited in claim 16 , wherein a first line connects a calculated top left corner of the tetragon to a calculated bottom right corner of the tetragon,

wherein a second line connects a calculated top right corner of the tetragon and a calculated bottom left corner of the tetragon, and

wherein the first line and the second line intersect inside the tetragon.

18. A computer-implemented method, comprising:

capturing image data depicting a digital representation of a document;

defining a plurality of candidate edge points within the image data; and

defining four sides of a tetragon based on at least some of the plurality of candidate edge points, wherein defining the plurality of candidate edge points comprises adjusting for illumination drift inherent to the image based at least in part on calculating statistics descriptive of background for each of a plurality of distinct portions of the image;

wherein each side of the tetragon corresponds to a different side of the document;

wherein an area of the tetragon comprises at least a threshold percentage of a total area of the digital image; and

wherein the tetragon bounds the digital representation of the document.

19. The method as recited in claim 18 , wherein a first line connects a calculated top left corner of the tetragon to a calculated bottom right corner of the tetragon,

wherein a second line connects a calculated top right corner of the tetragon and a calculated bottom left corner of the tetragon, and

wherein the first line and the second line intersect inside the tetragon.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2024
From: KOFAX, INC.
To: TUNGSTEN AUTOMATION CORPORATION
Reel/Frame 067428/0392 →
FIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A. AS COLLATERAL AGENT
Reel/Frame 060757/0565 →
SECURITY INTEREST Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 060768/0159 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2020
From: MACCIOLA, ANTHONY; AMTRUP, JAN WILLERS; SHUSTOROVICH, ALEXANDER; THRASHER, CHRISTOPHER W.
To: KOFAX, INC.
Reel/Frame 052207/0613 →
Continuity (6)
Continuation 15394726 · Dec 29, 2016
Continuation 15339789 · Oct 31, 2016
Continuation 13740141 · Jan 11, 2013
Provisional Application 61720958 · Oct 31, 2012
Provisional Application 61586062 · Jan 12, 2012
Related Publication 20200219202A1 · Jul 9, 2020
Cited By (1)
US 12,340,552