IP Library Granted Patent US 8,879,120
Granted Patent B2
US 8,879,120 · App. 13/740,139 · Granted Nov 4, 2014

Systems and methods for mobile image capture and processing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,879,120
App. No.
13/740,139
Granted
Nov 4, 2014
Kind
B2
Abstract

In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impractical, and furthermore address unique challenges presented by images captured using a camera rather than a traditional flat-bed scanner, paper-feed scanner or multifunction peripheral.

Claims (113)

1. A method, comprising:

dividing, using a processor, a tetragon comprising a digital representation of a document in a digital image into a plurality of sections, each section comprising a plurality of pixels;

for each section:

determining whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

counting a total number of first direction sharp pixel-to-pixel transitions for the section (S S1 );

determining whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

counting a total number of first-direction blurred pixel-to-pixel transitions for the section (S B1 );

determining whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

counting a total number of second-direction sharp pixel-to-pixel transitions for the section (S S2 );

determining whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

counting a total number of second-direction blurred pixel-to-pixel transitions for the section (S B2 );

determining the section is blank upon determining:

S S1 is less than a predetermined sharp transition threshold,

S B1 is less than a predetermined blurred transition threshold,

S S2 is less than a predetermined sharp transition threshold, and

S B2 is less than a predetermined blurred transition threshold; and

determining, for all non-blank sections, a first direction blur ratio r 1 =S S1 /S B1 ;

determining, for all non-blank sections, a second direction blur ratio r 2 =S S2 /S B2 ;

determining that a non-blank section is blurred in the first direction upon

determining that r 1 is less than a predefined section blur ratio threshold; and

determining that a non-blank section is blurred in the second direction upon determining that r 2 is less than the predefined section blur ratio threshold; and

determining that a non-blank section is bluffed upon determining one or more of:

the section is blurred in the first direction, and

the section is blurred in the section direction; and

determining a total number of blurred sections;

calculating an image blur ratio R comprising: the total number blurred sections to a total number of sections; and

determining the digital image is blurred upon determining the image blur ratio is greater than a predetermined image blur threshold.

2. The method as recited in claim 1 , further comprising:

determining, for each section a distribution of brightness values of the plurality of pixels;

determining a characteristic variability v of the distribution of brightness values;

calculating a noticeable brightness transition threshold η based on v;

calculating a large brightness transition threshold μ based on η;

analyzing, for each pixel within the plurality of pixels, a directional pattern of brightness change in a window surrounding the pixel; and

identifying one or more of: the sharp pixel-to-pixel transition and the blurred pixel-to-pixel transitions based on the analysis.

3. The method as recited in claim 2 , further comprising:

defining a plurality of center pixels;

sequentially analyzing each of the plurality of center pixels within one or more small windows of pixels surrounding each center pixel;

identifying the sharp pixel-to-pixel transition upon determining:

the large brightness transition exists within an immediate vicinity of one of the center pixels;

a first small brightness variation exists before the large brightness transition; and

a second small brightness variation exists after the large brightness transition;

detecting the sharp pixel-to-pixel transition upon determining:

the large transition exists within one or more of the small windows,

a monotonic change in brightness exists in the large transition; and

detecting the bluffed pixel-to-pixel transition upon determining:

a noticeable transition occurs within a small window; and

the monotonic change in brightness exists in the noticeable transition.

4. The method as recited in claim 3 , further comprising, for each section:

counting a total number of sharp transitions in each of one or more chosen directions;

counting a total number of blurred transitions in each chosen direction;

determining that a section is blank upon determining:

the total number of sharp transitions is less than a predefined sharp transition threshold and

the total number of blurred transitions is less than a predefined blurred transition threshold;

determining the non-blank section is blurred upon determining a section blurriness ratio comprising the total number of sharp transitions to the total number of blurred transitions is less than a section blur ratio threshold in at least one of the chosen directions; and

determining that the section is sharp upon determining the section is neither blank nor blurred.

5. The method as recited in claim 4 , further comprising:

determining a total number of blank sections within the plurality of sections (N blank );

determining a total number of blurred sections within the plurality of sections (N blur );

determining a total number of sharp sections within the plurality of sections (N sharp );

determining a blurriness ratio (R B )=N blur /(N blur +N sharp ); and

determining that the digital image is sharp if the R B is less than a blurriness threshold.

6. The method as recited in claim 1 , wherein the processor is part of a mobile device, the mobile device having an integrated camera.

7. A system, comprising:

a processor configured to execute logic;

logic for dividing, using a processor, a tetragon comprising a digital representation of a document in a digital image into a plurality of sections, each section comprising a plurality of pixels;

logic for determining whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

logic for counting a total number of first direction sharp pixel-to-pixel transitions for the section (S S1 );

logic for determining whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

logic for counting a total number of first-direction blurred pixel-to-pixel transitions for the section (S B1 );

logic for determining whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

logic for counting a total number of second direction sharp pixel-to-pixel transitions for the section (S S2 );

logic for determining whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

logic for counting a total number of second-direction blurred pixel-to-pixel transitions for the section (S B2 );

logic for determining the section is blank upon determining:

S S1 is less than a predetermined sharp transition threshold,

S B1 is less than a predetermined blurred transition threshold,

S S2 is less than a predetermined sharp transition threshold, and

S B2 is less than a predetermined blurred transition threshold; and

logic for determining, for all non-blank sections, a first direction blur ratio r 1 =S S1 /S B1 ;

logic for determining, for all non-blank sections, a second direction blur ratio r 2 =S S2 /S B2 ;

logic for determining that a non-blank section is blurred in the first direction upon determining that r 1 is less than a predefined section blur ratio threshold; and

logic for determining that a non-blank section is blurred in the second direction upon determining that r 2 is less than the predefined section blur ratio threshold; and

logic for determining that a non-blank section is blurred upon determining one or more of:

the section is blurred in the first direction; and

the section is blurred in the section direction; and

logic for determining a total number of blurred sections;

logic for calculating an image blur ratio R comprising: the total number blurred sections to a total number of sections; and

logic for determining the digital image is blurred upon determining the image blur ratio is greater than a predetermined image blur threshold.

8. A computer program product comprising a non-transitory computer readable storage medium having computer readable program code stored thereon, the computer readable program code comprising:

computer readable program code configured to divide, using a processor, a tetragon comprising a digital representation of a document in a digital image into a plurality of sections, each section comprising a plurality of pixels

computer readable program code configured to determine whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

computer readable program code configured to count a total number of first direction sharp pixel-to-pixel transitions for the section (S S1 );

computer readable program code configured to determine whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

computer readable program code configured to count a total number of first-direction blurred pixel-to-pixel transitions for the section (S B1 );

computer readable program code configured to determine whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

computer readable program code configured to count a total number of second direction sharp pixel-to-pixel transitions for the section (S S2 );

computer readable program code configured to determine whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

computer readable program code configured to count a total number of second-direction blurred pixel-to-pixel transitions for the section (S B1 );

computer readable program code configured to determine the section is blank upon determining:

S s1 is less than a predetermined sharp transition threshold,

S B1 is less than a predetermined blurred transition threshold,

S S2 is less than a predetermined sharp transition threshold, and

S B2 is less than a predetermined blurred transition threshold; and

computer readable program code configured to determine, for all non-blank sections, a first direction blur ratio r 1 =S S1 /S B1 ;

computer readable program code configured to determine, for all non-blank sections, a second direction blur ratio r 2 =S S2 /S B2 ;

computer readable program code configured to determine that a non-blank section is blurred in the first direction upon determine that r 1 is less than a predefined section blur ratio threshold; and

computer readable program code configured to determine that a non-blank section is blurred in the second direction upon determine that r 2 is less than the predefined section blur ratio threshold; and

computer readable program code configured to determine that a non-blank section is blurred upon determine one or more of:

the section is blurred in the first direction; and

the section is blurred in the section direction; and

computer readable program code configured to determine a total number of blurred sections;

computer readable program code configured to calculate an image blur ratio R comprising: the total number blurred sections to a total number of sections; and

computer readable program code configured to determine the digital image is blurred upon determine the image blur ratio is greater than a predetermined image blur threshold.

Assignments (8)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2024
From: KOFAX, INC.
To: TUNGSTEN AUTOMATION CORPORATION
Reel/Frame 067428/0392 →
RELEASE OF SECURITY INTEREST Recorded Jul 21, 2022
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: KAPOW TECHNOLOGIES, INC.; KOFAX, INC.
Reel/Frame 060805/0161 →
FIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A. AS COLLATERAL AGENT
Reel/Frame 060757/0565 →
SECURITY INTEREST Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 060768/0159 →
SECURITY INTEREST Recorded Jul 7, 2017
From: KOFAX, INC.
To: CREDIT SUISSE
Reel/Frame 043108/0207 →
RELEASE OF SECURITY INTEREST Recorded May 26, 2015
From: BANK OF AMERICA, N.A.
To: KOFAX, INC.; ATALASOFT, INC.; KAPOW TECHNOLOGIES, INC.
Reel/Frame 035773/0930 →
SECURITY AGREEMENT Recorded Oct 21, 2013
From: KOFAX, INC.
To: BANK OF AMERICA, N.A.
Reel/Frame 031447/0490 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2013
From: THRASHER, CHRISTOPHER W.; MACCIOLA, ANTHONY; SHUSTOROVICH, ALEXANDER
To: KOFAX, INC.
Reel/Frame 029659/0451 →