IP Library Granted Patent US 11,062,176
Granted Patent B2
US 11,062,176 · App. 16/206,912 · Granted Jul 13, 2021

Object detection and image cropping using a multi-detector approach

Inventors: Jiyong Ma (San Diego, CA); Stephen M. Thompson (Oceanside, CA); Jan W. Amtrup (Silver Spring, MD)
Assignee: KOFAX, INC.
G06K9/6227G06K9/00711G06K9/4604G06K9/6262G06N7/00G06N20/00G06T7/0002G06T7/11G06T7/12G06T7/13G06T7/194G06T7/70G06T7/90G06K9/38G06T2207/10016G06T2207/10024G06T2207/20081G06T2207/30176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,062,176
App. No.
16/206,912
Granted
Jul 13, 2021
Kind
B2
Abstract

Computerized techniques for real-time object detection from video data include: defining an analysis profile comprising an initial number of analysis cycles dedicated to each of a plurality of detectors, each detector being independently configured to detect objects according to a unique set of analysis parameters; receiving a plurality of frames of digital video data, the digital video data depicting an object; analyzing the plurality of frames using the plurality of detectors and in accordance with the analysis profile, wherein analyzing the plurality of frames produces an analysis result for each of the plurality of detectors; determining a confidence score for each of the analysis results; and updating the analysis profile by adjusting the number of analysis cycles dedicated to at least one of the plurality of detectors based on the confidence scores. Corresponding systems and computer program products are also disclosed.

Claims (57)

1. A computer-implemented method of detecting objects within digital video data, the method comprising:

defining an analysis profile comprising an initial number of analysis cycles dedicated to each of a plurality of detectors, each detector being independently configured to detect objects according to a unique set of analysis parameters;

receiving a plurality of frames of digital video data, the digital video data depicting an object;

analyzing the plurality of frames using the plurality of detectors and in accordance with the analysis profile, wherein analyzing the plurality of frames produces an analysis result for each of the plurality of detectors;

determining a confidence score for each of the analysis results; and

updating the analysis profile by adjusting the initial number of analysis cycles dedicated to at least one of the plurality of detectors based on the confidence scores.

2. The method as recited in claim 1 , wherein the adjusting:

affords additional cycles to detectors having confidence results greater than or equal to one or more predetermined confidence thresholds, wherein each confidence score is calibrated to one of the plurality of detectors; and

reduces, but never eliminates, a number of analysis cycles for detectors having confidence results less than a predetermined minimum confidence threshold.

3. The method as recited in claim 1 , wherein each analysis cycle corresponds to a sampled frame of the digital video data; and wherein consecutive analysis cycles do not necessarily correspond to consecutive frames of the digital video data.

4. The method as recited in claim 1 , comprising performing a plurality of iterations of the analyzing, the determining, and the updating, and the method further comprising:

storing the confidence score determined for each detector in a historical record; and

wherein the updating is based on confidence scores determined for each of the detectors in a plurality of prior iterations.

5. The method as recited in claim 4 , wherein the confidence scores upon which the updating is based are weighted according to chronological proximity to a current iteration of the analyzing, the determining, and the updating.

6. The method as recited in claim 1 , wherein the plurality of detectors comprise at least two detectors selected from a group consisting of:

a color transition detector configured to detect transitions between a background and a foreground of frames of digital video data;

a line segment detector configured to identify a plurality of line segments corresponding to substantially straight lines or line segments within frames of digital video data; and

a text line detector configured to identify blocks of text and orientation thereof within frames of digital video data.

7. The method as recited in claim 1 , wherein the digital video data depict a plurality of objects each having unique characteristics, and wherein the method comprises detecting the plurality of objects within one or more of the frames of digital video data based on a plurality of iterations of the analyzing, the determining and the updating.

8. The method as recited in claim 7 , wherein, each object is detected according to a different analysis profile, and wherein the method comprises dynamically adjusting the analysis profile to optimally detect the object in accordance with the unique characteristics thereof.

9. The method as recited in claim 1 , further comprising:

selecting the analysis result having a highest confidence score as an optimum object location result; and

one or more of:

outputting, based on the optimum object location result, a projected location of one or more edges of the object to a memory; and

displaying, based on the optimum object location result, a projected location of the one or more edges of the object on a display.

10. A computer program product for detecting objects within digital video data, the computer program product comprising a computer readable storage medium having embodied therewith computer readable program instructions, wherein the computer readable program instructions are configured to cause a processor, upon execution thereof, to perform a method comprising:

defining, using the processor, an analysis profile comprising an initial number of analysis cycles dedicated to each of a plurality of detectors, each detector being independently configured to detect objects according to a unique set of analysis parameters;

receiving, using the processor, a plurality of frames of digital video data, the digital video data depicting an object;

analyzing, using the processor, the plurality of frames using the plurality of detectors and in accordance with the analysis profile, wherein analyzing the plurality of frames produces an analysis result for each of the plurality of detectors;

determining, using the processor, a confidence score for each of the analysis results; and

updating, using the processor, the analysis profile by adjusting the initial number of analysis cycles dedicated to at least one of the plurality of detectors based on the confidence scores.

11. The computer program product as recited in claim 10 , wherein adjusting the number of analysis cycles:

affords additional cycles to detectors having confidence results greater than or equal to a predetermined confidence threshold; and

reduces, but never eliminates, a number of analysis cycles for detectors having confidence results less than a predetermined minimum confidence threshold.

12. The computer program product as recited in claim 10 , wherein each analysis cycle corresponds to a sampled frame of the digital video data; and wherein consecutive analysis cycles do not necessarily correspond to consecutive frames of the digital video data.

13. The computer program product as recited in claim 10 , comprising program instructions configured to cause the processor, upon execution thereof, to:

perform a plurality of iterations of the analyzing, the determining, and the updating; and

store, using the processor, the confidence score determined for each detector in a historical record; and

wherein the updating is based on confidence scores determined for each of the detectors in a plurality of prior iterations.

14. The computer program product as recited in claim 13 , wherein the confidence scores upon which the updating is based are weighted according to chronological proximity to a current iteration of the analyzing, the determining, and the updating.

15. The computer program product as recited in claim 10 , wherein the plurality of detectors comprise at least two detectors selected from a group consisting of:

a color transition detector configured to detect transitions between a background and a foreground of frames of digital video data;

a line segment detector configured to identify a plurality of line segments corresponding to substantially straight lines or line segments within frames of digital video data; and

a text line detector configured to identify blocks of text and orientation thereof within frames of digital video data.

16. The computer program product as recited in claim 10 , wherein the digital video data depict a plurality of objects each having unique characteristics, and wherein the program instructions further comprise program instructions configured to cause the processor, upon execution thereof, to detect, using the processor, the plurality of objects within one or more of the frames of digital video data based on a result of a plurality of iterations of the analyzing, the determining and the updating.

17. The computer program product as recited in claim 16 , wherein, each object is detected according to a different analysis profile, and wherein the program instructions further comprise program instructions configured to cause the processor, upon execution thereof, to dynamically adjust, using the processor, the analysis profile to optimally detect the object in accordance with the unique characteristics thereof.

18. The computer program product as recited in claim 10 , further comprising program instructions configured to cause the processor, upon execution thereof, to:

select the analysis result having a highest confidence score as an optimum object location result; and

either or both of:

output, using the processor and based on the optimum object location result, a projected location of one or more edges of the object to a memory; and

display, using the processor and based on the optimum object location result, a projected location of the one or more edges of the object on a display.

19. A system for detecting objects within digital video data, the system comprising a processor and logic integrated and/or executable by the processor, wherein the logic is configured to cause the processor to perform a method comprising:

defining, using the processor, an analysis profile comprising an initial number of analysis cycles dedicated to each of a plurality of detectors, each detector being independently configured to detect objects according to a unique set of analysis parameters;

receiving, using the processor, a plurality of frames of digital video data, the digital video data depicting an object;

analyzing, using the processor, the plurality of frames using the plurality of detectors and in accordance with the analysis profile, wherein analyzing the plurality of frames produces an analysis result for each of the plurality of detectors;

determining, using the processor, a confidence score for each of the analysis results; and

updating, using the processor, the analysis profile by adjusting the initial number of analysis cycles dedicated to at least one of the plurality of detectors based on the confidence scores.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2024
From: KOFAX, INC.
To: TUNGSTEN AUTOMATION CORPORATION
Reel/Frame 067428/0392 →
FIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A. AS COLLATERAL AGENT
Reel/Frame 060757/0565 →
SECURITY INTEREST Recorded Jul 20, 2022
From: KOFAX, INC.; PSIGEN SOFTWARE, INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 060768/0159 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2019
From: MA, JIYONG; THOMPSON, STEPHEN M.; AMTRUP, JAN W.
To: KOFAX, INC.
Reel/Frame 048094/0397 →
Cited By (1)
US 12,340,552