IP Library Granted Patent US 10,769,431
Granted Patent B2
US 10,769,431 · App. 16/133,654 · Granted Sep 8, 2020

Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device

Inventors: Martin T. King (Vashon Island, WA); Dale L. Grover (Ann Arbor, MI); Clifford A. Kushler (Nevada City, CA); James Q. Stafford-Fraser (Cambridge, GB)
Assignee: Google LLC
G06K9/00483G06F16/5846G06K9/00422G06K9/00469G06K9/00979G06K9/228G06Q10/10G06Q30/00H04N1/107H04N1/32101G06K2209/01H04N1/0049H04N1/00241H04N1/00307H04N1/00488H04N2201/0041H04N2201/0055H04N2201/0058H04N2201/0081H04N2201/0084H04N2201/0096H04N2201/3249H04N2201/3264H04N2201/3269
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,769,431
App. No.
16/133,654
Granted
Sep 8, 2020
Kind
B2
Abstract

A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand.

Claims (54)

1. A computer-implemented method to capture data from digital images, comprising:

receiving, by one or more computing devices, an input to capture digital images;

providing, by the one or more computing devices, a user interface that displays instructions to capture digital images;

initiating, by the one or more computing devices, an image capturing module of the one or more computing devices;

receiving, by the one or more computing devices, a digital image from the image capturing module;

determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is readable;

performing, by the one or more computing devices, an optical character recognition algorithm on text in the digital image; and

providing, by the one or more computing devices, a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

2. The computer implemented method of claim 1 , further comprising:

receiving, by the one or more computing devices, an input of an annotation to the digital image; and

communicating, by the one or more computing devices, the annotation with the digital image to a processing system.

3. The computer implemented method of claim 1 , further comprising:

determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is not viewable; and

providing, by the one or more computing devices, a notification on the user interface to obtain a second digital image.

4. The computer implemented method of claim 1 , further comprising:

providing, by the one or more computing devices, results of the optical character recognition algorithm to a processing system.

5. The computer implemented method of claim 1 , wherein the instructions are provided in a pop-up menu.

6. The computer implemented method of claim 1 , further comprising communicating, by the one or more computing devices, a request to a processing system to initiate a financial transaction in connection with the digital image.

7. The computer implemented method of claim 1 , wherein the one or more computing devices comprise a mobile computing device.

8. The computer implemented method of claim 7 , wherein the image capturing module is a function of the mobile computing device.

9. A computer program product, comprising:

a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by one or more computing devices cause the computer to capture data from digital images, the computer-readable program instructions comprising computer-executable instructions to:

receive an input to capture digital images;

provide a user interface that displays instructions to capture digital images;

initiate an image capturing module of the one or more computing devices;

receive a digital image from the image capturing module;

determine that at least a portion of a set of text in the digital image is readable;

perform an optical character recognition algorithm on text in the digital image; and

provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

10. The computer program product of claim 9 , further comprising computer-executable instructions to:

receive an input of an annotation to the digital image; and

communicate the annotation with the digital image to a processing system.

11. The computer program product of claim 9 , further comprising computer-executable instructions to:

determine that at least a portion of a set of text in the digital image is not viewable; and

provide a notification on the user interface to obtain a second digital image.

12. The computer program product of claim 10 , further comprising computer-executable instructions to:

provide the results of the optical character recognition algorithm to a processing system.

13. A system to capture data from digital images, comprising:

a storage device; and

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to:

receive an input to capture digital images;

provide a user interface that displays instructions to capture digital images;

initiate an image capturing module of the processor;

receive a digital image from the image capturing module;

determine that at least a portion of a set of text in the digital image is readable;

perform an optical character recognition algorithm on text in the digital image; and

provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

14. The system of claim 13 , wherein the instructions are provided in a pop-up menu.

15. The system of claim 13 , wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate a request to a processing system to initiate a financial transaction in connection with the digital image.

16. The system of claim 13 , wherein the system comprises a mobile computing device.

17. The system of claim 13 , wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.

18. The computer implemented method of claim 1 , further comprising:

communicating, by the one or more computing devices, the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.

19. The computer program product of claim 9 , further comprising computer-executable instructions to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.

Assignments (4)
NUNC PRO TUNC ASSIGNMENT Recorded Aug 3, 2021
From: GOOGLE LLC
To: KYOCERA CORPORATION
Reel/Frame 057651/0445 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2018
From: KING, MARTIN T.; GROVER, DALE L.; KUSHLER, CLIFFORD A.; STAFFORD-FRASER, JAMES Q.
To: EXBIBLIO B.V.
Reel/Frame 047864/0876 →
ENTITY CONVERSION Recorded Dec 28, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 047989/0157 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2018
From: EXBIBLIO B.V.
To: GOOGLE INC.
Reel/Frame 047992/0956 →
Continuity (81)
Continuation 15679237 · Aug 17, 2017
Continuation 15281732 · Sep 30, 2016
Continuation 14080305 · Nov 14, 2013
Division 12899462 · Oct 6, 2010
Continuation In Part 11236330 · Sep 27, 2005
Continuation In Part 11004637 · Dec 3, 2004
Continuation In Part 11097961 · Apr 1, 2005
Continuation In Part 11097093 · Apr 1, 2005
Continuation In Part 11098038 · Apr 1, 2005
Continuation In Part 11098014 · Apr 1, 2005
Continuation In Part 11097103 · Apr 1, 2005
Continuation In Part 11098043 · Apr 1, 2005
Continuation In Part 11097981 · Apr 1, 2005
Continuation In Part 11097089 · Apr 1, 2005
Continuation In Part 11097835 · Apr 1, 2005
Continuation In Part 11098016 · Apr 1, 2005
Continuation In Part 11097828 · Apr 1, 2005
Continuation In Part 11097833 · Apr 1, 2005
Continuation In Part 11097836 · Apr 1, 2005
Continuation In Part 11098042 · Apr 1, 2005
Continuation In Part 11096704 · Apr 1, 2005
Continuation In Part 11110353 · Apr 19, 2005
Continuation In Part 11131945 · May 17, 2005
Continuation In Part 11185908 · Jul 19, 2005
Continuation In Part 11208408 · Aug 18, 2005
Continuation In Part 11208457 · Aug 18, 2005
Continuation In Part 11208458 · Aug 18, 2005
Continuation In Part 11208461 · Aug 18, 2005
Continuation In Part 11209333 · Aug 23, 2005
Continuation In Part 11210260 · Aug 23, 2005
Continuation In Part PCTUS2005011533 · Apr 1, 2005
Continuation In Part PCTUS2005013586 · Apr 6, 2005
Continuation In Part PCTUS2005012510 · Apr 12, 2005
Provisional Application 60613243 · Sep 27, 2004
Provisional Application 60613628 · Sep 27, 2004
Provisional Application 60613632 · Sep 27, 2004
Provisional Application 60613589 · Sep 27, 2004
Provisional Application 60613242 · Sep 27, 2004
Provisional Application 60613602 · Sep 27, 2004
Provisional Application 60613340 · Sep 27, 2004
Provisional Application 60613634 · Sep 27, 2004
Provisional Application 60613461 · Sep 27, 2004
Provisional Application 60613455 · Sep 27, 2004
Provisional Application 60613460 · Sep 27, 2004
Provisional Application 60613400 · Sep 27, 2004
Provisional Application 60613456 · Sep 27, 2004
Provisional Application 60613341 · Sep 27, 2004
Provisional Application 60613361 · Sep 27, 2004
Provisional Application 60613454 · Sep 27, 2004
Provisional Application 60613339 · Sep 27, 2004
Provisional Application 60613633 · Sep 27, 2004
Provisional Application 60615378 · Oct 1, 2004
Provisional Application 60615112 · Oct 1, 2004
Provisional Application 60615538 · Oct 1, 2004
Provisional Application 60617122 · Oct 7, 2004
Provisional Application 60622906 · Oct 28, 2004
Provisional Application 60633452 · Dec 6, 2004
Provisional Application 60633678 · Dec 6, 2004
Provisional Application 60633486 · Dec 6, 2004
Provisional Application 60633453 · Dec 6, 2004
Provisional Application 60634627 · Dec 9, 2004
Provisional Application 60634739 · Dec 9, 2004
Provisional Application 60647684 · Jan 26, 2005
Provisional Application 60648746 · Jan 31, 2005
Provisional Application 60653372 · Feb 15, 2005
Provisional Application 60653663 · Feb 16, 2005
Provisional Application 60653669 · Feb 16, 2005
Provisional Application 60653899 · Feb 16, 2005
Provisional Application 60653679 · Feb 16, 2005
Provisional Application 60653847 · Feb 16, 2005
Provisional Application 60654379 · Feb 17, 2005
Provisional Application 60654368 · Feb 18, 2005
Provisional Application 60654326 · Feb 18, 2005
Provisional Application 60654196 · Feb 18, 2005
Provisional Application 60655279 · Feb 22, 2005
Provisional Application 60655280 · Feb 22, 2005
Provisional Application 60655987 · Feb 22, 2005
Provisional Application 60655697 · Feb 22, 2005
Provisional Application 60655281 · Feb 22, 2005
Provisional Application 60657309 · Feb 28, 2005
Related Publication 20190087656A1 · Mar 21, 2019
Cited By (2)
US 12,483,533 US 12,501,225