IP Library Granted Patent US 11,080,808
Granted Patent B2
US 11,080,808 · App. 15/832,499 · Granted Aug 3, 2021

Automatically attaching optical character recognition data to images

Inventors: Aaron Brown (San Francisco, CA); Samantha Puth (San Francisco, CA)
Assignee: LendingClub Corporation
G06T1/0021G06K9/00469G06K2209/01G06K2209/27G06T2201/0062G06T2201/0065
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,080,808
App. No.
15/832,499
Granted
Aug 3, 2021
Kind
B2
Abstract

Techniques for automatically attaching optical character recognition data to images are provided. The techniques include receiving an image file containing an image and performing optical character recognition on the image to generate text output. The techniques then continue by identifying a particular text item from within the generated text output and determining that the particular text item is a value for particular corresponding key. Then metadata that indicates that the particular text item is a value for the particular key is stored in the image file.

Claims (92)

1. A method comprising:

receiving an image file containing an image;

performing optical character recognition on the image to extract text output from the image;

within the text output, identifying a particular text item;

determining a document type associated with the text output from the image;

based at least in part on the image, automatically determining that the particular text item is a value for a particular key;

wherein automatically determining that the particular text item is a value for the particular key comprises:

responsive to determining that the document type associated with the text output from the image is a first document type, identifying the particular key from among a first set of keys associated with the first document type,

responsive to determining that the document type associated with the text output from the image is a second document type, identifying the particular key from among a second set of keys associated with the second document type,

wherein the first set of keys is different from the second set of keys;

generating metadata (a) that includes the particular text item and (b) that indicates that the particular text item is a value for the particular key; and

creating an updated image file by inserting, within the image file, the metadata;

wherein the method is performed using one or more computing devices.

2. The method of claim 1 , further comprising:

determining a location, within the image, that corresponds to the particular text item;

wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on the location of the particular text item within the image.

3. The method of claim 1 , wherein the particular text item is located at a particular location of the image, the method further comprising based, at least in part, on the particular location, displaying the particular text item as rendered text within a display of the image.

4. The method of claim 3 , wherein:

the particular text item is displayed at the particular location, within the image, that corresponds to the particular text item;

the method further comprises:

receiving an interaction event from a user interface, and

in response to receiving the interaction event from the user interface, ceasing to display the rendered text within the display of the image.

5. The method of claim 1 , further comprising:

determining a location, within the image, that corresponds to the particular text item;

wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on both: the image being of the document type, and the location of the particular text item within the image.

6. The method of claim 1 , wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on a format of the particular text item.

7. The method of claim 1 , further comprising:

within the text output, identifying a particular label that corresponds to the particular key;

wherein the metadata stored within the updated image file associates the particular text item with the particular label.

8. The method of claim 1 , further comprising:

within the text output, identifying a particular label that corresponds to the particular key;

determining a first location, within the image, that corresponds to the particular text item; and

determining a second location, within the image, that corresponds to the particular label;

wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on the first location and the second location.

9. The method of claim 8 , wherein automatically determining that the particular text item is a value for the particular key is further based, at least in part, on a relationship between the first location and the second location.

10. The method of claim 1 , wherein the metadata comprises a text-label pair in XMP format.

11. The method of claim 1 , wherein:

the image is associated with a particular entity;

a row in a database table is associated with the particular entity; and

the metadata further comprises data that associates the image file with the row of the database table.

12. The method of claim 1 , further comprising:

within the text output, identifying a second particular text item;

based at least in part on the image, automatically determining that the second particular text item is a value for a second particular key;

wherein the metadata further includes the second particular text item and indicates that the second particular text item is a value for the second particular key;

wherein the second particular key is different than the particular key.

13. The method of claim 1 , further comprising:

receiving, from a user interface, a correction of at least a portion of the particular text item; and

updating, in the metadata of the updated image file, the particular text item based, at least in part, on the correction.

14. The method of claim 1 , wherein said determining the document type associated with the text output from the image is based, at least in part, on analysis of the image.

15. One or more non-transitory computer-readable media storing one or more sequences of instructions that, when executed by one or more computing devices, cause:

receiving an image file containing an image;

performing optical character recognition on the image to extract text output from the image;

within the text output, identifying a particular text item;

determining a document type associated with the text output from the image;

based at least in part on the image, automatically determining that the particular text item is a value for a particular key;

wherein automatically determining that the particular text item is a value for the particular key comprises:

responsive to determining that the document type associated with the text output from the image is a first document type, identifying the particular key from among a first set of keys associated with the first document type,

responsive to determining that the document type associated with the text output from the image is a second document type, identifying the particular key from among a second set of keys associated with the second document type,

wherein the first set of keys is different from the second set of keys;

generating metadata (a) that includes the particular text item and (b) that indicates that the particular text item is a value for the particular key; and

creating an updated image file by inserting, within the image file, the metadata.

16. The one or more non-transitory computer-readable media of claim 15 , wherein the particular text item is located at a particular location of the image and wherein the one or more sequences of instructions further comprise instructions that, when executed by one or more computing devices, cause: based, at least in part, on the particular location, displaying the particular text item as rendered text within a display of the image.

17. The one or more non-transitory computer-readable media of claim 16 , wherein:

the particular text item is displayed at the particular location, within the image, that corresponds to the particular text item;

the one or more sequences of instructions further comprise instructions that, when executed by one or more computing devices, cause:

receiving an interaction event from a user interface, and

in response to receiving the interaction event from the user interface, ceasing to display the rendered text within the display of the image.

18. A system, comprising:

one or more computing devices; and

one or more non-transitory computer-readable media storing instructions that, when executed by the one or more computing devices, cause:

receiving an image file containing an image;

performing optical character recognition on the image to extract text output from the image;

within the text output, identifying a particular text item;

determining a document type associated with the text output from the image;

based at least in part on the image, automatically determining that the particular text item is a value for a particular key;

wherein automatically determining that the particular text item is a value for the particular key comprises:

responsive to determining that the document type associated with the text output from the image is a first document type, identifying the particular key from among a first set of keys associated with the first document type,

responsive to determining that the document type associated with the text output from the image is a second document type, identifying the particular key from among a second set of keys associated with the second document type,

wherein the first set of keys is different from the second set of keys;

generating metadata (a) that includes the particular text item and (b) that indicates that the particular text item is a value for the particular key; and

creating an updated image file by inserting, within the image file, the metadata.

19. The system of claim 18 , wherein the instructions further comprise instructions that, when executed by the one or more computing devices, cause:

determining a location, within the image, that corresponds to the particular text item;

wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on the location of the particular text item within the image.

20. The system of claim 18 , wherein the instructions further comprise instructions that, when executed by the one or more computing devices, cause:

determining a location, within the image, that corresponds to the particular text item;

wherein automatically determining that the particular text item is a value for the particular key is based, at least in part, on both: the image being of the document type, and the location of the particular text item within the image.

21. The system of claim 18 , wherein the instructions further comprise instructions that, when executed by the one or more computing devices, cause:

within the text output, identifying a particular label that corresponds to the particular key;

wherein the metadata stored within the updated image file associates the particular text item with the particular label.

22. The system of claim 18 , wherein the metadata comprises a text-label pair in XMP format.

23. The system of claim 18 , wherein said determining the document type associated with the text output from the image is based, at least in part, on analysis of the image.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2022
From: LENDINGCLUB CORPORATION
To: LENDINGCLUB BANK, NATIONAL ASSOCIATION
Reel/Frame 059910/0275 →
SECURITY INTEREST Recorded Apr 20, 2018
From: LENDINGCLUB CORPORATION
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 045600/0405 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2017
From: BROWN, AARON; PUTH, SAMANTHA
To: LENDINGCLUB CORPORATION
Reel/Frame 044439/0638 →
Continuity (1)
Related Publication 20190172171A1 · Jun 6, 2019