IP Library Granted Patent US 11,093,690
Granted Patent B1
US 11,093,690 · App. 16/540,951 · Granted Aug 17, 2021

Synchronization and tagging of image and text data

Inventors: Suchan Lee (Sunnyvale, CA); Jon Paek (Sunnyvale, CA)
Assignee: Palantir Technologies Inc.
G06F40/117G06F9/451G06F16/51G06F16/5846G06F16/93G06F40/109G06F40/134G06K9/46G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,093,690
App. No.
16/540,951
Granted
Aug 17, 2021
Kind
B1
Abstract

A computing system accesses an image-based document and a text document having text extracted from the image-based document and provides a user interface displaying at least a portion of the image-based document. In response to selection of a text portion of the image-based document, the system determines an occurrence of the text portion within at least a portion of the image-based document and then applies a search model on the text document to identify the same occurrence of the text portion. Once matched, alignment data indicating a relationship between a selected tag and both the text portion of the image-based document and the text portion of the text document is stored.

Claims (55)

1. A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:

accessing a first image-based document and a second text document, wherein each of the first and second documents include common text;

providing a user interface displaying at least a portion of the image-based document;

receiving selection from a user of a text portion of the image-based document;

receiving selection of a tag to be associated with the text portion of the image-based document;

determining an occurrence of the text portion within at least a portion of the image-based document;

searching at least a portion of the text document to identify the same occurrence of the text portion in the text document;

storing an alignment data indicating a relationship between the selected tag and both the text portion of the image-based document and the text portion of the text document; and

alternatively displaying the image-based document and the text document in the user interface responsive to user selection of a user interface control, wherein the alignment data is usable to identify the text portion in the image-based document when displayed and to identify the text portion in the text document when displayed.

2. The computerized method of claim 1 , wherein the tag indicates that the text portion is one of an object, a link to one or more existing objects, or a property of one or more existing objects.

3. The computerized method of claim 1 , wherein the at least a portion of the image-based document is the entire image-based document, a certain page of the image-based document, or a character range surrounding the text portion of the image-based document.

4. The computerized method of claim 1 , wherein the image-based document is a portable document format (PDF) document.

5. The computerized method of claim 1 , wherein the image-based document includes one or more graphical images.

6. The computerized method of claim 1 , further comprising:

receiving selection of a graphical area of the image-based document;

receiving selection of a second tag to be associated with the graphical area; and

storing alignment data indicating a second relationship between the second tag and the graphical area.

7. The computerized method of claim 6 , further comprising:

determining a second portion of the text document corresponding to a location of the graphical area of the image-based document, wherein the alignment data indicates the determined second portion of the text document.

8. The computerized method of claim 6 , further comprising:

extracting the graphical area;

formatting the graphical area as an image file; and

storing the image file.

9. The computerized method of claim 1 , further comprising:

normalizing the text portion of the image-based document by performing one or more of:

removing extra punctuation;

removing extra spaces;

removing formatting; and

removing prefixes and/or suffixes of words.

10. The computerized method of claim 1 , further comprising:

transmitting one or more properties of the tag to one or more third party databases with a request for further information.

11. The computerized method of claim 1 , wherein the tag includes one or more properties indicating the text portion or a title associated with the tag.

12. The computerized method of claim 1 , further comprising:

determining a custom model to be used in searching the text document, wherein said searching the text document is performed based on the custom model.

13. The computerized method of claim 12 , wherein the custom model is selected based on a file type of the image-based document or content of the image-based document.

14. The computerized method of claim 12 , wherein the custom model indicates keys, tags, headers, or other indicator of content in the image-based document to be used in said searching the text document.

15. The computerized method of claim 1 , further comprising:

exporting to an external computing system one or more of the tag,

data related to the tag received from one or more third-party databases; or

image files extracted from the image-based document.

16. The computerized method of claim 15 , wherein a PDF file is generated to contain the exported information.

17. A computing system comprising:

a hardware computer processor;

a non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising:

accessing a first image-based document and a second text document, wherein each of the first and second documents include common text;

providing a user interface displaying at least a portion of the image-based document;

receiving selection from a user of a text portion of the image-based document;

receiving selection of a tag to be associated with the text portion of the image-based document;

determining an occurrence of the text portion within at least a portion of the image-based document;

searching at least a portion of the text document to identify the same occurrence of the text portion in the text document;

storing an alignment data indicating a relationship between the selected tag and both the text portion of the image-based document and the text portion of the text document; and

alternatively displaying the image-based document and the text document in the user interface responsive to user selection of a user interface control, wherein the alignment data is usable to identify the text portion in the image-based document when displayed and to identify the text portion in the text document when displayed.

18. The computing system of claim 17 , wherein the tag indicates that the text portion is one of an object, a link to one or more existing objects, or a property of one or more existing objects.

19. The computing system of claim 17 , wherein the at least a portion of the image-based document is the entire image-based document, a certain page of the image-based document, or a character range surrounding the text portion of the image-based document.

20. The computing system of claim 17 , wherein the image-based document is a portable document format (PDF) document.

Assignments (2)
SECURITY INTEREST Recorded Jul 3, 2022
From: PALANTIR TECHNOLOGIES INC.
To: WELLS FARGO BANK, N.A.
Reel/Frame 060572/0506 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2019
From: LEE, SUCHAN; PAEK, JON
To: PALANTIR TECHNOLOGIES INC.
Reel/Frame 050982/0860 →