Retrieving electronic documents by converting them to synthetic text
The present invention relies on the two-dimensional information in documents and encodes two-dimensional structures into a one-dimensional synthetic language such that two-dimensional documents can be searched at text search speed. The system comprises: an indexing module, a retrieval module, an encoder, a quantization module, a retrieval engine and a control module coupled by a bus. Electronic documents are first indexed by the indexing module and stored as a synthetic text library. The retrieval module then converts an input image to synthetic text and searches for matches to the synthetic text in the synthetic text library. The matches can be in turn used to retrieve the corresponding electronic documents. In one or more embodiments, the present invention includes a method for comparing the synthetic text to documents that have been converted to synthetic text for a match.
1. A system comprising:
one or more processors;
a document storage configured to store a plurality of electronic documents; and
a memory coupled with the one or more processors, the memory configured to store modules for execution by the one or more processors, the modules including:
an indexing module configured to, in response to receiving an image patch, identify a first and second structure in the image patch wherein the first and second structure have a vertical positional relationship, generate a first one-dimensional text string that encodes two-dimensional location and size information for the first structure using a relative location and size of the second structure; and
a retrieval module configured to search a library for a second one-dimensional text string that is similar to the first one-dimensional text-string and retrieve, from the document storage, an electronic document corresponding to the second one-dimensional text string that is similar to the first one-dimensional text string.
2. The system of claim 1 , wherein the indexing module uses one of a snake, spiral, or Scale Invariant Feature Transform (SIFT) encoder to generate the first one-dimensional text string.
3. The system of claim 1 , wherein each of the first and the second structure in the image patch comprises a bounding box.
4. The system of claim 1 , wherein each of the first and the second structure in the image patch comprises a Scale Invariant Feature Transform (SIFT) feature.
5. The system of claim 1 , wherein the indexing module is further configured to identify an additional structure in the image patch and generate the first one-dimensional text string using a relative location and size of the second structure and the additional structure.
6. The system of claim 1 , wherein the retrieval module uses a conventional search engine to search for the second one-dimensional text string using the first one-dimensional text string.
7. The system of claim 1 , wherein the retrieval module is further configured to:
retrieve a plurality of electronic documents from the document storage;
determine a rank order using a similarity between the second one-dimensional text string in each of the plurality of electronic documents and the first one-dimensional text string of the image patch; and
present the plurality of electronic documents in the rank order.
8. A method comprising:
receiving, at one or more processors, an image patch;
identifying a first and second structure in the image patch, wherein the first and second structure have a vertical positional relationship;
generating a first one-dimensional text string that encodes two-dimensional location and size information for the first structure using a relative location and size of the second structure;
searching a library for a second one-dimensional text string that is similar to the first one-dimensional text string; and
retrieving, from a document storage, an electronic document corresponding to the second one-dimensional text string that is similar to the first one-dimensional text string.
9. The method of claim 8 , wherein generating comprises using one of a snake, spiral, or Scale Invariant Feature Transform (SIFT) encoder.
10. The method of claim 8 , wherein each of the first and the second structure in the image patch comprises a bounding box.
11. The method of claim 8 , wherein each of the first and the second structure in the image patch comprises a Scale Invariant Feature Transform (SIFT) feature.
12. The method of claim 8 , further comprising identifying an additional structure in the image patch and generating the first one-dimensional text string using a relative location and size of the second structure and the additional structure.
13. The method of claim 8 , wherein the searching is performed by a conventional search engine.
14. The method of claim 8 , wherein retrieving further comprises:
retrieving a plurality of electronic documents from the document storage;
determining a rank order using a similarity between the second one-dimensional text string in each of the plurality of electronic documents and the first one-dimensional text string of the image patch; and
presenting the plurality of electronic documents in the rank order.
15. A non-transitory computer-readable medium carrying instructions which, when processed by one or more processors, causes:
receiving, at the one or more processors, an image patch;
identifying a first and second structure in the image patch, wherein the first and second structure have a vertical positional relationship;
generating a first one-dimensional text string that encodes two-dimensional location and size information for the first structure using a relative location and size of the second structure;
searching a library for a second one-dimensional text string that is similar to the first one-dimensional text string; and
retrieving, from a document storage, an electronic document corresponding to the second one-dimensional text string that is similar to the first one-dimensional text string.
16. The non-transitory computer-readable medium of claim 15 , wherein generating comprises using one of a snake, spiral, or Scale Invariant Feature Transform (SIFT) encoder.
17. The non-transitory computer-readable medium of claim 15 , wherein each of the first and the second structure in the image patch comprises a bounding box.
18. The non-transitory computer-readable medium of claim 15 , wherein each of the first and the second structure in the image patch comprises a Scale Invariant Feature Transform (SIFT) feature.
19. The non-transitory computer-readable medium of claim 15 , wherein the searching is performed by a conventional search engine.
20. The non-transitory computer-readable medium of claim 15 , wherein retrieving further comprises:
retrieving a plurality of electronic documents from the document storage;
determining a rank order using a similarity between the second one-dimensional text string in each of the plurality of electronic documents and the first one-dimensional text string of the image patch; and
presenting the plurality of electronic documents in the rank order.