Mixed media reality recognition using multiple specialized indexes
An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a distributor for distributing the image to corresponding content type specific index tables and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The techniques described herein also include a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.
1. A dispatcher apparatus having:
one or more processors;
a distributor stored on a memory and executable by the one or more processors, the distributor for receiving an image, producing one or more content-type specific representations of the image, and submitting the one or more content-type specific representations to one or more content-type specific index tables for recognition; and
an integrator stored on the memory and executable by the one or more processors, the integrator for receiving recognition results from the one or more content-type specific index tables, integrating the recognition results into an integrated result based on a level of agreement between the recognition results, and transmitting the integrated result.
2. The dispatcher apparatus of claim 1 , wherein the one or more content-type specific representations of the image are selected from a group of black text on white background, black and white natural images, color natural images, black and white diagrams, color diagrams, headings, and color text.
3. The dispatcher apparatus of claim 1 , wherein information regarding the one or more content-type specific representations is received with the image.
4. The dispatcher apparatus of claim 1 , further comprising an image feature order unit stored on the memory and executable by the one or more processors, the image feature order unit for analyzing the image and predicting the time required to recognize the image based on features contained in the image.
5. The dispatcher apparatus of claim 1 , wherein integrating the recognition results comprises consideration of at least one factor selected from a competence factor associated with the recognition results, context information for the recognition results, surface area of an image segment, and prior probability of correctness for the one or more corresponding content-type specific index tables.
6. The dispatcher apparatus of claim 1 , wherein the integrated result includes a determination that a same recognition result was produced by the one or more content-type specific index tables.
7. A computer-implemented method of processing content-type specific queries, comprising:
receiving an image;
producing, with one or more processors, one or more content-type specific representations of the image;
submitting the one or more content-type specific representations to one or more corresponding content-type specific index tables for recognition;
receiving recognition results from the one or more content-type specific index tables;
integrating the recognition results into an integrated result based on a level of agreement between the recognition results; and
transmitting the integrated result.
8. The method of claim 7 , wherein information regarding the one or more content-type specific representations is received with the image.
9. The method of claim 7 , further comprising analyzing the image and predicting the time required to recognize the image based on features contained in the image.
10. The method of claim 7 , wherein the one or more content-type specific representations of the image are selected from black text on white background, black and white natural images, color natural images, black and white diagrams, color diagrams, headings, and color text.
11. The method of claim 7 , further comprising determining the one or more content-type specific representations included in the one or more corresponding content-type specific index tables.
12. The method of claim 7 , wherein integrating the recognition results comprises consideration of at least one factor selected from a competence factor associated with the recognition results, context information for the recognition results, surface area of an image segment, and prior probability of correctness for the one or more corresponding content-type specific index tables.
13. The method of claim 7 , wherein the integrated result includes a determination that a same recognition result was produced by the content-type specific index tables.
14. A computer program product comprising a non-transitory machine-readable medium including a computer-readable program, wherein the computer readable program when executed on a computer causes the computer to:
receive an image;
produce, with one or more processors, one or more content-type specific representations of the image;
submit the one or more content-type specific representations to one or more corresponding content-type specific index tables for recognition;
receive recognition results from the one or more content-type specific index tables;
integrate the recognition results into an integrated result based on a level of agreement between the recognition results; and
transmit the integrated result.
15. The computer program product of claim 14 , wherein information regarding the one or more content-type specific representations is received with the image.
16. The computer program product of claim 14 , wherein the computer-readable program causes the computer to analyze the image and predicting the time required to recognize the image based on features contained in the image.
17. The computer program product of claim 14 , wherein the one or more content-type specific representations are selected from black text on white background, black and white natural images, color natural images, black and white diagrams, color diagrams, headings, and color text.
18. The computer program product of claim 14 , wherein the computer-readable program causes the computer to determine the one or more content-type specific representations included in the one or more corresponding content-type specific index tables.
19. The computer program product of claim 14 , wherein integrating the recognition results comprises consideration of at least one factor selected from a competence factor associated with the recognition results, context information for the recognition results, surface area of an image segment, and prior probability of correctness for the one or more corresponding content-type specific index tables.
20. The computer program product of claim 14 , wherein the integrated result includes a determination that a same recognition result was produced by the content-type specific index tables.