Using extracted image text
View Patent ↗Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.
1. A computer-implemented method comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
compositing two or more images into a composite image using the plurality images;
detecting one or more features in each of one or more regions of the composite image;
determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.
2. The method of claim 1 , further comprising:
increasing contrast of the composite image including normalizing pixel values in the composite image.
3. The method of claim 2 wherein normalizing pixel values in the composite image comprises:
computing a mean and variance of pixel values in the composite image; and
scaling pixel values in the composite image according to the computed mean and variance.
4. The method of claim 1 , further comprising:
generating a superresolution version of the candidate text region.
5. The method of claim 4 , wherein generating the superresolution version of the candidate text region comprises:
obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;
aligning the versions of the candidate text region to a high resolution grid; and
compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.
6. The method of claim 5 , further comprising:
supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.
7. The method of claim 5 , wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.
8. The method of claim 5 wherein aligning the versions of the candidate text region comprises:
receiving ranging data and movement information associated with each version of the candidate text region; and
aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.
9. The method of claim 5 , wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:
computing a median value of pixels in each version of the candidate region; and
combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.
10. The method of claim 5 , wherein compositing the image comprises performing a high dynamic range process to generate a high dynamic range image.
11. A system comprising:
one or more data processing apparatus; and
a data store storing instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
compositing two or more images into a composite image using the plurality images;
detecting one or more features in each of one or more regions of the composite image;
determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.
12. The system of claim 11 , wherein the operations further comprise:
increasing contrast of the composite image including normalizing pixel values in the composite image.
13. The system of claim 12 wherein normalizing pixel values in the composite image comprises:
computing a mean and variance of pixel values in the composite image; and
scaling pixel values in the composite image according to the computed mean and variance.
14. The system of claim 11 , wherein the operations further comprise:
generating a superresolution version of the candidate text region.
15. The system of claim 14 , wherein generating the superresolution version of the candidate text region comprises:
obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;
aligning the versions of the candidate text region to a high resolution grid; and
compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.
16. The system of claim 15 , wherein the operations further comprise:
supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.
17. The system of claim 15 , wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.
18. The system of claim 15 wherein aligning the versions of the candidate text region comprises:
receiving ranging data and movement information associated with each version of the candidate text region; and
aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.
19. The system of claim 15 , wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:
computing a median value of pixels in each version of the candidate region; and
combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.
20. A computer readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
compositing two or more images into a composite image using the plurality images;
detecting one or more features in each of one or more regions of the composite image;
determining for each region of the composite image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.