Using extracted image text
Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.
1. A computer-implemented method comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
generating a high dynamic range image using the plurality images;
detecting one or more features in each of one or more regions of the high dynamic range image;
determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.
2. The method of claim 1 , further comprising:
increasing contrast of the high dynamic range image including normalizing pixel values in the high dynamic range image.
3. The method of claim 2 , wherein normalizing pixel values in the high dynamic range image comprises:
computing a mean and variance of pixel values in the high dynamic range image; and
scaling pixel values in the high dynamic range image according to the computed mean and variance.
4. The method of claim 1 , further comprising:
generating a superresolution version of the candidate text region.
5. The method of claim 4 , wherein generating the superresolution version of the candidate text region comprises:
obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;
aligning the versions of the candidate text region to a high resolution grid; and
compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.
6. The method of claim 5 , further comprising:
supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.
7. The method of claim 5 , wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.
8. The method of claim 5 , wherein aligning the versions of the candidate text region comprises:
receiving ranging data and movement information associated with each version of the candidate text region; and
aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.
9. The method of claim 5 , wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:
computing a median value of pixels in each version of the candidate region; and
combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.
10. A system, comprising:
one or more processing devices; and
a memory storage apparatus in data communication with the one or more processing devices and storing instructions that cause the one or more processing devices to perform operations comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
generating a high dynamic range image using the plurality images;
detecting one or more features in each of one or more regions of the high dynamic range image;
determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.
11. The system of claim 10 , the operations further comprising:
increasing contrast of the high dynamic range image including normalizing pixel values in the high dynamic range image.
12. The system of claim 11 , wherein normalizing pixel values in the high dynamic range image comprises:
computing a mean and variance of pixel values in the high dynamic range image; and
scaling pixel values in the high dynamic range image according to the computed mean and variance.
13. The system of claim 10 , the operations further comprising:
generating a superresolution version of the candidate text region.
14. The system of claim 13 , wherein generating the superresolution version of the candidate text region comprises:
obtaining a plurality of versions of the candidate text region, each version obtained from a corresponding image of the plurality of images;
aligning the versions of the candidate text region to a high resolution grid; and
compositing the aligned versions of the candidate text region to generate the superresolution version of the candidate text region.
15. The system of claim 14 , the operations further comprising:
supersampling the obtained versions of a particular candidate text region from each image of the plurality of images.
16. The system of claim 14 , wherein aligning the versions of the candidate text region comprises aligning pixels of the versions of the candidate text region using block matching.
17. The system of claim 14 , wherein aligning the versions of the candidate text region comprises:
receiving ranging data and movement information associated with each version of the candidate text region; and
aligning the versions of the candidate text region based at least in part on the received ranging data and movement information.
18. The system of claim 14 , wherein compositing the aligned versions of the candidate text region comprises combining pixels from each version of the candidate text region including:
computing a median value of pixels in each version of the candidate region; and
combining the computed median values for corresponding pixels in the aligned versions of the candidate text region.
19. A memory storage apparatus storing instructions that cause the one or more processing devices to perform operations comprising:
receiving a plurality of different images of a first scene, wherein each image has a different exposure level;
generating a high dynamic range image using the plurality images;
detecting one or more features in each of one or more regions of the high dynamic range image;
determining for each region of the high dynamic range image whether the region is a candidate text region potentially containing text based on the detected one or more features; and
generating text by performing optical character recognition on a plurality of the regions determined to contain text.