Systems and methods for detection and high-quality capture of documents on a cluttered tabletop with an automatically controlled camera
Described are systems and methods for recognizing paper documents on a tabletop using an overhead camera mounted on pan-tilt servos. The described automated system first finds paper documents on a cluttered desk based on a text probability map, constructed using multiple images acquired at fixed grid positions, and then captures a sequence of high-resolution overlapping frames of the located document(s), which are then fused together and perspective-rectified, using computed homography, to reconstruct a high quality and fronto-parallel document image that is of sufficient quality required for optical character recognition. The extracted textual information may be used, for example, for indexing and search, document repository and/or language translation applications.
1. A computer-implemented method being performed in a computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, the computer-implemented method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface;
b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface;
c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and
d. extracting the text using the second captured plurality of images.
2. The computer-implemented method of claim 1 , wherein the processing unit is configured to cause the turret to move the camera during the scanning and capturing.
3. The computer-implemented method of claim 1 , wherein in a. the first plurality of images is acquired by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the first plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was acquired.
4. The computer-implemented method of claim 3 , wherein a. further comprises, for each image in the first plurality of images of the surface, computing feature points and a text response map.
5. The computer-implemented method of claim 4 , wherein the each text response map is computed using a probability histogram, the probability histogram being pre-computed based on the feature points.
6. The computer-implemented method of claim 4 , wherein a. further comprises stitching the text response maps corresponding to the first plurality of images into a single text response map based on the plurality of fixed positions corresponding to the first plurality of images.
7. The computer-implemented method of claim 6 , wherein a. further comprises detecting text blobs in the single text response map.
8. The computer-implemented method of claim 7 , wherein a. further comprises identifying at least one large rectangular shaped blob from the detected text blobs as the location of the text on the surface.
9. The computer-implemented method of claim 1 , wherein in b. the second plurality of images is captured by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the second plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was captured.
10. The computer-implemented method of claim 9 , wherein the determined location of the text on the surface is a location of a text blob and wherein the predetermined path is a center line through a bounding box of the corresponding text blob.
11. The computer-implemented method of claim 9 , wherein the images of the second plurality of images overlap with one another.
12. The computer-implemented method of claim 11 , further comprising stitching and fusing the images of the second plurality of images to obtain a second stitched image.
13. The computer-implemented method of claim 12 , further comprising fitting lines around a boundary of a text blob in the second stitched image and estimating vanishing points.
14. The computer-implemented method of claim 12 , further comprising performing a perspective rectification on the second stitched image based on a computed homography.
15. The computer-implemented method of claim 14 , wherein in d. the text is extracted using an optical character recognition performed on the perspective rectified second stitched image.
16. A non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in a computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, cause the computerized system to perform a method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface;
b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface;
c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and
d. extracting the text using the second captured plurality of images.
17. The non-transitory computer-readable medium of claim 16 , wherein the set of computer-executable instructions configures the processing unit to cause the turret to move the camera during the scanning and capturing.
18. The non-transitory computer-readable medium of claim 16 , wherein in a. the first plurality of images is acquired by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the first plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was acquired.
19. The non-transitory computer-readable medium of claim 16 , wherein a. further comprises, for each image in the first plurality of images of the surface, computing feature points and a text response map.
20. A computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, the memory storing a set of computer-executable instructions causing the computerized system to perform a method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface;
b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface;
c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and
d. extracting the text using the second captured plurality of images.