Data extraction from form images
An image processing system accesses an image of a completed form document. The image of the form document includes one or more features, such as form text, at particular locations within the image. The image processing system accesses a template of the form document and computes a rotation and zoom of the image of the form document relative to the template of the form document based on the locations of the features within the image of the form document relative to the locations of the corresponding features within the template of the form document. The image processing system performs a rotation operation and a zoom operation on the image of the form document, and extracts data entered into fields of the modified image of the form document. The extracted data can be then accessed or stored for subsequent use.
1. A method comprising:
accessing an image of a form document comprising a plurality of fields, wherein each field includes or is adjacent to a set of form text associated with the field, and wherein each field includes entered text;
computing a distance between a first set of form text and a second set of form text within the image of the form document;
computing an angle of a line intersecting the first set of form text and the second set of form text; and
modifying the image of the form document by:
performing a zoom operation on the image of the form document based on a comparison of the computed distance and an expected distance between the first set of form text and the second set of form text to produce a zoomed form document image; and
performing a rotation operation on the zoomed form document image based on a comparison of the computed angle and an expected angle between the first set of form text and the second set of form text to produce a modified form document image.
2. The method of claim 1 , wherein the first set of form text is associated with a first bounding box around the first set of form text and the second set of form text is associated with a second bounding box around the second set of form text within the image.
3. The method of claim 2 , wherein the computed distance comprises a distance between a corner of the first bounding box and a corresponding corner of a second bounding box.
4. The method of claim 1 , wherein performing the zoom operation comprises:
determining a difference between the computed distance and the expected distance; and
if the difference is greater than a threshold difference, modifying dimensions of the image by a factor selected based on the determined difference.
5. The method of claim 1 , wherein performing the rotation operation comprises:
determining a difference between the computed angle and the expected angle; and
if the difference is greater than a threshold difference, modifying an orientation of the image by a factor selected based on the determined difference.
6. The method of claim 1 , further comprising:
retrieving entered text from a field of the modified form document image;
identifying a field of the form document corresponding to the field of the modified form document image; and
modifying a database field corresponding to the identified field of the form document to include the retrieved entered text.
7. The method of claim 1 , further comprising:
automatically scanning the entered text of each field from the modified form document image.
8. The method of claim 1 , wherein computing a second distance between the image coordinates comprises applying a Hough transform to the image coordinates.
9. A non-transitory computer-readable storage medium containing computer program code that, when executed by a processor, causes the processor to perform steps comprising:
accessing an image of a form document comprising a plurality of fields, wherein each field includes or is adjacent to a set of form text associated with the field, and wherein each field includes entered text;
computing a distance between a first set of form text and a second set of form text within the image of the form document;
computing an angle of a line intersecting the first set of form text and the second set of form text; and
modifying the image of the form document by:
performing a zoom operation on the image of the form document based on a comparison of the computed distance and an expected distance between the first set of form text and the second set of form text to produce a zoomed form document image; and
performing a rotation operation on the zoomed form document image based on a comparison of the computed angle and an expected angle between the first set of form text and the second set of form text to produce a modified form document image.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the first set of form text is associated with a first bounding box around the first set of form text and the second set of form text is associated with a second bounding box around the second set of form text within the image.
11. The non-transitory computer-readable storage medium of claim 10 , wherein the computed distance comprises a distance between a corner of the first bounding box and a corresponding corner of a second bounding box.
12. The non-transitory computer-readable storage medium of claim 9 , wherein performing the zoom operation comprises:
determining a difference between the computed distance and the expected distance; and
if the difference is greater than a threshold difference, modifying dimensions of the image by a factor selected based on the determined difference.
13. The non-transitory computer-readable storage medium of claim 9 , wherein performing the rotation operation comprises:
determining a difference between the computed angle and the expected angle; and
if the difference is greater than a threshold difference, modifying an orientation of the image by a factor selected based on the determined difference.
14. The non-transitory computer-readable storage medium of claim 9 , wherein the program code, when executed by the processor, causes the processor to perform further steps comprising:
retrieving entered text from a field of the modified form document image;
identifying a field of the form document corresponding to the field of the modified form document image; and
modifying a database field corresponding to the identified field of the form document to include the retrieved entered text.
15. A system comprising:
a hardware processor; and
a non-transitory computer-readable medium containing instructions that, when executed by the hardware processor, cause the hardware processor to:
accessing an image of a form document comprising a plurality of fields, wherein each field includes or is adjacent to a set of form text associated with the field, and wherein each field includes entered text;
computing a distance between a first set of form text and a second set of form text within the image of the form document;
computing an angle of a line intersecting the first set of form text and the second set of form text; and
modifying the image of the form document by:
performing a zoom operation on the image of the form document based on a comparison of the computed distance and an expected distance between the first set of form text and the second set of form text to produce a zoomed form document image; and
performing a rotation operation on the zoomed form document image based on a comparison of the computed angle and an expected angle between the first set of form text and the second set of form text to produce a modified form document image.
16. The system of claim 15 , wherein the first set of form text is associated with a first bounding box around the first set of form text and the second set of form text is associated with a second bounding box around the second set of form text within the image.
17. The system of claim 16 , wherein the computed distance comprises a distance between a corner of the first bounding box and a corresponding corner of a second bounding box.
18. The system of claim 15 , wherein performing the zoom operation comprises:
determining a difference between the computed distance and the expected distance; and
if the difference is greater than a threshold difference, modifying dimensions of the image by a factor selected based on the determined difference.
19. The system of claim 15 , wherein performing the rotation operation comprises:
determining a difference between the computed angle and the expected angle; and
if the difference is greater than a threshold difference, modifying an orientation of the image by a factor selected based on the determined difference.
20. The system of claim 15 , further containing instructions that cause the hardware processor to:
retrieve entered text from a field of the modified form document image;
identify a field of the form document corresponding to the field of the modified form document image; and
modify a database field corresponding to the identified field of the form document to include the retrieved entered text.