Systems and methods for processing mobile images to identify and extract content from forms
Systems and methods for matching a received image with a template image are disclosed herein. Such systems and methods can advantageously enable an image captured by a mobile device (such as a smartphone or digital camera) to be correctly identified by the processing application. In some embodiments, the received image is first resized in one or both dimensions in order to match or approximately match the dimensions of a given template. The received image and template image can then be superimposed. Next, an optimal translative transformation value can be calculated in order to generate a confidence level for the current possible match. After confidence levels for each template are generated and recorded, the template with the highest confidence level can be selected as the best match for the received image.
1. A non-transitory computer readable medium containing instructions which, when executed by a computer, perform a process of resizing a dimension of a received image to match or approximately match a corresponding dimension of template image, the process comprising:
identifying a first set of lines in a received image and a second set of lines in a template image;
selecting a first subset of lines from the first set of lines, and selecting a second subset of lines from the second set of lines, wherein each line of the first subset is longer than a first predetermined minimum length, and wherein each line of the subset is longer than a second predetermined minimum length;
calculating distances between subsequent lines in the first subset, and calculating distances between subsequent lines in the second subset;
calculating ratios between successive distances in the first subset, and calculating ratios between subsequent distances in the second subset;
pairing ratios in the first subset with ratios in the second subset if the differences between two ratios exceeds a predetermined threshold of similarity;
for each matching pair of ratios, calculating a ratio similarity coefficient that is less than or equal to a predetermined value;
storing each ratio similarity coefficient in a similarity coefficient vector;
sorting the similarity coefficient vector;
calculating a hypothesis similarity coefficient based at least in part upon the median of the sorted similarity coefficient vector;
selecting the hypothesis similarity coefficient with the greatest value from a set of all hypothesis similarity coefficients; and
resizing the dimension of the received image based at least in part upon the selected hypothesis similarity coefficient.
2. The computer readable medium of claim 1 , wherein the first and second sets of lines consist of horizontal lines.
3. The computer readable medium of claim 1 , wherein the first and second sets of lines consist of vertical lines.
4. The computer readable medium of claim 1 , wherein the first predetermined minimum length is 70% of the length of the longest line of the first set of lines.
5. The computer readable medium of claim 1 , wherein the second predetermined minimum length is 70% of the length of the longest line of the second set of lines.
6. The computer readable medium of claim 1 , wherein a ratio of the first subset is paired to a single ratio of the second subset.
7. The computer readable medium of claim 1 , wherein no more than a single ratio of the first subset is paired to a ratio of the second subset.
8. The computer readable medium of claim 1 , wherein the process is repeated a second time, but while reversing the sequence of lines contained within the first subset in order to account for vertical inversion of the received image.
9. The computer readable medium of claim 1 , wherein resizing the dimension of the received image is based at least in part upon a determined ratio, the determined ratio comprising a coordinate difference between a first and last line of the received image, and a coordinate difference between a first and last line of the template image.
10. The computer readable medium of claim 1 , wherein the dimension is the height of the received image.
11. The computer readable medium of claim 1 , wherein the dimension is the width of the received image.
12. A non-transitory computer readable medium containing instructions which, when executed by a computer, perform a process of calculating an optimal translation transformation, the process comprising: identifying data items within a received image and within a template image; inflating the pixels of one or more of the data items according to their respective aspect ratios; creating a bounding rectangle for each inflated item; for each corresponding pair of data items, calculating the difference between the geometric centers of each respective bounding rectangle; creating a weighted sum of translation transformations, wherein the weighted sum comprises the values of each calculated difference; and determining a registration confidence level based at least in part upon the weighted sum.
13. The computer readable medium of claim 12 , wherein the data items comprise horizontal lines, vertical lines, text, and boxes.
14. The computer readable medium of claim 13 , wherein the boxes are identified based at least in part upon previously detected horizontal and vertical lines.
15. The computer readable medium of claim 13 , wherein if the data item is a horizontal line or text, inflating the pixels comprises inflating the data item for a number of pixels in a vertical direction.
16. The computer readable medium of claim 13 , wherein if the data item is a vertical line, inflating the pixels comprises inflating the data item for a number of pixels in a horizontal direction.
17. The computer readable medium of claim 13 , wherein if the data item is a box, inflating the pixels comprises inflating the data item for a number of pixels in a both a horizontal and a vertical direction.
18. The computer readable medium of claim 13 , wherein determining the registration confidence level is determined based at least in part upon calculating a weighted sum of cross-correlation coefficients.
19. A non-transitory computer readable medium containing instructions which, when executed by a computer, performs a process of matching a received image to a corresponding template image, the process comprising: receiving an image; for a set of template images remaining: resizing one or both dimensions of the received image to match or approximately match a corresponding dimension of the current template image, calculating the optimal translation transformation of the received image relative to the current template image, recording a calculated confidence level that is based at least in part on the optimal translation transformation; and selecting the template image which has the highest confidence level.
20. The computer readable medium of claim 19 , wherein resizing one or both dimensions of the received image to match or approximately match a corresponding dimension of the current template image further comprises:
identifying a first set of lines in the received image and a second set of lines in the template image;
selecting a first subset of lines from the first set of lines, and selecting a second subset of lines from the second set of lines, wherein each line of the first subset is longer than a first predetermined minimum length, and wherein each line of the subset is longer than a second predetermined minimum length;
calculating distances between subsequent lines in the first subset, and calculating distances between subsequent lines in the second subset;
calculating ratios between successive distances in the first subset, and calculating ratios between subsequent distances in the second subset; pairing ratios in the first subset with ratios in the second subset if the differences between two ratios exceeds a predetermined threshold of similarity;
for each matching pair of ratios, calculating a ratio similarity coefficient that is less than or equal to a predetermined value;
storing each ratio similarity coefficient in a similarity coefficient vector;
sorting the similarity coefficient vector;
calculating a hypothesis similarity coefficient based at least in part upon the median of the sorted similarity coefficient vector;
selecting the hypothesis similarity coefficient with the greatest value from a set of all hypothesis similarity coefficients calculated; and
resizing a dimension of the received image based at least in part upon the selected hypothesis similarity coefficient.
21. The computer readable medium of claim 19 , wherein calculating the optimal translation transformation of the received image relative to the current template image further comprises: identifying data items within the received image and within the template image; inflating the pixels of one or more of the data items according to their respective aspect ratios; creating a bounding rectangle for each inflated item; for each corresponding pair of data items, calculating the difference between the geometric centers of each respective bounding rectangle; creating a weighted sum of translation transformations, wherein the weighted sum comprises the values of each calculated difference; and determining a registration confidence level based at least in part upon the weighted sum.