Iterative recognition-guided thresholding and data extraction
Techniques for binarization and extraction of information from image data are disclosed. The inventive concepts include independently binarizing portions of the image data on the basis of individual features, e.g. per connected component, and using multiple different binarization thresholds to obtain the best possible binarization result for each portion of the image data. Determining the quality of each binarization result may be based on attempted recognition and/or extraction of information therefrom. Independently binarized portions may be assembled into a contiguous result. In one embodiment, a method includes: identifying a region of interest within a digital image; generating a plurality of binarized images based on the region of interest using different binarization thresholds; subjecting the region of interest within a digital image to a plurality of thresholding and extraction iterations; and extracting data from some or all of the plurality of binarized images. Corresponding systems and computer program products are disclosed.
1. A computer-implemented method, comprising:
identifying a region of interest within a digital image;
subjecting the region of interest within the digital image to a plurality of thresholding and extraction iterations to generate a plurality of binarized images, wherein some or all of the binarized images are generated using different binarization thresholds selected from a plurality of binarization thresholds;
extracting data from some or all of the plurality of binarized images;
wherein the extracted data comprises one or more connected components represented in the plurality of binarized images;
wherein one or more of the connected components overlap or are obscured by one or more unique background elements such that no single binarization threshold applied to a region encompassing the one or more connected components can identify the one or more of the connected components that overlap or are obscured by the one or more unique background elements;
wherein extracting the data comprises estimating an identity of some or all of the one or more connected components within one or more of the plurality of binarized images;
wherein the estimated identity of some or all of the one or more connected components comprises a character identity, a location of the connected component, a size of the connected component, a shape of the connected component, and/or a color of the connected component; and
the method further comprising determining a confidence of the estimated identity of some or all of the one or more connected components, wherein determining the confidence of the estimated identity of some or all of the one or more connected components comprises:
comparing the estimated identity of each respective one of the one or more connected components for which the identity was estimated with an expected identity of the respective one of the one or more connected components;
comparing an estimated location of each respective one of the one or more connected components for which the identity was estimated with an expected location of the respective one of the one or more connected components; and/or
classifying some or all of the connected components for which the identity was estimated based on image features.
2. The computer-implemented method as recited in claim 1 , wherein the region of interest depicts a driver license, and wherein the data are extracted from the driver license.
3. The computer-implemented method as recited in claim 1 , wherein the region of interest depicts a credit card, and wherein the data are extracted from the credit card.
4. The computer-implemented method as recited in claim 1 , wherein the digital image is characterized by an insufficient contrast causing boundaries between foreground and background elements of the digital image to be obscured.
5. The computer-implemented method as recited in claim 1 , wherein the digital image is characterized by an excessive contrast causing single elements of the digital image to be broken into constituent elements.
6. The computer-implemented method as recited in claim 1 , wherein extracting the data is performed on a per-component basis for at least some of the one or more connected components.
7. The computer-implemented method as recited in claim 1 , comprising:
validating the extracted data; and
inferring a classification of an object depicted in the digital image based on validating the extracted data.
8. A computer program product, comprising a non-transitory computer readable medium having embodied therewith computer readable program instructions configured to cause a processor, upon execution thereof, to:
identify, using the processor, a region of interest within a digital image;
subject the region of interest within the digital image to a plurality of thresholding and extraction iterations to generate a plurality of binarized images, wherein some or all of the binarized images are generated using different binarization thresholds selected from a plurality of binarization thresholds; and
extract, using the processor, data from some or all of the plurality of binarized images;
wherein extracting the data from some or all of the plurality of binarized images comprises:
generating, using the processor, at least one sequence of candidate extraction results for each grouping of one or more connected components depicted within the region of interest;
determining, using the processor, an optimal extraction result within each sequence of candidate extraction results; and
assembling, using the processor, all of the optimal extraction results into a single string of the one or more connected components; and
wherein determining the optimal extraction result within each sequence of candidate extraction results comprises selecting, using the processor, one extraction result within each sequence of candidate extraction results so as to minimize intensity differences between the optimal extraction results assembled into the single string;
wherein at least some of the connected components are text characters;
wherein extracting the data comprises estimating an identity of some or all of the one or more connected components within one or more of the plurality of binarized images;
wherein the estimated identity of some or all of the one or more connected components comprises a character identity, a location of the connected component, a size of the connected component, a shape of the connected component, and/or a color of the connected component; and
wherein the program instructions are further configured to cause the processor, upon execution thereof, to determine a confidence of the estimated identity of some or all of the one or more connected components, wherein determining the confidence of the estimated identity of some or all of the one or more connected components comprises:
comparing the estimated identity of each respective one of the one or more connected components for which the identity was estimated with an expected identity of the respective one of the one or more connected components;
comparing an estimated location of each respective one of the one or more connected components for which the identity was estimated with an expected location of the respective one of the one or more connected components; and/or
classifying some or all of the one or more connected components for which the identity was estimated based on image features.
9. The computer program product as recited in claim 8 , wherein the region of interest depicts a driver license, and wherein the data are extracted from the driver license.
10. The computer program product as recited in claim 8 , wherein the region of interest depicts a credit card, and wherein the data are extracted from the credit card.
11. The computer program product as recited in claim 8 , wherein the digital image is characterized by an insufficient contrast causing boundaries between foreground and background elements of the digital image to be obscured.
12. The computer program product as recited in claim 8 , wherein the digital image is characterized by an excessive contrast causing single elements of the digital image to be broken into constituent elements.
13. The computer program product as recited in claim 8 , wherein extracting the data is performed on a per-component basis for at least some of the one or more connected components.
14. The computer program product as recited in claim 8 , comprising: validating the extracted data; and
inferring a classification of an object depicted in the digital image based on validating the extracted data.
15. The computer program product as recited in claim 8 , wherein each sequence of candidate extraction results comprises a plurality of candidate extraction results each corresponding to the same grouping of one or more of the connected components depicted within the region of interest; and
wherein each of the plurality of candidate extraction results in each respective sequence of candidate extraction results corresponds to a different one of the plurality of binarization thresholds.
16. A system, comprising: a processor; and logic integrated with and/or executable by the processor to cause the processor to:
identify a region of interest within a digital image;
subject the region of interest within the digital image to a plurality of thresholding and extraction iterations to generate a plurality of binarized images, wherein some or all of the binarized images are generated using different binarization thresholds selected from a plurality of binarization thresholds;
extract data from some or all of the plurality of binarized images;
wherein extracting the data from some or all of the plurality of binarized images comprises:
generating at least one sequence of candidate extraction results for each grouping of one or more connected components depicted within the region of interest;
determining an optimal extraction result within each sequence of candidate extraction results; and
assembling all of the optimal extraction results into a single string of the one or more connected components; and
wherein determining the optimal extraction result within each sequence of candidate extraction results comprises selecting one extraction result within each sequence of candidate extraction results so as to minimize intensity differences between the optimal extraction results assembled into the single string;
wherein at least some of the connected components are text characters;
wherein extracting the data comprises estimating an identity of some or all of the one or more connected components within one or more of the plurality of binarized images;
wherein the estimated identity of some or all of the one or more connected components comprises a character identity, a location of the connected component, a size of the connected component, a shape of the connected component, and/or a color of the connected component; and
wherein the logic is further integrated with and/or executable by the processor to cause the processor to determine a confidence of the estimated identity of some or all of the one or more connected components, wherein determining the confidence of the estimated identity of some or all of the one or more connected components comprises:
comparing the estimated identity of each respective one of the one or more connected components for which the identity was estimated with an expected identity of the respective one of the one or more connected components;
comparing an estimated location of each respective one of the one or more connected components for which the identity was estimated with an expected location of the respective one of the one or more connected components; and/or
classifying some or all of the one or more connected components for which the identity was estimated based on image features.