System and method for forms classification by line-art alignment
A system and method to classify forms. An image representing a form of an unknown document type is received. The image includes line-art. Further, a plurality of template models corresponding to a plurality of different document types is received. The plurality of different document types is intended to include the correct document type of the unknown document. A subset of the plurality of template models are selected as candidate template models. The candidate template models include line-art junctions best matching line-art junctions of the received image. One of the candidate template models is selected as a best candidate template model. The best candidate template model includes horizontal and vertical lines best matching horizontal and vertical lines of the received image, respectively, aligned to the best candidate template model.
1. A method for classifying forms, said method comprising:
receiving by at least one processor an image representing a form of an unknown document type, the image including line-art;
receiving by the at least one processor a plurality of template models corresponding to a plurality of different document types;
selecting by the at least one processor a subset of the plurality of template models as candidate template models, the candidate template models including line-art junctions best matching line-art junctions of the received image; and,
selecting by the at least one processor one of the candidate template models as a best candidate template model, the best candidate template model including horizontal and vertical lines best matching horizontal and vertical lines of the received image, respectively, aligned to the best candidate template model.
2. The method according to claim 1 , wherein the selecting the subset of the plurality of template models includes:
generating a fingerprint histogram of the received image;
determining, for each of the plurality of template models, a match quality between the fingerprint histogram of the received image and a fingerprint histogram of the template model; and,
determining the candidate template models based on the determined match qualities, the candidate template models including template models of the plurality of template models with best match qualities.
3. The method according to claim 2 , wherein the generation of the fingerprint histogram of the received image includes:
extracting line-art junctions from received image;
generating fingerprints from the extracted line-art junctions, the extracted line-art junctions being keypoints of the fingerprints;
for each of the generated fingerprints, determining a match count indicating the number of times the fingerprint appears in the received image; and,
combining the generated fingerprints and the determined match counts into the fingerprint histogram of the received image.
4. The method according to claim 1 , wherein the selecting the one of the candidate template models includes:
extracting horizontal and vertical lines from the received image;
for each of the candidate template models:
aligning the extracted horizontal and vertical lines to horizontal and vertical lines of the candidate template model, respectively;
determining a forward match score indicating a match quality of the aligned horizontal and vertical lines to the horizontal and vertical lines of the candidate template model, respectively; and,
determining a backward match score indicating a match quality of the horizontal and vertical lines of the candidate template model to the aligned horizontal and vertical lines, respectively; and,
determining the best candidate template model based on the forward and backward match scores.
5. The method according to claim 4 , wherein the aligning includes:
comparing the extracted horizontal and vertical lines to the horizontal and vertical lines of the candidate template model, respectively, to determine a list of corresponding points;
determining a transformation matrix registering received-image points of the list of corresponding points to candidate-template-model points of the list of corresponding points; and,
transforming the extracted horizontal and vertical lines using the transformation matrix to align the extracted horizontal and vertical lines.
6. The method according to claim 5 , wherein the determining the transformation matrix only considers horizontal positioning of points of the list of corresponding points corresponding to vertical lines and only considers vertical positioning of points of the list of corresponding points corresponding to horizontal lines.
7. The method according to claim 4 , wherein the first match score and the second score are determined using distance transforms of the horizontal and vertical lines of the candidate template model and distance transforms of the aligned horizontal and vertical lines, respectively.
8. The method according to claim 1 , further including:
removing background form information of the received image using the best candidate template model to create a background-subtracted image; and,
extracting data from the background-subtracted image using zonal optical character recognition.
9. The method according to claim 1 , wherein the horizontal and vertical lines of the best candidate template model are continuous and the horizontal and vertical lines of the received image are continuous.
10. The method according to claim 1 , wherein the horizontal and vertical lines of the best candidate template model correspond to line-art of the best candidate template model and the horizontal and vertical lines of the received image correspond to the line-art of the received image.
11. A system for classifying forms, said system comprising:
at least one processor programmed to:
receive an image representing a form of an unknown document type, the image including line-art;
receive a plurality of template models corresponding to a plurality of different document types;
select a subset of the plurality of template models as candidate template models, the candidate template models including line-art junctions best matching line-art junctions of the received image;
aligning continuous horizontal and vertical lines of the line-art to continuous horizontal and vertical lines of the candidate template models, respectively; and,
select one of the candidate template models as a best candidate template model, the best candidate template model including continuous horizontal and vertical lines best matching continuous horizontal and vertical lines of the line-art, respectively, aligned to the best candidate template model.
12. The system according to claim 11 , wherein the selecting the set of the plurality of template models includes:
generating a fingerprint histogram of the received image;
determining, for each of the plurality of template models, a match quality between the fingerprint histogram of the received image and a fingerprint histogram of the template model; and,
determining the candidate template models based on the determined match qualities, the candidate template models including template models of the plurality of template models with best match qualities.
13. The system according to claim 12 , wherein the generation of the fingerprint histogram of the received image includes:
extracting line-art junctions from received image;
generating fingerprints from the extracted line-art junctions, the extracted line-art junctions being keypoints of the generated predetermined number of fingerprints;
for each of the generated fingerprints, determining a match count indicating the number of times the fingerprint appears in the received image; and,
combining the generated predetermined number of fingerprints and the match counts into the fingerprint histogram of the received image.
14. The system according to claim 11 , wherein the selecting the one of the candidate template models includes:
extracting the horizontal and vertical lines from the received image;
for each of the candidate template models:
aligning the extracted horizontal and vertical lines to horizontal and vertical lines of the candidate template model, respectively;
determining a forward match score indicating a match quality of the aligned horizontal and vertical lines to the horizontal and vertical lines of the candidate template model, respectively; and,
determining a backward match score indicating a match quality of the horizontal and vertical lines of the candidate template model to the aligned horizontal and vertical lines, respectively; and,
determining the best candidate template model based on the first and second match scores.
15. The system according to claim 14 , wherein the aligning includes:
comparing the extracted horizontal and vertical lines to the horizontal and vertical lines of the candidate template model, respectively, to determine a list of corresponding points;
determining a transformation matrix registering received-image points of the list of corresponding points to candidate-template-model points of the list of corresponding points; and,
transforming the extracted horizontal and vertical lines using the transformation matrix to align the extracted horizontal and vertical lines.
16. The system according to claim 15 , wherein the determining the transformation matrix only considers horizontal positioning of points of the list of corresponding points corresponding to vertical lines and only considers vertical positioning of points of the list of corresponding points corresponding to horizontal lines.
17. The system according to claim 14 , wherein the first match score and the second score are determined using distance transforms of the horizontal and vertical lines of the candidate template model and distance transforms of the aligned horizontal and vertical lines, respectively.
18. The system according to claim 11 , wherein the at least one processor is further programmed to:
remove background form information of the received image using the best candidate template model to create a background-subtracted image; and,
extract data from the background-subtracted image using zonal optical character recognition.
19. A system for classifying forms, said system comprising:
at least one processor programmed to:
receive an image representing a form of an unknown document type, the image including line-art;
receive a plurality of template models corresponding to a plurality of different document types;
extract horizontal and vertical lines from the received image;
for each of the plurality of template models:
align extracted horizontal and vertical lines to horizontal and vertical lines of the template model, respectively;
determine a forward match score indicating a match quality of the aligned horizontal and vertical lines to the horizontal and vertical lines of the template model, respectively; and,
determine a backward match score indicating a match quality of the horizontal and vertical lines of the template model to the aligned horizontal and vertical lines, respectively; and,
determine a best one of the plurality of template models based on the first and second match scores.
20. The system according to claim 19 , wherein the aligning includes:
comparing the extracted horizontal and vertical lines to the horizontal and vertical lines of the candidate template model, respectively, to determine a list of corresponding points;
determining a transformation matrix registering received-image points of the list of corresponding points to candidate-template-model points of the list of corresponding points; and,
transforming the extracted horizontal and vertical lines using the transformation matrix to align the extracted horizontal and vertical lines.
21. The system according to claim 19 , wherein the first match score and the second score are determined using distance transforms of the horizontal and vertical lines of the candidate template model and distance transforms of the aligned horizontal and vertical lines, respectively.
22. The system according to claim 19 , wherein the processor is further programmed to:
remove background form information of the received image using the best template model to create a background-subtracted image; and,
extract data from the background-subtracted image using zonal optical character recognition.