Unsupervised removal of text from images using linear programming for optimal filter design
Techniques are disclosed for removing text from an image of a form document. The text is removed by determining a spectral domain representation of the image and applying a filter to remove the high-frequency components which correspond to the text in the form. An image is reconstructed from the filtered spectral domain representation which maintains the low-frequency components, while deemphasizing or removing the high-frequency components. A shape of the filter applied to the spectral domain representation is determined based on a similarity measure between the image of the form and the reconstructed image.
1. A method for removing text from an image of a form comprising:
generating a matrix representing pixels of the image, wherein the image depicts at least a portion of the form, and wherein the form includes layout elements and content elements;
applying a matrix flattening operation to the matrix;
generating a frequency domain representation of the image by applying a frequency domain transformation to the flattened matrix;
generating a filter to apply to the frequency domain representation;
applying the filter to the frequency domain representation to remove a portion of the frequency domain representation of the image;
generating a reconstructed matrix from the filtered frequency domain representation, wherein a format of the reconstructed matrix is a format of the matrix representing pixels of the image;
generating a reconstructed image of at least the portion of the form by applying an inverse frequency domain transformation to the reconstructed matrix filtered frequency domain representation, wherein the reconstructed image depicts only a portion of the content elements;
matching the reconstructed image to a form template;
generating respective text from the content elements of the image; and
mapping portions of the respective text to a corresponding field of the form template based on a position of the content element in the image.
2. The method of claim 1 , wherein the reconstructed image is matched to the form template by comparing the reconstructed image to each of a plurality of form templates until the matching form template is found.
3. The method of claim 1 , wherein the filter comprises a low-pass filter.
4. The method of claim 1 , wherein the frequency domain transformation is a discrete Fast Fourier Transform.
5. The method of claim 1 , wherein the content elements of the form include text.
6. The method of claim 1 , wherein generating the filter comprises maximizing a similarity measure between the image and the reconstructed image.
7. The method of claim 6 , wherein the similarity measure is a dot product of the frequency domain representation of the image and the reconstructed image.
8. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, performs an operation for removing text from an image of a form, the operation comprising:
applying a matrix flattening operation to the matrix;
generating a frequency domain representation of the image by applying a frequency domain transformation to the flattened matrix;
generating a filter to apply to the frequency domain representation;
applying the filter to the frequency domain representation to remove a portion of the frequency domain representation of the image;
generating a reconstructed matrix from the filtered frequency domain representation, wherein a format of the reconstructed matrix is a format of the matrix representing pixels of the image;
generating a reconstructed image of at least the portion of the form by applying an inverse frequency domain transformation to the reconstructed matrix filtered frequency domain representation, wherein the reconstructed image depicts only a portion of the content elements;
matching the reconstructed image to a form template;
generating respective text from the content elements of the image; and
mapping portions of the respective text to a corresponding field of the form template based on a position of the content element in the image.
9. The non-transitory computer-readable medium of claim 8 , wherein generating the filter comprises maximizing a similarity measure between the image and the reconstructed image.
10. The non-transitory computer-readable medium of claim 9 , wherein the similarity measure is a dot product of the frequency domain representation of the image and the reconstructed image.
11. The non-transitory computer-readable medium of claim 8 , wherein the two dimensional frequency domain transformation is a two dimensional discrete Fourier transform.
12. The non-transitory computer-readable medium of claim 8 , wherein the reconstructed image is matched to the form template by comparing the reconstructed image to each of a plurality of form templates until the matching form template is found.
13. A system, comprising:
a processor; and
memory hosting an application, which, when executed on the processor, performs an operation for removing text from an image of a form, the operation comprising:
applying a matrix flattening operation to a matrix to produce a flattened matrix;
generating a frequency domain representation of the image by applying a frequency domain transformation to the flattened matrix;
generating a filter to apply to the frequency domain representation;
applying the filter to the frequency domain representation to remove a portion of the frequency domain representation of the image;
generating a reconstructed matrix from the filtered frequency domain representation, wherein a format of the reconstructed matrix is a format of the matrix representing pixels of the image;
generating a reconstructed image of at least the portion of the form by applying an inverse frequency domain transformation to the reconstructed matrix filtered frequency domain representation, wherein the reconstructed image depicts only a portion of the content elements;
matching the reconstructed image to a form template;
generating text from the content elements of the image; and
mapping portions of the text to a corresponding field of the form template based on a position of the content element in the image.
14. The system of claim 13 , wherein generating the filter comprises maximizing a similarity measure between the image and the reconstructed image.
15. The system of claim 14 , wherein the similarity measure is a dot product of the frequency domain representation of the image and the reconstructed image.
16. The system of claim 13 , wherein the two dimensional frequency domain transformation is a two dimensional discrete Fourier transform.
17. The system of claim 13 , wherein the reconstructed image is matched to the form template by comparing the reconstructed image to each of a plurality of form templates until the matching form template is found.
18. The system of claim 13 , wherein the filter comprises a low-pass filter.
19. The system of claim 13 , wherein the content elements of the form include text.
20. The non-transitory computer-readable medium of claim 8 , wherein the filter comprises a low-pass filter.