Synthetic augmentation of document images
A computerized method and system for adding distortions to a computer-generated image of a document stored in an image file. An original computer-generated image file is selected and is processed to generate one or more distorted image files for each original computer-generated image file by selecting one or more augmentation modules from a set of augmentation modules to form an augmentation sub-system. The original computer-generated image file is processed with the augmentation sub-system to generate an augmented image file by altering the original computer-generated image file to add distortions that simulate distortions introduced during scanning of a paper-based representation of a document represented in the original computer-generated image file.
1. A computerized method executable by one or more processors for adding distortions to a computer-generated image of a document stored in an image file, the method comprising:
retrieving an original computer-generated image file from a set of image files;
processing the retrieved computer-generated image file one or more times to generate one or more distorted image files for each original computer-generated image file by,
selecting one or more augmentation modules, executable by the one or more processors, from a set of augmentation modules to form an augmentation sub-system;
processing, with the one or more processors, the original computer-generated image file with the augmentation sub-system to generate an augmented image file by altering the original computer-generated image file to add distortions that simulate distortions introduced during scanning of a paper-based representation of a document represented in the original computer-generated image file; and
storing the augmented image file.
2. The computerized method of claim 1 wherein selecting one or more augmentation modules, executable by the one or more processors, from a set of augmentation modules to form an augmentation sub-system comprises:
randomly selecting two or more augmentation modules from the set of augmentation modules; and
sequentially ordering the two or more augmentation modules to form the augmentation sub-system, executable by the one or more processors, comprising an input to receive the original computer-generated image file into a first of the selected augmentation modules, and an output to provide, by a last of the selected augmentation modules, the augmented image file.
3. The computerized method of claim 2 further comprising:
generating multiple augmentation sub-systems, executable by the one or more processors, where each augmentation sub-system comprises a set of augmentation modules different from each other augmentation sub-system; and
processing, with the one or more processors, the original computer-generated image file with each augmentation sub-system to generate a plurality of augmented image files, each of the plurality of augmented image files simulating a different set of distortions introduced during scanning of a paper-based representation of a document represented in the original computer-generated image file.
4. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that performs an affine transformation on a computer-generated image of a document stored in an image file.
5. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that performs a non-affine transformation on a computer-generated image of a document stored in an image file.
6. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces scan lines, that may added by a scanner when scanning a document, to a computer-generated image of a document stored in an image file.
7. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces blotches, comprising uniformly black irregular spots, to a computer-generated image of a document stored in an image file.
8. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces multiscale uniform noise, comprising simulation of physical effects of paper and ink of a scanned document, to a computer-generated image of a document stored in an image file.
9. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces ink noise, to a computer-generated image of a document stored in an image file.
10. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation, executable by the one or more processors, module that introduces paper noise, to a computer-generated image of a document stored in an image file.
11. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces compression artifacts, to a computer-generated image of a document stored in an image file.
12. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces a computer-generated imitation of handwriting, to a computer-generated image of a document stored in an image file.
13. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces salt and pepper noise lines, to a computer-generated image of a document stored in an image file.
14. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces distortions caused by a gaussian filter, to a computer-generated image of a document stored in an image file.
15. The computerized method of claim 1 wherein the augmentation modules, executable by the one or more processors, comprise:
an augmentation module, executable by the one or more processors, that introduces distortions caused by binarizing, to a computer-generated image of a document stored in an image file.
16. The computerized method of claim 1 further comprising:
providing the augmented image file to a deep neural network as a training set of documents to train the deep neural network.
17. A document processing system comprising:
data storage for storing a first set of original documents that are in image format, and a second set of documents comprising augmented documents in an image format, where each augmented document corresponds to and represents an augmented version of an original document in the first set of documents; and
one or more processors operatively coupled to the data storage and configured to execute instructions that when executed cause the one or more processors to alter a selected original document from the first set of original documents to generate an augmented document for the second set of documents, wherein the augmented document contains distortions that resemble distortions introduced during scanning of a paper based representation of a corresponding first document image, by:
selecting an original document from the first set of original documents;
selecting one or more augmentation modules from a set of augmentation modules to form an augmentation sub-system, executable by the one or more processors, wherein the set of augmentation modules comprises augmentation modules selected from modules that, perform an affine transformation, perform a non-affine transformation, introduce scan lines, introduce blotches, introduce multiscale uniform noise, introduce ink noise, introduce paper noise, introduce compression artifacts, introduce a computer-generated imitation of handwriting, introduce salt and pepper noise lines, introduce distortions caused by a gaussian filter, and introduce distortions caused by binarizing;
processing, with the one or more processors, the original document with the augmentation sub-system to generate an augmented document by altering the original document to add distortions that simulate distortions introduced during scanning of a paper-based representation of a document represented in the original document; and
storing the augmented document to the document storage.
18. The document processing system of claim 17 wherein the one or more processors are configured to execute instructions that when executed cause the one or more processors to perform the operation of selecting one or more augmentation modules from a set of augmentation modules to form an augmentation sub-system, executable by the one or more processors, by randomly selecting one or more augmentation modules from the set of augmentation modules; and
sequentially ordering the one or more augmentation modules to form the augmentation sub-system executable by the one or more processors, comprising an input to receive the original document into a first of the selected augmentation modules, and an output to provide, by a last of the selected augmentation modules, the augmented document.
19. The document processing system of claim 18 wherein the one or more processors are configured to execute instructions that when executed cause the one or more processors to:
generate multiple augmentation sub-systems, where each augmentation sub-system is executable by the one or more processors and comprises a set of augmentation modules different from each other augmentation sub-system; and
process the original document with each augmentation sub-system to generate a plurality of augmented documents, each of the plurality of augmented documents simulating a different set of distortions introduced during scanning of a paper-based representation of a document represented in the original document.
20. The document processing system of claim 19 wherein the one one or more processors are configured to execute instructions that when executed cause the one or more processors to provide the augmented documents in the document storage to a deep neural network as a training set of documents to train the deep neural network.
21. A document processing system comprising:
data storage for storing a first set of original documents that are in image format, and a second set of documents comprising augmented documents in an image format, where each augmented document corresponds to and represents an augmented version of an original document in the first set of documents; and
one or more processors operatively coupled to the data storage and configured to execute instructions that when executed cause the one or more processors to alter a selected original document from the first set of original documents to generate an augmented document for the second set of documents, wherein the augmented document contains distortions that resemble distortions introduced during scanning of a paper based representation of a corresponding first document image, by:
selecting an original document from the first set of original documents;
selecting one or more augmentation modules from a set of augmentation modules to form an augmentation sub-system, executable by the one or more processors, wherein the set of augmentation modules comprises augmentation modules selected from modules that recreate noise generated in flatbed-scanned document images;
processing the original document with the augmentation sub-system to generate an augmented document by altering the original document to add distortions that simulate distortions introduced during scanning of a paper-based representation of a document represented in the original document; and
storing the augmented document to the document storage.