Document image quality detection
Techniques are disclosed relating to automatically determining image quality for images of documents. In some embodiments, a computer system receives an image of a document captured at a user computing device. Using a neural network, the computer system analyzes the image to determine whether the image satisfies a quality threshold, where the analyzing includes determining whether one or more features in the image used in an authentication process are obscured. The computer system transmits, to the user computing device, a quality result, where the quality result is generated based on an image classification output by the neural network. Automatically determining whether a received image of a document satisfies a quality threshold may advantageously improve the chances of a system being able to complete an authentication process quickly, which in turn may improve user experience while reducing fraudulent activity.
1. A method, comprising:
receiving, by a server computer system, an image of a document captured at a user computing device;
analyzing, by the server computer system using a neural network, the image to determine whether the image satisfies a quality threshold, wherein the analyzing using the neural network includes:
extracting, via an inception portion of the neural network, a primary set of user identifying features from the image;
selecting, via a self-attention portion of the neural network, a subset of the primary set of user identifying features from the image to be weighted greater than other user identifying features in the set of user identifying features; and
determining, via a classification portion of the neural network based on the weighting of the subset of user identifying features, whether one or more of the subset of user identifying features in the image used in a user authentication process are obscured; and
transmitting, by the server computer system to the user computing device, a quality result, wherein the quality result is generated based on an image classification output by the neural network.
2. The method of claim 1 , wherein the neural network includes at least one self-attention block in the self-attention portion for generating a set of attention weight maps from a set of features extracted from the image, and wherein the set of attention weight maps indicates the one or more user identifying features in the image to be used in the user authentication process.
3. The method of claim 2 , wherein the neural network further includes a convolutional block that extracts features from the image to generate a set of feature maps for the image and at least one inception block that extracts features from the image at a higher level of abstraction than the convolutional block to generate a set of down-sampled high-abstraction feature maps.
4. The method of claim 3 , wherein the neural network further includes a second self-attention block that generates a set of attention weight maps from the set of down-sampled high-abstraction feature maps.
5. The method of claim 3 , wherein the neural network is a convolutional neural network (CNN), and wherein the neural network includes a fully connected layer with a plurality of neurons that processes the set of down-sampled high-abstraction feature maps prior to generating a binary classification for the image via a classification layer.
6. The method of claim 1 , wherein the analyzing includes identifying whether the image includes one or more of the following: a blur, a glare, a reflection, an obstructing object, a partial document, a low-resolution document, an invalid document, and no document.
7. The method of claim 1 , further comprising:
in response to the image classification output by the neural network indicating a quality image, extracting, by the server computer system, one or more features to be used in the user authentication process, wherein the extracting is performed using a computer vision model; and
performing, by the server computer system using the one or more extracted features, the user authentication process.
8. The method of claim 1 , further comprising, prior to the analyzing, training, by the server computer system, the neural network using a set of training data generated by performing one or more image augmentation operations of the following augmentation operations on a set of existing images of documents: rotating, cropping, blurring, distorting, and adding a glare;
wherein the set of training data includes both the set of existing images of documents and a set of augmented images generated by performing the one or more image augmentation operations.
9. The method of claim 8 , wherein the set of training data is further generated by labeling respective images in the set of training data, wherein the labeling includes:
displaying, to a plurality of users via respective user interfaces, images included in the set of training data;
receiving, from the plurality of users via the respective user interfaces, suggested labels for the images, wherein the labels specify a binary quality classification for the image; and
assigning, based on comparing a total number of different types of suggested labels and a consensus threshold, final labels for the images.
10. A non-transitory computer-readable medium having instructions stored thereon that are executable by a server computer system to perform operations comprising:
receiving an image of a document captured at a user computing device;
analyzing, using a machine learning model, the image to determine whether the image satisfies a quality threshold, wherein the analyzing using the machine learning model includes:
extracting, via an inception portion of the machine learning model, a primary set of user identifying features from the image;
selecting, via a self-attention portion of the machine learning model, a subset of the primary set of user identifying features from the image to be weighted greater than other user identifying features in the set of user identifying features; and
determining, via a classification portion of the machine learning model based on the weighting of the subset of user identifying features, whether one or more of the subset of user identifying features in the image used in a user authentication process are obscured by one or more imperfections; and
transmitting, to the user computing device, a quality result, wherein the quality result is generated based on an image classification output by the machine learning model.
11. The non-transitory computer-readable medium of claim 10 wherein the machine learning model includes at least one self-attention block in the self-attention portion that generates a set of attention weight maps from the subset of user identifying features extracted from the image, and wherein the set of attention weight maps indicates the one or more user identifying features in the image to be used in the user authentication process.
12. The non-transitory computer-readable medium of claim 11 , wherein the machine learning model further includes a convolutional block that extracts features from the image to generate a set of feature maps for the image and at least one inception block that extracts features from the image at a higher level of abstraction than the convolutional block to generate a set of down-sampled high-abstraction feature maps.
13. The non-transitory computer-readable medium of claim 10 , further comprising, prior to the analyzing, training the machine learning model using a set of training data generated by:
augmenting images included in a set of existing images of documents; and
assigning, based on binary classifications for images in the set of existing images and the augmented images received from a plurality of users, labels to respective ones of the augmented images and images in the set of existing images.
14. The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
prior to inputting the image into the machine learning model, preprocessing the image of the document, including shrinking a size of the image to a predetermined size and a predetermined number of color dimensions.
15. The non-transitory computer-readable medium of claim 10 , wherein the analyzing includes identifying whether the image includes one or more of the following: a blur, a glare, a reflection, and an obstructing object.
16. A system, comprising:
at least one processor; and
a memory having instructions stored thereon that are executable by the at least one processor to cause the system to:
receive an image of a document captured at a user computing device;
analyze, using a neural network, the image to determine whether the image satisfies a quality threshold, wherein the analyzing using the neural network includes:
extracting, via an inception portion of the neural network, a primary set of user identifying features from the image;
selecting, via a self-attention portion of the neural network, a subset of the primary set of user identifying features from the image to be weighted greater than other user identifying features in the set of user identifying features; and
determining, via a classification portion of the neural network based on the weighting of the subset of user identifying features, whether one or more of the subset of user identifying features in the image used in a user authentication process are obscured; and
transmit a quality result, wherein the quality result is generated based on an image classification output by the neural network.
17. The system of claim 16 , wherein the user computing device is the system, wherein determining whether the image satisfies the quality threshold is performed by the user computing device via execution of the neural network that is trained at the user computing device, and wherein the user computing device is configured to train the neural network by implementing a 5G standard for using 5G cellular technology.
18. The system of claim 16 , wherein the system is a server computer system configured to train and execute the neural network.
19. The system of claim 16 , wherein the neural network includes at least one self-attention block in the self-attention portion for generating a set of attention weight maps from the subset of user identifying features extracted from the image, and wherein the set of attention weight maps indicates the one or more features in the image to be used in the user authentication process.
20. The system of claim 16 , wherein the quality result transmitted to the user computing device indicates that a classification output by the neural network specifies that the image does not meet the quality threshold and prompts a user of the user computing device, via a user interface of the user computing device, to submit a new, higher quality document image that is higher quality than the image of the document captured at the user computing device, wherein image quality is determined at least based on image resolution, wherein the prompt further specifies one or more portions of the image that include imperfections that caused the image to not meet the quality threshold.