Facial recognition for masked individuals
Methods and systems for training a neural network include generate an image of a mask. A copy of an image is generated from an original set of training data. The copy is altered to add the image of a mask to a face detected within the copy. An augmented set of training data is generated that includes the original set of training data and the altered copy. A neural network model is trained to recognize masked faces using the augmented set of training data.
1. A method for training a neural network model, comprising:
generating an image of a mask;
generating a copy of an image from an original set of training data;
altering the copy to add the image of a mask to a face detected within the copy by generating a style transformation of the detected face using a generative adversarial network, including blending the altered copy to conform to an original three-dimensional structure of the detected face;
generating an augmented set of training data that includes the original set of training data and the altered copy; and
training a neural network model to recognize masked faces using the augmented set of training data.
2. The method of claim I, wherein altering the copy includes performing an affine transformation on a mask image to align the mask image to the detected face.
3. The method of claim 2 , wherein performing the affine transformation includes identifying facial key points of the detected face.
4. The method of claim 3 , wherein performing the affine transformation includes scaling the mask image based on a width of the detected face, using the facial key points.
5. The method of claim 3 , wherein performing the affine transformation includes aligning a center of the mask image to a center of the detected face, using the facial key points.
6. The method of claim 3 , wherein performing the affine transformation includes rotating the mask image to match a head pose yaw angle of the detected face, based on the facial key points.
7. The method of claim 1 , wherein generating the style transformation includes identifying a lower portion of the detected face.
8. The method of claim 7 , wherein generating the style transformation includes modifying the lower portion of the detected face to include a mask, without altering a portion of the detected face outside the lower portion.
9. A system for training a neural network model, comprising:
a hardware processor; and
a memory that stores a computer program product, which, when executed by the hardware processor, causes the hardware processor to:
generate an image of a mask;
generate a copy of an image from an original set of training data;
alter the copy to add the image of a mask to a face detected within the copy using a style transformation of the detected face by a generative adversarial network that blends the altered copy to conform to an original three-dimensional structure of the detected face;
generate an augmented set of training data that includes the original set of training data and the altered copy; and
train a neural network model to recognize masked faces using the augmented set of training data.
10. The system of claim 9 , wherein altering the copy includes performing an affine transformation on a mask image to align the mask image to the detected face.
11. The system of claim 10 , wherein performing the affine transformation includes identifying facial key points of the detected face.
12. The system of claim 11 , wherein performing the affine transformation includes scaling the mask image based on a width of the detected face, using the facial key points.
13. The system of claim 11 , wherein, performing the affine transformation includes aligning a center of the mask image to a center of the detected face, using the facial key points.
14. The system of claim 11 , wherein performing the affine transformation includes rotating the mask image to match a head pose yaw angle of the detected face, based on the facial key points.
15. The system of claim 9 , wherein generating the style transformation includes identifying a lower portion of the detected face.
16. The system of claim 15 , wherein generating the style transformation includes modifying the lower portion of the detected face to include a mask, without altering a portion of the detected face outside the lower portion.