Spoof detection using reinforcement learning to generate improved training data
The disclosure includes a system and method for spoof image detection. A spoof detection model is trained based at least in part on spoofed images and/or videos generated by a spoof agent. The spoof agent is trained using a reinforcement learning algorithm that is rewarded for successfully fooling the spoof detection model. Thus, the spoof agent learns to improve its abilities to fool the spoof detection model, while the spoof detection model learns to become better at detecting spoofing.
1 . A computer system for training a spoof detector to detect spoofing attacks, comprising:
at least one computing unit having a processor and a memory, the computer system including computer program instructions to implement:
a spoof detection machine learning model;
a spoof agent;
the spoof detection machine learning model trained to detect spoof attempts of faces based at least in part on training data generated by the spoof agent, the spoof agent having a reinforcement learning model in which the spoof agent is rewarded for generating simulated attempted spoofing attacks of faces that fool the spoof detection machine learning model, wherein the spoof detection machine learning model is trained iteratively with the spoof agent utilizing reinforcement learning to improve its ability to train the spoof detection machine learning model;
wherein the spoof agent controls a 6 degrees of freedom robotic arm holding a camera to select an x, y, z, roll, pitch, and yaw of spoofed facial images captured by the camera in the simulated attempted spoofing attacks of faces;
wherein the spoof agent controls lighting parameters of spoofed facial images captured by the camera in the simulated attempted spoofing attacks of faces, including controlling a number of lights used, a position and direction of lights used, a color spectrum of lights used, and an intensity of lights used;
wherein the spoof agent selects samples of facial images and gets N attempts to spoof a given sample, where N is a number selected to control a size of an action space;
wherein an additional agent is used to select a subgroup of samples to attempt to exploit at least one potential bias of the spoof detector for detecting identity spoofing, including at least one of gender, ethnic group, skin tone, and age group; and
wherein the spoof detection machine learning model is trained to detect spoofing attacks in biometric identity verification.
2 . The computer system of claim 1 , wherein the spoof detection machine learning model comprises a neural network model.
3 . The computer system of claim 1 , wherein the spoof agent controls at least one camera parameter of spoofed facial images captured by the camera in the simulated spoofing attacks of faces, wherein the at least one camera parameter is selected from the group consisting of an exposure setting, shutter speed setting, f-stop setting, and ISO setting.
4 . The computer system of claim 1 , wherein the simulated attempted spoofing attacks of faces are based on at least one of paper-based attacks of faces, screen-based spoof attacks of faces, and three-dimensional spoof attacks of faces.
5 . The computer system of claim 1 , further comprising pre-training the spoof agent.
6 . A method of biometric authentication, comprising:
receiving facial images for biometric identity verification; and
performing biometric authentication of facial images using a spoof detector having a spoof detection machine learning model;
wherein the spoof detection machine learning model is trained to detect spoof attempts of faces based at least in part on training data generated by a spoof agent, the spoof agent having a reinforcement learning model in which the spoof agent is rewarded for generating simulated attempted spoofing attacks of faces that fool the spoof detection machine learning model, wherein the spoof detection machine learning model is trained iteratively with the spoof agent utilizing reinforcement learning to improve its ability to train the spoof detection machine learning model;
wherein the spoof agent controls a 6 degrees of freedom robotic arm holding a camera to select an x, y, z, roll, pitch, and yaw of spoofed facial images captured by the camera in the simulated spoofing attacks of faces;
wherein the spoof agent controls lighting parameters of spoofed facial images captured by the camera in the simulated spoofing attacks of faces, including controlling a number of lights used, a position and direction of lights used, a color spectrum of lights used, and an intensity of lights used;
wherein the spoof agent selects samples of facial images and gets N attempts to spoof a given sample, where N is a number selected to control a size of an action space; and
wherein an additional agent is used to select a subgroup of samples to attempt to exploit at least one potential bias of the spoof detector for detecting identity spoofing, including at least one of gender, ethnic group, skin tone, and age group.
7 . The method of claim 6 , wherein the spoof detection machine learning model comprises a neural network model.
8 . The method of claim 6 , further comprising the spoof agent controlling at least one camera parameter of spoofed facial images captured by the camera in the simulated spoofing attacks of faces, wherein the at least one camera parameter is selected from the group consisting of an exposure setting, shutter speed setting, f-stop setting, and ISO setting.
9 . The method of claim 6 , wherein the simulated attempted spoofing attacks of faces are based on at least one of paper-based attacks of faces, screen-based spoof attacks of faces, and three-dimensional spoof attacks of faces.
10 . The method of claim 6 , further comprising pre-training the spoof agent.
11 . A method of generating training data for a spoof detector, comprising:
generating, by a spoof agent, simulated attempted spoof attacks of biometric facial images targeted at a spoof detection model;
controlling, by the spoof agent, a 6 degrees of freedom robotic arm holding a camera to select an x, y, z, roll, pitch, and yaw of spoofed biometric facial images captured by a camera as part of a simulated attempted spoof attack;
controlling, by the spoof agent, lighting parameters of spoofed facial images captured by the camera in the simulated spoofing attacks of faces, including controlling a number of lights used, a position and direction of lights used, a color spectrum of lights used, and an intensity of lights used;
selecting samples, by the spoof agent, of facial images in which the spoof agent gets N attempts to spoof a given sample, where N is a number selected to control a size of an action space;
selecting samples, by an additional agent, to attempt to exploit at least one potential bias of the spoof detector for detecting biometric images including at least one of gender, ethnic group, skin tone and age group;
identifying successful and unsuccessful simulated spoof attacks of biometric facial images by the spoof detection model;
generating a reinforcement learning reward for the spoof agent successfully fooling the spoof detection machine learning model, whereby the spoof agent learns to adapt its spoof attempts; and
utilizing results of the simulated attempted spoof attacks as the source of training data for the spoof detection machine learning model to detect spoofing attacks in biometric identity verification.
12 . The method of claim 11 , wherein the spoof detection machine learning model comprises a neural network model.
13 . The method of claim 11 , wherein the spoof agent comprises a neural network model.
14 . The method of claim 11 , wherein the generating simulated spoof attack attempts includes selecting at least one action for capturing spoofed images and/or videos.
15 . The method of claim 11 , wherein the at least one action comprises selecting a location for capturing an image and/or video.
16 . The method of claim 11 , wherein the reference location is selected for a robotic arm with respect to a camera used to capture images and/or videos.
17 . The method of claim 11 , wherein the at least one action comprises selecting a lighting condition and at least one camera parameter selected from the group consisting of an exposure setting, shutter speed setting, f-stop setting, and ISO setting.
18 . The method of claim 11 , wherein the simulated attempted spoof attacks are based on at least one of paper-based attacks, screen-based spoof attacks, and three-dimensional spoof attacks.
19 . The method of claim 11 , further comprising selecting at least one sample selection rule by the spoof agent.
20 . The method of claim 11 , further comprising pre-training the spoof agent.