Acoustic-based face anti-spoofing system and method
Two-dimensional face presentation attacks are one of most notorious and pervasive face spoofing types, causing security issues to facial authentication systems. To tackle these issues, a cost-effective face anti-spoofing (FAS) system based on acoustic modality, named as Echo-FAS, is devised, which employs a crafted acoustic signal to probe the presented face. First, a large-scale, high-diversity, acoustic-based FAS database, named as Echo-Spoof, is built. Based upon Echo-Spoof, we design a two-branch framework combining global and local frequency features of the presented face to distinguish live vs. spoofing faces. Echo-FAS has the following merits: (1) it only needs one speaker and one microphone; (2) it can capture three-dimensional geometrical information of the presented face and achieve a remarkable FAS performance; and (3) it can be handily allied with RGB-based FAS models to mitigate the overfitting problem in the RGB modality and make the FAS model more accurate and robust.
1. A method for detecting liveness of a presented face, the method comprising:
generating a first acoustic signal and projecting the generated first acoustic signal onto the presented face for probing the presented face, wherein the first acoustic signal comprises a plurality of time-limited chirps, causing a face-echo signal to be reflected from the presented face when the presented face receives an individual time-limited chirp, whereby a plurality of face-echo signals is created for the plurality of time-limited chirps;
receiving a second acoustic signal for capturing an acoustic response of the presented face due to the first acoustic signal, wherein the plurality of face-echo signals is embedded in the second acoustic signal;
preprocessing the received second acoustic signal to yield a plurality of extracted signal segments, wherein the preprocessing of the second acoustic signal includes extracting the plurality of face-echo signals from the received second acoustic signal such that an individual extracted signal segment contains a corresponding face-echo signal;
applying a Fourier transform (FT) to the individual extracted signal segment to yield a frequency segment, whereby a plurality of frequency segments for the plurality of extracted signal segments is obtained;
processing the plurality of frequency segments with a machine-learning transformer model to yield a global frequency feature of the presented face;
applying a short-time Fourier transform (STFT) to the plurality of extracted signal segments to yield a spectrogram;
processing the spectrogram with a convolutional neural network (CNN) to yield a local frequency feature of the presented face; and
combining the global and local frequency features to yield an enriched feature of the presented face for determining whether the presented face is a genuine face or a spoofer.
2. The method of claim 1 , wherein the FT is realized by a fast Fourier transform (FFT).
3. The method of claim 1 , wherein the individual time-limited chirp has a bandwidth of 5 kHz.
4. The method of claim 1 , wherein the plurality of time-limited chirps collectively covers a frequency span from 12 kHz to 21 kHz.
5. The method of claim 4 , wherein the individual time-limited chirp has a frequency sweep range selected from a first range of 12-17 kHz, a second range of 14-19 kHz, and a third range of 16-21 kHz.
6. The method of claim 1 , wherein in extracting the plurality of face-echo signals from the received second acoustic signal, one or more noise components are removed from the second acoustic signal while keeping the plurality of face-echo signals in the second acoustic signal.
7. The method of claim 6 , wherein the one or more noise components include:
a first portion of the first acoustic signal as directly transmitted from a sound generator to a sound receiver without visiting the presented face, the sound generator being used to emit the first acoustic signal, the sound receiver being used to receive the second acoustic signal; and
a second portion of the first acoustic signal as reflected from objects far away from the presented face.
8. The method of claim 1 , wherein:
the first acoustic signal further comprises a pilot for assisting time synchronization in processing the second acoustic signal, the pilot being transmitted before the plurality of time-limited chirps in the first acoustic signal; and
the preprocessing of the second acoustic signal further includes performing time synchronization with the received second acoustic signal by locating the pilot in the second acoustic signal.
9. The method of claim 1 further comprising:
training the transformer model with a training database before the transformer model processes the plurality of frequency segments; and
training the CNN with the training database before the CNN processes the spectrogram.
10. A face anti-spoofing (FAS) system for detecting liveness of a presented face, the FAS system comprising:
a sound generator for generating a first acoustic signal and projecting the generated first acoustic signal onto the presented face for probing the presented face;
a sound receiver for receiving a second acoustic signal for capturing an acoustic response of the presented face due to the first acoustic signal; and
one or more computing processors communicable with the sound generator and the sound receiver, the one or more computing processors being configured to:
control the sound generator to generate the first acoustic signal for projection to the presented face, wherein the first acoustic signal comprises a plurality of time-limited chirps, causing a face-echo signal to be reflected from the presented face when the presented face receives an individual time-limited chirp, whereby a plurality of face-echo signals is created for the plurality of time-limited chirps, and the plurality of face-echo signals is embedded in the second acoustic signal;
receive the second acoustic signal from the sound receiver;
preprocess the received second acoustic signal to yield a plurality of extracted signal segments, wherein the preprocessing of the received second acoustic signal includes extracting the plurality of face-echo signals from the received second acoustic signal such that an individual extracted signal segment contains a corresponding face-echo signal;
apply a Fourier transform (FT) to the individual extracted signal segment to yield a frequency segment, whereby a plurality of frequency segments for the plurality of extracted signal segments is obtained;
process the plurality of frequency segments with a machine-learning transformer model to yield a global frequency feature of the presented face;
apply a short-time Fourier transform (STFT) to the plurality of extracted signal segments to yield a spectrogram;
process the spectrogram with a convolutional neural network (CNN) to yield a local frequency feature of the presented face; and
combine the global and local frequency features to yield an enriched feature of the presented face for determining whether the presented face is a genuine face or a spoofer.
11. The FAS system of claim 10 , wherein the FT is realized by a fast Fourier transform (FFT).
12. The FAS system of claim 10 , wherein the individual time-limited chirp has a bandwidth of 5 kHz.
13. The FAS system of claim 10 , wherein the plurality of time-limited chirps collectively covers a frequency span from 12 kHz to 21 kHz.
14. The FAS system of claim 13 , wherein the individual time-limited chirp has a frequency sweep range selected from a first range of 12-17 kHz, a second range of 14-19 kHz, and a third range of 16-21 kHz.
15. The FAS system of claim 10 , wherein in extracting the plurality of face-echo signals from the received second acoustic signal, one or more noise components are removed from the second acoustic signal while keeping the plurality of face-echo signals in the second acoustic signal.
16. The FAS system of claim 15 , wherein the one or more noise components include:
a first portion of the first acoustic signal as directly transmitted from the sound generator to the sound receiver without visiting the presented face; and
a second portion of the first acoustic signal as reflected from objects far away from the presented face.
17. The FAS system of claim 10 , wherein:
the first acoustic signal further comprises a pilot for assisting time synchronization in processing the second acoustic signal, the pilot before transmitted before the plurality of time-limited chirps in the first acoustic signal; and
the preprocessing of the received second acoustic signal further includes performing time synchronization with the received second acoustic signal by locating the pilot in the second acoustic signal.
18. The FAS system of claim 10 , wherein the one or more computing processors are further configured to:
train the transformer model with a training database before the transformer model processes the plurality of frequency segments; and
train the CNN with the training database before the CNN processes the spectrogram.
19. A smartphone comprising the FAS system of claim 10 , wherein the sound generator is realized by one or more speakers, and the sound receiver is realized by one or more microphones.
20. The smartphone of claim 19 , wherein the sound generator is a single speaker, and the sound receiver is a single microphone.