SOURCE TRACING OF AUDIO DEEPFAKE SYSTEMS
Disclosed are systems and methods including software processes executed by a server that implement a machine-learning architecture for audio source tracing for deepfake detection. The computer extracts a feature vector representing features of the input audio signal. The machine-learning architecture includes one or more embedding extractors for extracting one or more feature vectors from the input audio signal. An attribute detector ingests an embedding and scoring layers generate a source-indicating attribute score. A source tracer includes a multi-class classifier to generate a signal source score using the attribute scores and generates a signal source class.
1 . A computer-implemented method for detecting fraudulent calls and source detection using machine-learning, the method comprising:
extracting, by a computer, a feature vector embedding representing a set of spoofing features extracted from an input audio signal;
generating, by the computer, a plurality of attribute scores using a plurality of attribute detectors of a machine-learning architecture based upon the feature vector embedding, each attribute detector includes a machine-learning model trained to generate an attribute score indicating a likelihood of a source-indicating attribute that generated the audio signal;
generating, by the computer, a signal source score based upon the plurality of attribute scores, the signal source score indicating a probability of an audio source technology that generated the audio signal;
identifying, by the computer, the audio source technology based upon the signal source score according to one or more class thresholds using a multi-class classifier; and
generating, by the computer, a notification for display at a user interface indicating the audio source technology that originated the audio signal.
2 . The method according to claim 1 , wherein the source-indicating attribute includes at least one of an input type, an acoustic model, or a vocoder.
3 . The method according to claim 1 , further comprising training, by the computer, a first embedding extractor to extract the feature vector embedding having the spoofing features using a plurality of training audio signals including the audio signal and corresponding training labels.
4 . The method according to claim 1 , further comprising training, by the computer, a plurality of embedding extractors for extracting a plurality of feature vector embeddings corresponding to the plurality of attribute detectors, including the first embedding extractor corresponding to a first attribute detector, and a second embedding extractor corresponding to the a second attribute detector.
5 . The method according to claim 1 , further comprising generating, by the computer, a first attribute score for a first source-indicating attribute using a first attribute detector based upon the feature vector embedding.
6 . The method according to claim 5 , further comprising:
extracting, by the computer, a second feature vector embedding representing a second set of spoofing features extracted from the audio signal; and
generating, by the computer, a second attribute score for a second source-indicating attribute based upon the second feature vector embedding using a second attribute detector.
7 . The method according to claim 1 , further comprising generating, by the computer, a loss for the signal source score using a loss function, the loss indicating a distance between the signal source and an expected signal source score indicated by a training label associated with the input audio signal.
8 . The method according to claim 7 , further comprising updating, by the computer, one or more parameters of the multi-class classifier model based upon the loss.
9 . The method according to claim 7 , further comprising updating, by the computer, one or more parameters of one or more embedding extractors model based upon the loss.
10 . The method according to claim 7 , further comprising updating, by the computer, one or more parameters of one or more source attribute detectors based upon the loss.
11 . A system for detecting fraudulent calls and source detection using machine-learning, the system comprising:
a computer comprising at least one processor, the computer configured to:
extract a feature vector embedding representing a set of spoofing features extracted from the audio signal;
generate a plurality of attribute scores using a plurality of attribute detectors of a machine-learning architecture based upon the feature vector embedding, each attribute detector includes a machine-learning model trained to generate an attribute score indicating a likelihood of a source-indicating attribute that generated the audio signal;
generate a signal source score based upon the plurality of attribute scores, the signal source score indicating a probability of an audio source technology that generated the audio signal;
identify the audio source technology based upon the signal source score according to one or more class thresholds using a multi-class classifier; and
generate a notification for display at a user interface indicating the audio source technology that originated the audio signal.
12 . The system according to claim 11 , wherein the source-indicating attribute includes at least one of an input type, an acoustic model, or a vocoder.
13 . The system according to claim 11 , wherein the computer is further configured to train a first embedding extractor to extract the feature vector embedding having the spoofing features using a plurality of training audio signals including the audio signal and corresponding training labels.
14 . The system according to claim 11 , wherein the computer is further configured to train a plurality of embedding extractors for extracting a plurality of feature vector embeddings corresponding to the plurality of attribute detectors, including the first embedding extractor corresponding to a first attribute detector, and a second embedding extractor corresponding to a second attribute detector.
15 . The system according to claim 11 , wherein the computer is further configured to generate a first attribute score for a first source-indicating attribute using a first attribute detector based upon the feature vector embedding.
16 . The system according to claim 15 , wherein the computer is further configured to:
extract a second feature vector embedding representing a second set of spoofing features extracted from the audio signal; and
generate a second attribute score for a second source-indicating attribute based upon the second feature vector embedding using a second attribute detector.
17 . The system according to claim 11 , wherein the computer is further configured to generate a loss for the signal source score using a loss function, the loss indicating a distance between the signal source and an expected signal source score indicated by a training label associated with the input audio signal.
18 . The system according to claim 17 , wherein the computer is further configured to update one or more parameters of the multi-class classifier model based upon the loss.
19 . The system according to claim 17 , wherein the computer is further configured to update one or more parameters of one or more embedding extractors model based upon the loss.
20 . The system according to claim 17 , wherein the computer is further configured to update one or more parameters of one or more source attribute detectors based upon the loss.