System and method for post-ASR false wake-up suppression
A method includes obtaining a speech signal. The method also includes predicting a first likelihood of a wake word or phrase being spoken in the speech signal using a first machine learning model trained to receive the speech signal as input. The method further includes, responsive to the first likelihood exceeding a first threshold, performing automatic speech recognition on the speech signal to determine a textual representation of the speech signal. The method also includes predicting a second likelihood of the wake word or phrase being spoken in the speech signal using a second machine learning model trained to receive at least one of the textual representation, audio features associated with the speech signal, and context features associated with the electronic device. In addition, the method includes, responsive to the second likelihood exceeding a second threshold, generating instructions to perform an action requested in the speech signal.
1 . A method comprising:
obtaining, by at least one processing device of an electronic device, a speech signal;
predicting, by the at least one processing device, a first likelihood of a wake word or phrase being spoken in the speech signal using a first machine learning model trained to receive the speech signal as input;
responsive to the first likelihood exceeding a first threshold, performing, by the at least one processing device, automatic speech recognition on the speech signal to determine a textual representation of the speech signal;
predicting, by the at least one processing device, a second likelihood of the wake word or phrase being spoken in the speech signal using a second machine learning model trained to receive at least one of the textual representation, audio features associated with the speech signal, and context features associated with the electronic device;
responsive to the second likelihood exceeding a second threshold, generating, by the at least one processing device, instructions to perform an action requested in the speech signal;
determining, by the at least one processing device, an accuracy of the first machine learning model used to predict the first likelihood using a plurality of previous speech signals; and
updating, by the at least one processing device, the second threshold based on the determined accuracy.
2 . The method of claim 1 , wherein predicting the second likelihood using the second machine learning model includes:
performing, by the at least one processing device, natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
providing, by the at least one processing device to a multi-layer perceptron model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receiving, by the at least one processing device, an output from the multi-layer perceptron model indicating whether or not to suppress a wake-up operation of a voice assistant.
3 . The method of claim 1 , wherein predicting the second likelihood using the second machine learning model includes:
performing, by the at least one processing device, natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
providing, by the at least one processing device to a random forest classifier model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receiving, by the at least one processing device, an output from the random forest classifier model indicating whether or not to suppress a wake-up operation of a voice assistant.
4 . The method of claim 1 , further comprising:
determining, by the at least one processing device, a background noise environment associated with the speech signal; and
setting the second threshold based on the determined background noise environment.
5 . The method of claim 1 , wherein the audio features include at least one of: a bag of words, a total word count, a total character count, a unique word count, a stop word count, an audio time, and a signal-to-noise ratio.
6 . The method of claim 1 , wherein the context features include at least one of: at least one user characteristic, a launch method, and foreground application information.
7 . The method of claim 4 , wherein:
determining the background noise environment associated with the speech signal includes determining that the background noise environment is associated with an environment in which use of a voice assistant is likely; and
setting the second threshold based on the determined background noise environment includes decreasing the second threshold.
8 . An electronic device comprising:
at least one processing device configured to:
obtain a speech signal;
predict a first likelihood of a wake word or phrase being spoken in the speech signal using a first machine learning model trained to receive the speech signal as input;
responsive to the first likelihood exceeding a first threshold, perform automatic speech recognition on the speech signal to determine a textual representation of the speech signal;
predict a second likelihood of the wake word or phrase being spoken in the speech signal using a second machine learning model trained to receive at least one of the textual representation, audio features associated with the speech signal, and context features associated with the electronic device;
responsive to the second likelihood exceeding a second threshold, generate instructions to perform an action requested in the speech signal;
determine an accuracy of the first machine learning model used to predict the first likelihood using a plurality of previous speech signals; and
update the second threshold based on the determined accuracy.
9 . The electronic device of claim 8 , wherein, to predict the second likelihood using the second machine learning model, the at least one processing device is configured to:
perform natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
provide, to a multi-layer perceptron model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receive an output from the multi-layer perceptron model indicating whether or not to suppress a wake-up operation of a voice assistant.
10 . The electronic device of claim 8 , wherein, to predict the second likelihood using the second machine learning model, the at least one processing device is configured to:
perform natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
provide, to a random forest classifier model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receive an output from the random forest classifier model indicating whether or not to suppress a wake-up operation of a voice assistant.
11 . The electronic device of claim 8 , wherein the at least one processing device is further configured to:
determine a background noise environment associated with the speech signal; and
set the second threshold based on the determined background noise environment.
12 . The electronic device of claim 8 , wherein the audio features include at least one of: a bag of words, a total word count, a total character count, a unique word count, a stop word count, an audio time, and a signal-to-noise ratio.
13 . The electronic device of claim 8 , wherein the context features include at least one of: at least one user characteristic, a launch method, and foreground application information.
14 . The electronic device of claim 11 , wherein;
to determine the background noise environment associated with the speech signal, the at least one processing device is configured to determine that the background noise environment is associated with an environment in which use of a voice assistant is likely; and
to set the second threshold based on the determined background noise environment, the at least one processing device is configured to decrease the second threshold.
15 . A non-transitory machine readable medium containing instructions that when executed cause at least one processor of an electronic device to:
obtain a speech signal;
predict a first likelihood of a wake word or phrase being spoken in the speech signal using a first machine learning model trained to receive the speech signal as input;
responsive to the first likelihood exceeding a first threshold, perform automatic speech recognition on the speech signal to determine a textual representation of the speech signal;
predict a second likelihood of the wake word or phrase being spoken in the speech signal using a second machine learning model trained to receive at least one of the textual representation, audio features associated with the speech signal, and context features associated with the electronic device;
responsive to the second likelihood exceeding a second threshold, generate instructions to perform an action requested in the speech signal;
determine an accuracy of the first machine learning model used to predict the first likelihood using a plurality of previous speech signals; and
update the second threshold based on the determined accuracy.
16 . The non-transitory machine readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to predict the second likelihood using the second machine learning model comprise instructions that when executed cause the at least one processor to:
perform natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
provide, to a multi-layer perceptron model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receive an output from the multi-layer perceptron model indicating whether or not to suppress a wake-up operation of a voice assistant.
17 . The non-transitory machine readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to predict the second likelihood using the second machine learning model comprise instructions that when executed cause the at least one processor to:
perform natural language processing on the textual representation using a deep transformer model included in the second machine learning model and outputting one or more results;
provide, to a random forest classifier model of the second machine learning model, the one or more results of the natural language processing, the audio features, and the context features; and
receive an output from the random forest classifier model indicating whether or not to suppress a wake-up operation of a voice assistant.
18 . The non-transitory machine readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to:
determine a background noise environment associated with the speech signal; and
set the second threshold based on the determined background noise environment.
19 . The non-transitory machine readable medium of claim 15 , wherein the audio features include at least one of: a bag of words, a total word count, a total character count, a unique word count, a stop word count, an audio time, and a signal-to-noise ratio.
20 . The non-transitory machine readable medium of claim 15 , wherein the context features include at least one of: at least one user characteristic, a launch method, and foreground application information.