Selective adaptation and utilization of noise reduction technique in invocation phrase detection
Techniques are described for selectively adapting and/or selectively utilizing a noise reduction technique in detection of one or more features of a stream of audio data frames. For example, various techniques are directed to selectively adapting and/or utilizing a noise reduction technique in detection of an invocation phrase in a stream of audio data frames, detection of voice characteristics in a stream of audio data frames (e.g., for speaker identification), etc. Utilization of described techniques can result in more robust and/or more accurate detections of features of a stream of audio data frames in various situations, such as in environments with strong background noise. In various implementations, described techniques are implemented in combination with an automated assistant, and feature(s) detected utilizing techniques described herein are utilized to adapt the functionality of the automated assistant.
1. A method of detecting an invocation phrase for an automated assistant, the method implemented by one or more processors of a client device and comprising:
for each audio data frame of a first set of sequential audio data frames of a stream of audio data frames that are based on output from one or more microphones of the client device:
generating a corresponding noise reduction filter based on the audio data frame, and
storing the corresponding noise reduction filter in a first in, first out buffer;
for a given audio data frame, of the stream of audio data frames, that immediately follows the first set of sequential audio data frames:
generating a filtered data frame based on processing the given audio frame using the corresponding noise reduction filter that is at a head of the first in, first out buffer,
wherein the corresponding noise reduction filter, that is at the head of the first in, first out buffer, and that is used in generating the filtered data frame, was generated based on an earlier in time audio data frame, of the first set of sequential audio data frames, but not generated based on any more recent in time of the first set of sequential audio data frames, and
determining whether the filtered data frame indicates presence of one or more phonemes of the invocation phrase based on processing the filtered data frame using a trained machine learning model;
determining whether the invocation phrase is present in the stream of audio data frames based on whether the filtered data frame indicates presence of one or more of the phonemes of the invocation phrase.
2. The method of claim 1 , further comprising:
in response to determining that the invocation phrase is present:
causing at least one function of the automated assistant to be activated.
3. The method of claim 2 , wherein causing the at least one function of the automated assistant to be activated comprises causing subsequently received audio data frames of the stream to be transmitted to one or more remote automated assistant servers for further processing.
4. The method of claim 2 , wherein causing the at least one function of the automated assistant to be activated comprises causing subsequently received audio data frames of the stream to be processed by one or more local engines.
5. The method of claim 4 , wherein the one or more local engines comprise a local speech-to-text engine.
6. The method of claim 1 , wherein storing the corresponding noise reduction filter in the first in, first out buffer comprises storing coefficients, of the corresponding noise reduction filter, in the first in, first out buffer.
7. The method of claim 1 , wherein the earlier in time audio data frame precedes the given audio data frame by at least one second.
8. The method of claim 1 , wherein the earlier in time audio data frame precedes the given audio data frame by at least two seconds.
9. A method of detecting one or more voice characteristics for speaker identification, the method implemented by one or more processors of a client device and comprising:
for each audio data frame of a first set of sequential audio data frames of a stream of audio data frames that are based on output from one or more microphones of the client device:
generating a corresponding noise reduction filter based on the audio data frame, and
storing the corresponding noise reduction filter in a first in, first out buffer;
for a given audio data frame, of the stream of audio data frames, that immediately follows the first set of sequential audio data frames:
generating a filtered data frame based on processing the given audio frame using the corresponding noise reduction filter that is at a head of the first in, first out buffer,
wherein the corresponding noise reduction filter, that is at the head of the first in, first out buffer, and that is used in generating the filtered data frame, was generated based on an earlier in time audio data frame, of the first set of sequential audio data frames, but not generated based on any more recent in time of the first set of sequential audio data frames, and
determining whether the filtered data frame indicates presence of the one or more voice characteristics based on processing the filtered data frame using a trained machine learning model;
in response to determining that the filtered data frame indicates presence of the one or more voice characteristics:
adapting processing performed by an automated assistant.
10. The method of claim 9 , wherein the one or more voice characteristics are stored in association with a profile and wherein adapting processing performed by the automated assistant comprises:
generating, based on the profile, content to be rendered by the automated assistant, wherein the content is generated based on the profile in response to the filtered data frame indicating presence of the one or more voice characteristics, and the one or more voice characteristics being stored in association with the profile.
11. The method of claim 9 , wherein adapting processing performed by the automated assistant comprises:
causing at least one inactive function of the automated assistant to be activated.
12. The method of claim 11 , wherein the at least one inactive function comprises natural language processing.
13. The method of claim 11 , wherein the at least one inactive function comprises dialog state tracking.
14. The method of claim 9 , wherein storing the corresponding noise reduction filter in the first in, first out buffer comprises storing coefficients, of the corresponding noise reduction filter, in the first in, first out buffer.
15. The method of claim 9 , wherein the earlier in time audio data frame precedes the given audio data frame by at least one second.
16. The method of claim 9 , wherein the earlier in time audio data frame precedes the given audio data frame by at least two seconds.
17. A client device comprising:
one or more microphones
one or more processors;
memory operably coupled with the one or more processors, wherein the memory stores instructions that, when executed by the one or more processors, cause the one or more processors to:
for each audio data frame of a first set of sequential audio data frames of a stream of audio data frames that are based on output from the one or more microphones:
generate a corresponding noise reduction filter based on the audio data frame, and
store the corresponding noise reduction filter in a first in, first out buffer;
for a given audio data frame, of the stream of audio data frames, that immediately follows the first set of sequential audio data frames:
generate a filtered data frame based on processing the given audio frame using the corresponding noise reduction filter that is at a head of the first in, first out buffer,
wherein the corresponding noise reduction filter, that is at the head of the first in, first out buffer, and that is used in generating the filtered data frame, was generated based on an earlier in time audio data frame, of the first set of sequential audio data frames, but not generated based on any more recent in time of the first set of sequential audio data frames, and
determine whether the filtered data frame indicates presence of one or more phonemes of an invocation phrase based on processing the filtered data frame using a trained machine learning model; and
determine whether the invocation phrase is present in the stream of audio data frames based on whether the filtered data frame indicates presence of one or more of the phonemes of the invocation phrase.