Selectively storing, with multiple user accounts and/or to a shared assistant device: speech recognition biasing, NLU biasing, and/or other data
Some implementations relate to performing speech biasing, NLU biasing, and/or other biasing based on historical assistant interaction(s). It can be determined, for one or more given historical interactions of a given user, whether to affect future biasing for (1) the given user account, (2) additional user account(s), and/or (3) the shared assistant device as a whole. Some implementations disclosed herein additionally and/or alternatively relate to: determining, based on utterance(s) of a given user to a shared assistant device, an association of first data and second data; storing the association as accessible to a given user account of the given user; and determining whether to store the association as also accessible by additional user account(s) and/or the shared assistant device.
1. A method comprising:
receiving, via one or more microphones of a shared assistant device, audio data that captures a spoken utterance of a user;
generating, based on processing the audio data using a local speech-to-text (STT) engine of the shared assistant device, a transcription that corresponds to the spoken utterance captured in the audio data;
resolving, based on processing the transcription using a local natural language understanding (NLU) engine of the shared assistant device, an assistant action to perform in response to receiving the spoken utterance;
determining whether to store, locally at the shared assistant device, one or more NLU biasing parameters that are based on the assistant action resolved locally at the shared assistant device using the local NLU engine;
in response to determining to store the one or more NLU biasing parameters locally at the shared assistant device:
storing the one or more NLU biasing parameters locally at the shared assistant device;
wherein storing the one or more NLU biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local NLU engine using the one or more NLU biasing parameters.
2. The method of claim 1 , wherein resolving the assistant action to perform in response to receiving the spoken utterance comprises:
determining, based on processing the transcription using the local NLU, a first candidate parameter and a conflicting second candidate parameter;
prompting the user to disambiguate between the first candidate parameter and the second candidate parameter; and
resolving, based on a response from the user to the prompting, the assistant action to include the first candidate parameter in lieu of the second candidate parameter.
3. The method of claim 2 , wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate parameter.
4. The method of claim 3 , wherein the assistant action causes playing of a song and the first candidate parameter is a song parameter for the song.
5. The method of claim 1 , wherein resolving the assistant action to perform in response to receiving the spoken utterance comprises:
determining, based on processing the transcription using the local NLU, a first candidate intent and a conflicting second candidate intent;
prompting the user to disambiguate between the first candidate intent and the second candidate intent; and
resolving, based on a response from the user to the prompting, the assistant action to include the first candidate intent in lieu of the second candidate intent.
6. The method of claim 5 , wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate intent.
7. The method of claim 1 , wherein determining whether to store the one or more NLU biasing parameters in association with the shared assistant device comprises:
classifying the spoken utterance as being provided by the user;
classifying a previous spoken utterance as being provided by an additional user; and
determining to store the one or more NLU biasing parameters in association with the shared assistant device based on the NLU biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user.
8. A shared assistant device, comprising:
microphones;
memory storing instructions, a local speech-to-text (STT) engine, and a local natural language understanding (NLU) engine; and
one or more processors operable to execute the instructions stored in the memory to:
receive, via the microphones, audio data that captures a spoken utterance of a user;
generate, based on processing the audio data using the local STT engine, a transcription that corresponds to the spoken utterance captured in the audio data;
resolve, based on processing the transcription using the local NLU engine, an assistant action to perform in response to receiving the spoken utterance;
determine whether to store, locally at the shared assistant device, one or more NLU biasing parameters that are based on the assistant action resolved locally at the shared assistant device using the local NLU engine;
in response to determining to store the one or more NLU biasing parameters locally at the shared assistant device:
store the one or more NLU biasing parameters locally at the shared assistant device;
wherein storing the one or more NLU biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local NLU engine using the one or more NLU biasing parameters.
9. The shared assistant device of claim 8 , wherein in resolving the assistant action to perform in response to receiving the spoken utterance one or more of the processors are to:
determine, based on processing the transcription using the local NLU, a first candidate parameter and a conflicting second candidate parameter;
prompt the user to disambiguate between the first candidate parameter and the second candidate parameter; and
resolve, based on a response from the user to the prompting, the assistant action to include the first candidate parameter in lieu of the second candidate parameter.
10. The shared assistant device of claim 9 , wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate parameter.
11. The shared assistant device of claim 10 , wherein the assistant action causes playing of a song and the first candidate parameter is a song parameter for the song.
12. The shared assistant device of claim 8 , wherein in resolving the assistant action to perform in response to receiving the spoken utterance one or more of the processors are to:
determine, based on processing the transcription using the local NLU, a first candidate intent and a conflicting second candidate intent;
prompt the user to disambiguate between the first candidate intent and the second candidate intent; and
resolve, based on a response from the user to the prompting, the assistant action to include the first candidate intent in lieu of the second candidate intent.
13. The shared assistant device of claim 12 , wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate intent.
14. The shared assistant device of claim 8 , wherein in determining whether to store the one or more NLU biasing parameters in association with the shared assistant device one or more of the processors are to:
classify the spoken utterance as being provided by the user;
classify a previous spoken utterance as being provided by an additional user; and
determine to store the one or more NLU biasing parameters in association with the shared assistant device based on the NLU biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user.
15. A method comprising:
receiving, via one or more microphones of a shared assistant device, audio data that captures a spoken utterance of a user;
generating, based on processing the audio data using a local speech-to-text (STT) engine of the shared assistant device, a transcription that corresponds to the spoken utterance captured in the audio data;
determining, based on processing the transcription using a local natural language understanding (NLU) engine of the shared assistant device, an assistant action to perform in response to receiving the spoken utterance;
determining whether to store, locally at the shared assistant device, one or more speech biasing parameters that are based on one or more terms of the transcription generated locally at the shared assistant device using the local STT engine, wherein determining whether to store the one or more speech biasing parameters locally at the shared assistant device comprises:
classifying the spoken utterance as being provided by the user,
classifying a previous spoken utterance as being provided by an additional user, and
determining to store the one or more speech biasing parameters in association with the shared assistant device based on the speech biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user;
in response to determining to store the one or more speech biasing parameters locally at the shared assistant device:
storing the one or more speech biasing parameters locally at the shared assistant device;
wherein storing the one or more speech biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local STT engine using the one or more speech biasing parameters.
16. The method of claim 15 , wherein the one or more speech biasing parameters are based on the one or more terms of the transcription, and wherein the one or more speech biasing parameters include a first biasing parameter that is based on a particular term or a particular topic that is related to the one or more terms of the transcription, but that is not included in the one or more terms of the transcription.