System(s) and method(s) for implementing a personalized chatbot
Processor(s) of a client device of a user can receive a telephone call that is initiated by an additional user, and, in response to receiving the telephone call, identify an entity that is associated with the additional user, and determine, based on the entity that is associated with the additional user, whether to (1) fully automate the telephone call, or (2) partially automate the telephone call. In fully automating the telephone call, the processor(s) can cause a chatbot to engage in a corresponding conversation with the additional user and without prompting the user for any input. In partially automating the telephone call, the processor(s) can cause the chatbot to engage in a corresponding conversation with the additional user but with prompting the user for input(s) via suggestion chip(s). In some implementations, the processor(s) can further determine whether to (3) refrain from automating the telephone call entirely.
1. A method implemented by one or more processors, the method comprising:
receiving a telephone call, the telephone call being received via a client device of a user, and the telephone call being initiated by an additional user that is in addition to the user;
in response to receiving the telephone call:
identifying an entity that is associated with the additional user that initiated the telephone call; and
determining, based on an entity that is associated with the additional user, whether to (1) fully automate the telephone call on behalf of the user, or (2) partially automate the telephone call on behalf of the user;
in response to determining to (1) fully automate the telephone call on behalf of the user, causing the telephone call to be fully automated on behalf of the user; and
in response to determining to (2) partially automate the telephone call on behalf of the user, causing the telephone call to be partially automated on behalf of the user.
2. The method of claim 1 , wherein causing the telephone call to be fully automated on behalf of the user comprises:
causing a chatbot, that is implemented locally at the client device of the user, to engage in a conversation with the additional user; and
during the conversation with the additional user:
causing the chatbot to perform a task, that is specified by the additional user, on behalf of the user and without prompting the user to provide any input.
3. The method of claim 2 , wherein causing the chatbot to engage in a conversation with the additional user comprises:
receiving corresponding instances of audio data from the additional user;
generating, based on processing the corresponding instances of audio data from the additional user, corresponding instances of synthesized speech audio data that capture synthesized speech of the chatbot; and
causing the corresponding instances of the synthesized speech audio data to be audibly rendered for presentation to the additional user via an additional client device of the additional user.
4. The method of claim 3 , wherein processing a given instance of audio data, from among the corresponding instances of audio data received from the additional user, comprises:
processing, using an on-device automatic speech recognition (ASR) model that is stored locally at the client device, the given instance of audio data to generate given ASR output;
processing, using an on-device natural language understanding (NLU) model that is stored locally at the client device, the given ASR output to generate given NLU output; and
processing, using an on-device fulfillment model that is stored locally at the client device and/or on-device fulfillment rules that are stored locally at the client device, the given NLU to generate a given fulfillment output.
5. The method of claim 4 , wherein generating a given corresponding instance of synthesized speech audio data, from among the corresponding instances of the synthesized speech audio data, and in response to receiving the given instance of audio data, comprises:
generating, based on the given fulfillment output, a given instance of textual data in furtherance of fulfilling the task; and
processing, using an on-device text-to-speech (TTS) model that is stored locally at the client device, the given instance of textual data to generate the given corresponding instance of synthesized speech audio data.
6. The method of claim 2 , further comprising:
subsequent to concluding the conversation with the additional user:
generating a notification that identifies the entity and that includes a result of the task; and
causing the notification to be provided for presentation to the user via the client device.
7. The method of claim 1 , wherein causing the telephone call to be partially automated on behalf of the user comprises:
causing a chatbot, that is implemented locally at the client device of the user, to engage in a conversation with the additional user; and
during the conversation with the additional user:
causing the chatbot to perform a task, that is specified by the additional user, on behalf of the user but prompting the user to provide one or more inputs.
8. The method of claim 7 , wherein causing the chatbot to engage in a conversation with the additional user comprises:
receiving corresponding instances of audio data from the additional user;
generating, based on processing the corresponding instances of audio data from the additional user, corresponding instances of synthesized speech audio data that capture synthesized speech of the chatbot; and
causing the corresponding instances of the synthesized speech audio data to be audibly rendered for presentation to the additional user via an additional client device of the additional user.
9. The method of claim 8 , wherein processing a given instance of audio data, from among the corresponding instances of audio data received from the additional user, comprises:
processing, using an on-device automatic speech recognition (ASR) model that is stored locally at the client device, the given instance of audio data to generate given ASR output;
processing, using an on-device natural language understanding (NLU) model that is stored locally at the client device, the given ASR output to generate given NLU output; and
determining, based on the given ASR output and/or the given NLU output, one or more suggestions chips to be provided for presentation to the user.
10. The method of claim 9 , wherein generating a given corresponding instance of synthesized speech audio data, from among the corresponding instances of the synthesized speech audio data, and in response to receiving the given instance of audio data, comprises:
generating, based on receiving a user selection of a given suggestion chip from among the one or more suggestion chips, a given instance of textual data in furtherance of fulfilling the task; and
processing, using an on-device text-to-speech (TTS) model that is stored locally at the client device, the given instance of textual data to generate the given corresponding instance of synthesized speech audio data.
11. The method of claim 10 , wherein the given instance of textual data expresses a sentiment of the given suggestion chip in a context of the conversation with the additional user.
12. The method of claim 7 , further comprising:
subsequent to concluding the conversation with the additional user:
generating a notification that identifies the entity and that includes a result of the task; and
causing the notification to be provided for presentation to the user via the client device.
13. The method of claim 1 , wherein determining whether to (1) fully automate the telephone call on behalf of the user, or (2) partially automate the telephone call on behalf of the user based on the entity that is associated with the additional user comprises:
determining a type of entity of the entity that is associated with the additional user; and
determining whether a chatbot, that is implemented locally at the client device of the user, is capable of fully automating the telephone call on behalf of the user.
14. The method of claim 13 , wherein determining whether the automated assistant is capable of fully automating the telephone call on behalf of the user is based on the user engaging in one or more previous conversations with the entity and/or an additional entity of the same type of entity as the entity.
15. The method of claim 14 , wherein the chatbot is previously trained based on the one or more previous conversations locally at the client device.
16. The method of claim 14 , wherein determining whether the automated assistant is capable of fully automating the telephone call on behalf of the user is based on availability of user data, that is stored locally at the client device of the user, of the user of the client device.
17. The method of claim 1 , wherein determining whether to (1) fully automate the telephone call on behalf of the user, or (2) partially automate the telephone call on behalf of the user based on the entity that is associated with the additional user comprises:
determining, for the entity, whether the user previously provided user input that specifies:
causing the telephone call to be fully automated on behalf of the user, or
causing the telephone call to be partially automated on behalf of the user.
18. The method of claim 17 , further comprising:
determining whether the user input specifies temporal conditions associated with causing the telephone call to be fully automated on behalf of the user or causing the telephone call to be partially automated on behalf of the user, and
wherein determining whether to (1) fully automate the telephone call on behalf of the user, or (2) partially automate the telephone call on behalf of the user, is in response to determining that the temporal conditions are satisfied.
19. A method implemented by one or more processors, the method comprising:
receiving a telephone call, the telephone call being received via a client device of a user, and the telephone call being initiated by an additional user that is in addition to the user; and
in response to receiving the telephone call, causing the telephone call to be partially automated on behalf of the user, wherein causing the telephone call to be partially automated on behalf of the user comprises:
causing a chatbot, that is implemented locally at the client device of the user, to engage in a conversation with the additional user; and
during the conversation with the additional user:
receiving a given instance of audio data that requests the chatbot to perform a task;
determining, based on processing the given instance of audio data, a given suggestion chip that is associated with the task and that is to be provided for presentation to the user;
causing the given suggestion chip to be provided for presentation to the user;
in response to receiving a user selection, from the user, of the given suggestion chip and in response to determining that the additional user is associated with a first type of entity:
generating, based on the user selection of the given suggestion and based on the additional user being associated with the first type of entity, a first instance of synthesized speech audio data that captures first synthesized speech of the chatbot; and
causing the first instance of synthesized speech audio data to be audibly rendered for presentation to the additional user via an additional client device of the additional user; and
in response to receiving the user selection, from the user, of the given suggestion chip and in response to determining that the additional user is associated with a second type of entity that differs from the first entity type:
generating, based on the user selection of the given suggestion and based on the additional user being associated with the second type of entity, a second instance of synthesized speech audio data that captures second synthesized speech of the chatbot and that differs from the first synthesized speech of the chatbot; and
causing the second instance of synthesized speech audio data to be audibly rendered for presentation to the additional user via the additional client device of the additional user.
20. A method implemented by one or more processors, the method comprising:
receiving a telephone call, the telephone call being received via a client device of a user, and the telephone call being initiated by an additional user that is in addition to the user;
in response to receiving the telephone call:
identifying an entity that is associated with the additional user that initiated the telephone call; and
determining, based on the entity that is associated with the additional user, whether to (1) fully automate the telephone call on behalf of the user, (2) partially automate the telephone call on behalf of the user, or (3) refrain from automating the telephone call on behalf of the user;
in response to determining to (1) fully automate the telephone call on behalf of the user, causing the telephone call to be fully automated on behalf of the user;
in response to determining to (2) partially automate the telephone call on behalf of the user, causing the telephone call to be partially automated on behalf of the user; and
in response to determining to (3) refrain from automating the telephone call on behalf of the user, causing the user to be notified of the telephone call via the client device or an additional client device.