Detecting and handling failures in other assistants
Techniques are described herein for detecting and handling failures in other automated assistants. A method includes: executing a first automated assistant in an inactive state at least in part on a computing device operated by a user; while in the inactive state, determining, by the first automated assistant, that a second automated assistant failed to fulfill a request of the user; in response to determining that the second automated assistant failed to fulfill the request of the user, the first automated assistant processing cached audio data that captures a spoken utterance of the user comprising the request that the second automated assistant failed to fulfill, or features of the cached audio data, to determine a response that fulfills the request of the user; and providing, by the first automated assistant to the user, the response that fulfills the request of the user.
1. A method implemented by one or more processors, the method comprising:
determining, by a first automated assistant executed at least in part on a computing device operated by a user, that a second automated assistant provided an inaccurate result in response to a request of the user that was directed to the second automated assistant, wherein determining that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant comprises:
processing, by the first automated assistant, audio data that captures an initial response provided by the second automated assistant responsive to the request; and
determining, by the first automated assistant, and based on the processing of the audio data, that the initial response is the inaccurate result; and
in response to the first automated assistant determining that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant;
automatically determining, by the first automated assistant, a response that fulfills the request of the user, wherein automatically determining the response that fulfills the request of the user comprises automatically processing, by the first automated assistant, cached audio data, or features of the cached audio data, that captures a spoken utterance of the user comprising the request responsive to which the second automated assistant provided the inaccurate result; and
automatically providing, by the first automated assistant to the user, the response that fulfills the request of the user.
2. The method according to claim 1 , wherein the cached audio data further captures the initial response provided by the second automated assistant to the request.
3. The method according to claim 1 , wherein:
the second automated assistant is executed on the computing device, and
the cached audio data is received by the first automated assistant via a meta assistant that is executed on the computing device.
4. The method according to claim 1 , wherein:
the second automated assistant is executed on another computing device, and
the cached audio data is received by the first automated assistant via one or more microphones of the computing device.
5. The method according to claim 1 , wherein processing the audio data that captures the initial response provided by the second automated assistant responsive to the request comprises:
receiving the audio data that captures the initial response provided by the second automated assistant responsive to the request; and
using speaker identification on the audio data that captures the initial response to determine that the initial response is provided by the second automated assistant.
6. The method according to claim 5 , wherein processing the audio data that captures the initial response provided by the second automated assistant responsive to the request further comprises:
processing the audio data that captures the initial response using a hotword detection model to determine that the initial response is the inaccurate result.
7. The method according to claim 5 , wherein processing the audio data that captures the initial response provided by the second automated assistant responsive to the request further comprises:
processing the audio data that captures the initial response using automatic speech recognition to generate text; and
processing the text using natural language processing techniques to determine that the initial response is the inaccurate result.
8. The method according to claim 1 , wherein the first automated assistant causes the response that fulfills the request of the user to be provided on another computing device.
9. The method according to claim 1 , wherein the first automated assistant causes the response that fulfills the request of the user to be displayed on a display of the computing device.
10. A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
determine, by a first automated assistant executed at least in part on a computing device operated by a user, that a second automated assistant provided an inaccurate result in response to a request of the user that was directed to the second automated assistant, wherein the program instructions that are executable to determine that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant are executable to:
process, by the first automated assistant, audio data that captures an initial response provided by the second automated assistant responsive to the request; and
determine, by the first automated assistant, and based on the processing of the audio data, that the initial response is the inaccurate result; and
in response to the first automated assistant determining that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant:
automatically determine, by the first automated assistant, a response that fulfills the request of the user, wherein the program instructions that are executable to automatically determine the response that fulfills the request of the user are executable to automatically process, by the first automated assistant, cached audio data, or features of the cached audio data, that captures a spoken utterance of the user comprising the request responsive to which the second automated assistant provided the inaccurate result; and
automatically provide, by the first automated assistant to the user, an indication of availability of the response that fulfills the request of the user.
11. The computer program product according to claim 10 , wherein the cached audio data further captures the initial response provided by the second automated assistant to the request.
12. The computer program product according to claim 10 , wherein:
the second automated assistant is executed on the computing device, and
the cached audio data is received by the first automated assistant via a meta assistant that is executed on the computing device.
13. The computer program product according to claim 10 , wherein:
the second automated assistant is executed on another computing device, and
the cached audio data is received by the first automated assistant via one or more microphones of the computing device.
14. The computer program product according to claim 10 , wherein the program instructions that are executable to process the audio data that captures the initial response provided by the second automated assistant responsive to the request are executable to:
receive the audio data that captures the initial response provided by the second automated assistant responsive to the request;
use speaker identification on the audio data that captures the initial response to determine that the initial response is provided by the second automated assistant; and
process the audio data that captures the initial response using a hotword detection model to determine that the initial response is the inaccurate result.
15. The computer program product according to claim 10 , wherein the indication of availability is a visual indication provided by the computing device.
16. The computer program product according to claim 15 , the program instructions further being executable to:
receive, by the first automated assistant, a request for the response that fulfills the request of the user; and
in response to receiving the request for the response that fulfills the request of the user, provide, by the first automated assistant to the user, the response that fulfills the request of the user.
17. A system comprising:
a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
determine, by a first automated assistant executed at least in part on a computing device operated by a user, that a second automated assistant provided an inaccurate result in response to a request of the user that was directed to the second automated assistant, wherein the programs instructions that are executable to determine that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant are executable to:
process, by the first automated assistant, audio data that captures an initial response provided by the second automated assistant responsive to the request; and
determine, by the first automated assistant, and based on the processing of the audio data, that the initial response is the inaccurate result;
in response to the first automated assistant determining that the second automated assistant provided the inaccurate result in response to the request of the user that was directed to the second automated assistant, automatically provide an indication to the user that the first automated assistant is available to fulfill the request of the user; and
in response to receiving from the user an instruction to fulfill the request:
automatically determine, by the first automated assistant, a response that fulfills the request of the user, wherein the program instructions that are executable to automatically determine the response that fulfills the request of the user are executable to automatically process, by the first automated assistant, cached audio data, or features of the cached audio data, that captures a spoken utterance of the user comprising the request responsive to which the second automated assistant provided the inaccurate result; and
automatically provide, by the first automated assistant to the user, the response that fulfills the request of the user.