Cross-device data synchronization based on simultaneous hotword triggers
Techniques are described herein for cross-device data synchronization based on simultaneous hotword triggers. A method includes: executing a first instance of an automated assistant in an inactive state at least in part on a first computing device operated by a user; while in the inactive state, receiving, via one or more microphones of the first computing device, audio data that captures a spoken utterance of the user; processing the audio data using a machine learning model to generate a predicted output that indicates a probability of one or more hotwords being present in the audio data; determining that the predicted output satisfies a threshold that is indicative of the one or more hotwords being present in the audio data; in response to determining that the predicted output satisfies the threshold, performing arbitration with at least one other computing device that is executing at least in part at least one other instance of the automated assistant; and in response to performing arbitration with the at least one other computing device, initiating synchronization of user data or configuration data between the first instance of the automated assistant on the first computing device and the at least one other instance of the automated assistant on the at least one other computing device, the user data comprising data that is based on one or more interactions with the user at the first computing device, the one or more interactions occurring prior to the receiving of the audio data.
1 . A method implemented by one or more processors, the method comprising:
executing a first instance of an automated assistant in an inactive state at least in part on a first computing device operated by a user;
while in the inactive state, receiving, via one or more microphones of the first computing device, first audio data that captures a spoken utterance of the user;
while in the inactive state, processing the first audio data using one or more hotword detection models to generate:
a first predicted output that indicates a first probability of one or more synchronization hotwords being present in the first audio data, and
a second predicted output that indicates a second probability of one or more other hotwords being present in the first audio data, wherein the one or more other hotwords are distinct from the synchronization hotwords, and the one or more other hotwords are usable to activate speech recognition or natural language understanding of the automated assistant;
determining that the first predicted output satisfies a first threshold that is indicative of the one or more synchronization hotwords being present in the first audio data;
determining that there is a match between an attribute of the user that is identified as a joint secret and stored on each of the first computing device and the second computing device, wherein the determining includes causing the joint secret (i) to be used to generate encrypted data on one of the first and second computing devices and (ii) to be used to decrypt the encrypted data on the other of the first and second computing devices, wherein the joint secret is based on a date of birth of the user, an email address of the user, a phone number of the user, or preferences of the user; and
in response to determining that the first predicted output satisfies the first threshold and that there is a match between the attribute of the user identified as the joint secret, initiating peer-to-peer bidirectional synchronization of configuration data between the first instance of the automated assistant on the first computing device and the second instance of the automated assistant on the second computing device, wherein the bidirectional synchronization occurs at the first computing device and the second computing device.
2 . The method according to claim 1 , wherein the same first audio data is processed using one or more of the hotword detection models to determine both the first probability and the second probability.
3 . The method according to claim 1 , further comprising, prior to initiating the bidirectional synchronization of the configuration data, determining that there is a match between registered user profiles stored on each of the first computing device and the second computing device.
4 . The method according to claim 1 , wherein the joint secret is used to process a random string to generate the encrypted data.
5 . The method according to claim 1 , wherein the bidirectional synchronization includes bidirectional synchronization of user data, and wherein the user data that is synchronized comprises a personalized hotword model, a personalized speech recognition model, or a personalized query interpretation model.
6 . The method according to claim 1 , wherein the bidirectional synchronization includes bidirectional synchronization of user data, and wherein the user data that is synchronized comprises recently issued queries of the user.
7 . The method according to claim 1 , wherein the bidirectional synchronization includes bidirectional synchronization of user data, and wherein the user data that is synchronized comprises preferences of the user.
8 . The method according to claim 1 , wherein the bidirectional synchronization includes bidirectional synchronization of user data, and wherein the user data that is synchronized comprises a private knowledge graph of the user.
9 . The method according to claim 1 , wherein the bidirectional synchronization includes bidirectional synchronization of user data, and wherein the user data comprises data that is selected for synchronization based on capabilities of each of the first computing device and the second computing device.
10 . The method according to claim 1 , wherein the configuration data that is synchronized comprises settings of the first computing device or settings of the second computing device.
11 . The method according to claim 1 , further comprising:
executing the second instance of the automated assistant in the inactive state at least in part on the second computing device;
while in the inactive state, receiving, via one or more microphones of the second computing device, second audio data that captures the spoken utterance of the user;
processing the second audio data to generate a second predicted output that indicates a probability of the one or more synchronization hotwords being present in the second audio data; and
determining that the second predicted output satisfies a second threshold that is indicative of the one or more synchronization hotwords being present in the second audio data,
wherein initiating the bidirectional synchronization of the configuration data is further in response to determining the second predicted output satisfies the second threshold.
12 . A computer program product comprising one or more non-transitory computer-readable storage media having program instructions collectively stored on the one or more non-transitory computer-readable storage media, the program instructions executable to:
execute a first instance of an automated assistant in an inactive state at least in part on a first computing device operated by a user;
while in the inactive state, receive, via one or more microphones of the first computing device, first audio data that captures a spoken utterance of the user;
while in the inactive state, process the first audio data using one or more hotword detection models to generate:
a first predicted output that indicates a first probability of one or more synchronization hotwords being present in the first audio data, and
a second predicted output that indicates a second probability of one or more other hotwords being present in the first audio data, wherein the one or more other hotwords are distinct from the synchronization hotwords, and the one or more other hotwords are usable to activate speech recognition or natural language understanding of the automated assistant;
determine that the first predicted output satisfies a first threshold that is indicative of the one or more synchronization hotwords being present in the first audio data;
determine that there is a match between an attribute of the user that is identified as a joint secret and stored on each of the first computing device and the second computing device, wherein the determining includes causing the joint secret (i) to be used to generate encrypted data on one of the first and second computing devices and (ii) to be used to decrypt the encrypted data on the other of the first and second computing devices, wherein the joint secret is based on a date of birth of the user, an email address of the user, a phone number of the user, or preferences of the user; and
in response to determining that the first predicted output satisfies the first threshold and that there is a match between the attribute of the user identified as the joint secret, initiate peer-to-peer bidirectional synchronization of configuration data between the first instance of the automated assistant on the first computing device and the second instance of the automated assistant on the second computing device, wherein the bidirectional synchronization occurs at the first computing device and the second computing device.
13 . The computer program product according to claim 12 , wherein the same first audio data is processed using one or more of the hotword detection models to determine both the first probability and the second probability.
14 . The computer program product according to claim 12 , wherein the program instructions are further executable to, prior to initiating the bidirectional synchronization of the configuration data, determine that there is a match between registered user profiles stored on each of the first computing device and the second computing device.
15 . The computer program product according to claim 12 , wherein the joint secret is used to process a random string to generate the encrypted data.
16 . A system comprising:
a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
execute a first instance of an automated assistant in an inactive state at least in part on a first computing device operated by a user;
while in the inactive state, receive, via one or more microphones of the first computing device, first audio data that captures a spoken utterance of the user;
while in the inactive state, process the first audio data using one or more hotword detection models to generate:
a first predicted output that indicates a first probability of one or more synchronization hotwords being present in the first audio data, and
a second predicted output that indicates a second probability of one or more other hotwords being present in the first audio data, wherein the one or more other hotwords are distinct from the synchronization hotwords, and the one or more other hotwords are usable to activate speech recognition or natural language understanding of the automated assistant;
determine that the first predicted output satisfies a first threshold that is indicative of the one or more synchronization hotwords being present in the first audio data;
determine that there is a match between an attribute of the user that is identified as a joint secret and stored on each of the first computing device and the second computing device, wherein the determining includes causing the joint secret (i) to be used to generate encrypted data on one of the first and second computing devices and (ii) to be used to decrypt the encrypted data on the other of the first and second computing devices, wherein the joint secret is based on a date of birth of the user, an email address of the user, a phone number of the user, or preferences of the user; and
in response to determining that the first predicted output satisfies the first threshold and that there is a match between the attribute of the user identified as the joint secret, initiate peer-to-peer bidirectional synchronization of configuration data between the first instance of the automated assistant on the first computing device and the second instance of the automated assistant on the second computing device, wherein the bidirectional synchronization occurs at the first computing device and the second computing device.
17 . The system according to claim 16 , wherein the same first audio data is processed using one or more of the hotword detection models to determine both the first probability and the second probability.
18 . The system according to claim 16 , wherein the program instructions are further executable to, prior to initiating the bidirectional synchronization of the configuration data, determine that there is a match between registered user profiles stored on each of the first computing device and the second computing device.
19 . The system according to claim 16 , wherein the joint secret is used to process a random string to generate the encrypted data.