Short-lived repeat voice commands

1 . A computer-implemented method when executed by data processing hardware causes the data processing hardware to perform operations comprising:

obtaining a voice command log containing prior instances of a user of an assistant-enabled device speaking repeat voice commands for controlling a long-standing operation performed by a digital assistant immediately after detecting warm words from a set of one or more warm words in prior audio data;

activating the set of one or more warm words, wherein each corresponding warm word in the activated set of one or more warm words is associated with:

a respective action for controlling the long-standing operation performed by the digital assistant; and

a respective set of repeat warm words, wherein:

two or more repeat warm words in the respective set of repeat warm words is associated with the same respective action for controlling the long-standing operation as the corresponding warm word; and

at least one repeat warm word in the respective set of repeat warm words is learned based on the repeat voice commands; and

while the digital assistant is performing the long-standing operation and the set of one or more warm words are activated:

receiving audio data corresponding to a first utterance captured by an assistant-enabled device;

detecting, in the audio data, a warm word from the activated set of one or more warm words;

in response to detecting the warm word from the activated set of one or more warm words:

performing the respective action associated with the detected warm word for controlling the long-standing operation; and

activating the respective set of repeat warm words associated with the detected warm word for a limited duration of time after the warm word is detected in the audio data;

receiving additional audio data corresponding to a second utterance captured by the assistant-enabled device;

detecting, in the additional audio data, a repeat warm word from the activated set of repeat warm words, the detected repeat warm word associated with the same respective action as the performed respective action associated with the detected warm word for controlling the long-standing operation; and

in response to detecting the repeat warm word from the activated set of repeat warm words, performing the same respective action associated with the detected repeat warm word for controlling the long-standing operation.

2 . The method of claim 1 , wherein:

activating the set of repeat warm words comprises activating, for each corresponding repeat warm word in the activated set of repeat warm words, a respective repeat warm word model to run on the assistant-enabled device; and

detecting, in the additional audio data, the repeat warm word from the activated set of repeat warm words comprises detecting, using the respective repeat warm word model activated for the corresponding detected repeat warm word, the repeat warm word in the additional audio data without performing speech recognition on the additional audio data.

3 . The method of claim 2 , wherein detecting the repeat warm word in the additional audio data comprises:

extracting audio features of the additional audio data;

generating, using the respective repeat warm word model activated for the corresponding detected repeat warm word, a repeat warm word confidence score by processing the extracted audio features; and

determining that the additional audio data corresponding to the second utterance includes the corresponding repeat warm word when the repeat warm word confidence score satisfies a repeat warm word confidence threshold.

4 . The method of claim 2 , wherein:

detecting, in the audio data, the warm word from the activated set of one or more warm words comprises detecting, using a respective warm model activated for the corresponding warm word, the warm word in the audio data without performing speech recognition on the audio data; and

the respective repeat warm word model activated for the corresponding detected repeat warm word in the additional audio data is derived from the respective warm word model activated for detecting the corresponding warm word in the audio data.

5 . The method of claim 1 , wherein:

activating the set of repeat warm words comprises executing a speech recognizer on the assistant-enabled device, the speech recognizer biased to recognize the repeat warm words in the activated set of repeat warm words; and

detecting, in the additional audio data, the repeat warm word from the activated set of repeat warm words comprises recognizing, using the speech recognizer executing on the assistant-enabled device, the repeat warm word in the additional audio data.

6 . The method of claim 1 , wherein the operations further comprise receiving, in a user interface associated with the assistant-enabled device that presents a list of available repeat warm words to add and/or remove from the set of repeat warm words associated with the warm word and to be activated when the warm word is detected in the audio data, a user input indication indicating selection of one or more of the available repeat warm words to add to the set of repeat warm words.

7 . The method of claim 6 , wherein each corresponding available repeat warm word in the list of available repeat warm words presented by the user interface associated with the assistant-enabled device are displayed by the user interface as a respective graphical element that may be selected to add or remove the corresponding available repeat warm word to the set of repeat warm words associated with warm word and to be activated when the warm word is detected in the audio data.

8 . The method of claim 1 , wherein the operations further comprise:

prompting a user of the assistant-enabled device to add a repeat voice command contained in the voice command log to the set of repeat warm words associated with the detected warm word;

in response to receiving a user indication indicating an approval to add the repeat voice command to the set of repeat warm words, associating the repeat voice command with the detected warm word; and

activating the repeat voice command as a repeat warm word associated with the detected warm word.

9 . The method of claim 1 , wherein the respective action for controlling the long-standing operation that is associated with at least one repeat warm word in the activated set of repeat warm words is different than the respective action performed in response to detecting the associated warm word from the activated set of one or more warm words in the audio data.

10 . The method of claim 1 , wherein activating the set of repeat warm words associated with the detected warm word comprises activating the set of repeat warm words for a predetermined time duration.

11 . The method of claim 1 , wherein the operations further comprise, after performing the same respective action associated with the detected repeat warm word for controlling the long-standing operation, maintaining activation of at least one of the repeat warm words in the set of repeat warm words associated with the detected warm word.

12 . The method of claim 1 , wherein the operations further comprise, after performing the respective action associated with the detected repeat warm word for controlling the long-standing operation, deactivating the set of repeat warm words based on:

receiving a command that conflicts with the long-standing operation; or

failing to detect an additional repeat warm word of the activated set of repeat warm words after a predetermined time duration lapses.

13 . A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:

obtaining a voice command log containing prior instances of a user of an assistant-enabled device speaking repeat voice commands for controlling a long-standing operation performed by a digital assistant immediately after detecting warm words from a set of one or more warm words in prior audio data;

activating the set of one or more warm words, wherein each corresponding warm word in the activated set of one or more warm words is associated with:

a respective action for controlling the long-standing operation performed by the digital assistant; and

a respective set of repeat warm words, wherein:

two or more repeat warm words in the respective set of repeat warm words is associated with the same respective action for controlling the long-standing operation as the corresponding warm word; and

at least one repeat warm word in the respective set of repeat warm words is learned based on the repeat voice commands; and

while the digital assistant is performing the long-standing operation and the set of one or more warm words are activated:

receiving audio data corresponding to a first utterance captured by an assistant-enabled device;

detecting, in the audio data, a warm word from the activated set of one or more warm words;

in response to detecting the warm word from the activated set of one or more warm words:

performing the respective action associated with the detected warm word for controlling the long-standing operation; and

activating the respective set of repeat warm words associated with the detected warm word;

receiving additional audio data corresponding to a second utterance captured by the assistant-enabled device;

detecting, in the additional audio data, a repeat warm word from the activated set of repeat warm words, the detected repeat warm word associated with the same respective action as the performed respective action associated with the detected warm word for controlling the long-standing operation; and

in response to detecting the repeat warm word from the activated set of repeat warm words, performing the same respective action associated with the detected repeat warm word for controlling the long-standing operation.

14 . The system of claim 13 , wherein:

activating the set of repeat warm words comprises activating, for each corresponding repeat warm word in the activated set of repeat warm words, a respective repeat warm word model to run on the assistant-enabled device; and

detecting, in the additional audio data, the repeat warm word from the activated set of repeat warm words comprises detecting, using the respective repeat warm word model activated for the corresponding detected repeat warm word, the repeat warm word in the additional audio data without performing speech recognition on the additional audio data.

15 . The system of claim 14 , wherein detecting the repeat warm word in the additional audio data comprises:

extracting audio features of the additional audio data;

generating, using the respective repeat warm word model activated for the corresponding detected repeat warm word, a repeat warm word confidence score by processing the extracted audio features; and

determining that the additional audio data corresponding to the second utterance includes the corresponding repeat warm word when the repeat warm word confidence score satisfies a repeat warm word confidence threshold.

16 . The system of claim 14 , wherein:

detecting, in the audio data, the warm word from the activated set of one or more warm words comprises detecting, using a respective warm model activated for the corresponding warm word, the warm word in the audio data without performing speech recognition on the audio data; and

the respective repeat warm word model activated for the corresponding detected repeat warm word in the additional audio data is derived from the respective warm word model activated for detecting the corresponding warm word in the audio data.

17 . The system of claim 13 , wherein:

activating the set of repeat warm words comprises executing a speech recognizer on the assistant-enabled device, the speech recognizer biased to recognize the repeat warm words in the activated set of repeat warm words; and

detecting, in the additional audio data, the repeat warm word from the activated set of repeat warm words comprises recognizing, using the speech recognizer executing on the assistant-enabled device, the repeat warm word in the additional audio data.

18 . The system of claim 13 , wherein the operations further comprise receiving, in a user interface associated with the assistant-enabled device that presents a list of available repeat warm words to add and/or remove from the set of repeat warm words associated with the warm word and to be activated when the warm word is detected in the audio data, a user input indication indicating selection of one or more of the available repeat warm words to add to the set of repeat warm words.

19 . The system of claim 18 , wherein each corresponding available repeat warm word in the list of available repeat warm words presented by the user interface associated with the assistant-enabled device are displayed by the user interface as a respective graphical element that may be selected to add or remove the corresponding available repeat warm word to the set of repeat warm words associated with warm word and to be activated when the warm word is detected in the audio data.

20 . The system of claim 13 , wherein the operations further comprise:

prompting a user of the assistant-enabled device to add a repeat voice command contained in the voice command log to the set of repeat warm words associated with the detected warm word;

in response to receiving a user indication indicating an approval to add the repeat voice command to the set of repeat warm words, associating the repeat voice command with the detected warm word; and

activating the repeat voice command as a repeat warm word associated with the detected warm word.

21 . The system of claim 13 , wherein the respective action for controlling the long-standing operation that is associated with at least one repeat warm word in the activated set of repeat warm words is different than the respective action performed in response to detecting the associated warm word from the activated set of one or more warm words in the audio data.

22 . The system of claim 13 , wherein activating the set of repeat warm words associated with the detected warm word comprises activating the set of repeat warm words for a predetermined time duration.

23 . The system of claim 13 , wherein the operations further comprise, after performing the same respective action associated with the detected repeat warm word for controlling the long-standing operation, maintaining activation of at least one of the repeat warm words in the set of repeat warm words associated with the detected warm word.

24 . The system of claim 13 , wherein the operations further comprise, after performing the respective action associated with the detected repeat warm word for controlling the long-standing operation, deactivating the set of repeat warm words based on:

receiving a command that conflicts with the long-standing operation; or

failing to detect an additional repeat warm word of the activated set of repeat warm words after a predetermined time duration lapses.