Voice control user interface during low-power mode
A method include placing a first processor in a sleep operating mode and running a second processor that is operative to wake the first processor from the sleep operating mode in response to a speech command phrase. The method includes identifying, by the second processor, a speech command phrase segment and performing a control operation in response to detecting the segment in detected speech. The control operation is performed while the first processor is maintained in the sleep operating mode.
1. A computer-implemented method comprising:
during operation of a mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced:
receiving, by data processing hardware of the mobile device, an utterance;
while a display of the mobile device is off, detecting, by the data processing hardware, an initial portion of a predetermined command phrase in the utterance by matching the initial portion of the predetermined command phrase to a segment speech model associated with the predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance, and the segment speech model storing each word of the series of multiple words of the predetermined command phrase; and
in response to detecting the initial portion of the predetermined command phrase in the utterance:
turning on, by the data processing hardware, the display of the mobile device; and
displaying, by the data processing hardware for output on the display, at least a portion of a graphical user interface that was stored in a frame buffer of the mobile device before receiving the utterance.
2. The method of claim 1 , wherein the segment speech model is derived from an untrained speech model that contains a default command phrase comprising the predetermined command phrase.
3. The method of claim 2 , wherein the segment speech model is derived from a trained speech model created by a user of the mobile device and containing a user-created command phrase spoken by the user of the mobile device, the user-created command phrase comprising the predetermined command phrase.
4. The method of claim 1 , further comprising, in response to detecting the initial portion of the predetermined command phrase in the utterance:
determining, by the data processing hardware, whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and
when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating, by the data processing hardware, the display of the mobile device a second predetermined period of time after displaying the at least the portion of the graphical user interface for output on the display.
5. The method of claim 4 , further comprising, when the remaining portion of the predetermined command phrase is detected in the second utterance within the second predetermined period of time, commanding, by the data processing hardware, the mobile device to wake-up from the low-power mode and operate in an awake mode.
6. The method of claim 5 , further comprising, during operation of the mobile device in the awake mode in which full functionality of the speech recognition engine is provided, detecting, by the data processing hardware, a voice command in the second utterance that follows the predetermined command phrase.
7. The method of claim 6 , further comprising, after detecting the voice command in the second utterance that follows the predetermined command phrase, performing, by the data processing hardware, an action that corresponds to the voice command.
8. The method of claim 1 , wherein the segment speech model is stored in an on-board memory of the mobile device.
9. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
during operation of a mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced:
receiving an utterance;
while a display of the mobile device is off, detecting an initial portion of a predetermined command phrase in the utterance by matching the initial portion of the predetermined command phrase to a segment speech model associated with the predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance, and the segment speech model storing each word of the series of multiple words of the predetermined command phrase; and
in response to detecting the initial portion of the predetermined command phrase in the utterance:
turning on the display of the mobile device; and
displaying, for output on the display, at least a portion of a graphical user interface that was stored in a frame buffer of the mobile device before receiving the utterance.
10. The system of claim 9 , wherein the segment speech model is derived from an untrained speech model that contains a default command phrase comprising the predetermined command phrase.
11. The system of claim 10 , wherein the segment speech model is derived from a trained speech model created by a user of the mobile device and containing a user-created command phrase spoken by the user of the mobile device, the user-created command phrase comprising the predetermined command phrase.
12. The system of claim 9 , wherein the operations further comprise, in response to detecting the initial portion of the predetermined command phrase in the utterance:
determining whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and
when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating the display of the mobile device a second predetermined period of time after displaying the at least the portion of the graphical user interface for output on the display.
13. The system of claim 12 , wherein the operations further comprise, when the remaining portion of the predetermined command phrase is detected in the second utterance within the second predetermined period of time, commanding the mobile device to wake-up from the low-power mode and operate in an awake mode.
14. The system of claim 13 , wherein the operations further comprise, during operation of the mobile device in the awake mode in which full functionality of the speech recognition engine is provided, detecting a voice command in the second utterance that follows the predetermined command phrase.
15. The system of claim 14 , wherein the operations further comprise, after detecting the voice command in the second utterance that follows the predetermined command phrase, performing an action that corresponds to the voice command.
16. The system of claim 9 , wherein the segment speech model is stored in an on-board memory of the mobile device.
17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
during operation of a mobile device in a low-power mode in which functionality of a voice recognition engine of the mobile device is reduced:
receiving an utterance;
while a display of the mobile device is off, detecting an initial portion of a predetermined command phrase in the utterance by matching the initial portion of the predetermined command phrase to a segment speech model associated with the predetermined command phrase, the predetermined command phrase comprising a series of multiple words configured to cause the mobile device to wake-up from the low-power mode in response to detecting each word of the series of multiple words in the utterance, and the segment speech model storing each word of the series of multiple words of the predetermined command phrase; and
in response to detecting the initial portion of the predetermined command phrase in the utterance:
turning on the display of the mobile device; and
displaying, for output on the display, at least a portion of a graphical user interface that was stored in a frame buffer of the mobile device before receiving the utterance.
18. The non-transitory computer-readable medium of claim 17 , wherein the segment speech model is derived from an untrained speech model that contains a default command phrase comprising the predetermined command phrase.
19. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise, in response to detecting the initial portion of the predetermined command phrase in the utterance:
determining whether a remaining portion of the predetermined command phrase is detected in the utterance within a first predetermined period of time after detecting the initial portion of the predetermined command phrase in the utterance; and
when the remaining portion of the predetermined command phrase is not detected in the utterance within the first predetermined period of time, automatically deactivating the display of the mobile device a second predetermined period of time after displaying the at least the portion of the graphical user interface for output on the display.
20. The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise when the remaining portion of the predetermined command phrase is detected in the second utterance within the second predetermined period of time, commanding the mobile device to wake-up from the low-power mode and operate in an awake mode.