Method and apparatus for evaluating trigger phrase enrollment
An electronic device includes a microphone that receives an audio signal that includes a spoken trigger phrase, and a processor that is electrically coupled to the microphone. The processor measures characteristics of the audio signal, and determines, based on the measured characteristics, whether the spoken trigger phrase is acceptable for trigger phrase model training. If the spoken trigger phrase is determined not to be acceptable for trigger phrase model training, the processor rejects the trigger phrase for trigger phrase model training.
1. A computer-implemented method comprising:
during a trigger phrase enrollment process:
prompting, by a speech recognition-enabled electronic device, a user of the speech recognition-enabled electronic device to speak a trigger phrase;
receiving, at the speech recognition-enabled electronic device, a first audio signal corresponding to the user speaking the trigger phrase;
determining, by the speech recognition-enabled electronic device, a first number of frames in the first audio signal that have a measure of noise variability of background noise exceeding a noise variability threshold;
determining, by the speech recognition-enabled electronic device, a second number of frames in the first audio signal that have voice activity; and
when the first number of frames in the first audio signal is less than a threshold value and when the second number of frames in the first audio signal is within a frame count threshold range, training, by the speech recognition-enabled electronic device, a trigger phrase model with the first audio signal corresponding to the user speaking the trigger phrase, the trigger phrase model configured to detect the trigger phrase in a spoken utterance; and
after the trigger phrase enrollment process:
receiving, at the speech recognition-enabled electronic device and while the speech recognition-enabled electronic device is in a sleep mode, a second audio signal including an utterance of the trigger phrase spoken by the user; and
detecting, by the speech recognition-enabled electronic device and using the trigger phrase model trained during the trigger phrase enrollment process, the utterance of the trigger phrase in the second audio signal, the trigger phrase when detected in the second audio signal causing the speech recognition-enabled electronic device to wake from the sleep mode, the sleep mode comprising a power-saving mode of operation in which one or more parts of the speech recognition-enabled electronic device are in a low-power state or powered off.
2. The computer-implemented method of claim 1 , further comprising, when the first number of frames in the first audio signal exceed the threshold value and when the second number of frames in the first audio signal is outside the frame count threshold range:
prompting, by the speech recognition-enabled electronic device, the user to speak the trigger phrase again; and
rejecting, by the speech recognition-enabled electronic device, the first audio signal corresponding to the user speaking the trigger phrase for use in training the trigger phrase model.
3. The computer-implemented method of claim 1 , further comprising, for each frame in the received first audio signal:
identifying, by the speech recognition-enabled electronic device, audio characteristics of the user speaking the trigger phrase and audio characteristics of background noise for the corresponding frame in the received first audio signal;
comparing, by the speech recognition-enabled electronic device, the identified audio characteristics of the user speaking the trigger phrase to predetermined criteria; and
determining, by the speech recognition-enabled electronic device, a voice activity detection flag for the corresponding frame in the received first audio signal in response to comparing the identified audio characteristics of the user speaking the trigger phrase to the predetermined criteria.
4. The computer-implemented method of claim 3 , wherein determining the voice activity detection flag for the corresponding frame in the received first audio signal comprises:
generating, an accept enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being less than the predetermined threshold values; and
generating, a reject enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being greater than the predetermined threshold values.
5. The computer-implemented method of claim 4 , further comprising determining, by the speech recognition-enabled electronic device, a length of the user speaking the trigger phrase in the received first audio signal by counting a number of frames in the received first audio signal that comprise the accept enrollment flag.
6. The computer-implemented method of claim 5 , further comprising:
comparing, by the speech recognition-enabled electronic device, the length of the user speaking the trigger phrase to a lower phrase length threshold and to a higher phrase length threshold; and
when the length of the user speaking the hotword is less than the lower phrase length threshold or greater than the higher phrase threshold, prompting, by the speech recognition-enabled electronic device, the user to speak the trigger phrase in a second attempt.
7. The computer-implemented method of claim 6 , wherein the lower phrase length threshold is less than 70 frames in which the accept enrollment flag is found, the higher phrase length threshold is greater than 180 frames in which the accept enrollment flag is found, and each frame is 10 milliseconds in duration.
8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
during a trigger phrase enrollment process:
prompting a user of a speech recognition-enabled electronic device to speak a trigger phrase;
receiving a first audio signal corresponding to the user speaking the trigger phrase;
determining a first number of frames in the first audio signal that have a measure of noise variability of background noise exceeding a noise variability threshold;
determining a second number of frames in the first audio signal that have voice activity; and
when the first number of frames in the first audio signal is less than a threshold value and when the second number of frames in the first audio signal is within a frame count threshold range, training a trigger phrase model with the first audio signal corresponding to the user speaking the trigger phrase, the trigger phrase model configured to detect the trigger phrase in a spoken utterance; and
after the trigger phrase enrollment process:
receiving, while the speech recognition-enabled electronic device is in a sleep mode, a second audio signal including an utterance of the trigger phrase spoken by the user; and
detecting, using the trigger phrase model trained during the trigger phrase enrollment process, the utterance of the trigger phrase in the second audio signal, the trigger phrase when detected in the second audio signal causing the speech recognition-enabled electronic device to wake from the sleep mode, the sleep mode comprising a power-saving mode of operation in which one or more parts of the speech recognition-enabled electronic device are in a low-power state or powered off.
9. The system of claim 8 , wherein the operations further comprise, when the first number of frames in the first audio signal exceed the threshold value and when the second number of frames in the first audio signal is outside the frame count threshold range:
prompting the user to speak the trigger phrase again; and
rejecting the first audio signal corresponding to the user speaking the trigger phrase for use in training the trigger phrase model.
10. The system of claim 8 , wherein the operations further comprise, for each frame in the received first audio signal:
identifying audio characteristics of the user speaking the trigger phrase and audio characteristics of background noise for the corresponding frame in the received first audio signal;
comparing the identified audio characteristics of the user speaking the trigger phrase to predetermined criteria; and
determining a voice activity detection flag for the corresponding frame in the received first audio signal in response to comparing the identified audio characteristics of the user speaking the trigger phrase to the predetermined criteria.
11. The system of claim 10 , wherein determining the voice activation detection flag for the corresponding frame in the received first audio signal comprises:
generating an accept enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being less than predetermined threshold values; and
generating a reject enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being greater than the predetermined threshold values.
12. The system of claim 11 , wherein the operations further comprise determining a length of the user speaking the trigger phrase in the received first audio signal by counting a number of frames in the received first audio signal that comprise the accept enrollment flag.
13. The system of claim 12 , wherein the operations further comprise:
comparing the length of the user speaking the trigger phrase to a lower phrase length threshold and to a higher phrase length threshold; and
when the length of the user speaking the hotword is less than the lower phrase length threshold or greater than the higher phrase threshold, prompting the user to speak the trigger phrase in a second attempt.
14. The system of claim 13 , wherein the lower phrase length threshold is less than 70 frames in which the accept enrollment flag is found, the higher phrase length threshold is greater than 180 frames in which the accept enrollment flag is found, and each frame is 10 milliseconds in duration.
15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
during a trigger phrase enrollment process:
prompting a user of a speech recognition-enabled electronic device to speak a trigger phrase;
receiving a first audio signal corresponding to the user speaking the trigger phrase;
determining a first number of frames in the first audio signal that have a measure of noise variability of background noise exceeding a noise variability threshold;
determining a second number of frames in the first audio signal that have voice activity; and
when the first number of frames in the first audio signal is less than a threshold value and when the second number of frames in the first audio signal is within a frame count threshold range, training a trigger phrase model with the first audio signal corresponding to the user speaking the trigger phrase, the trigger phrase model configured to detect the trigger phrase in a spoken utterance; and
after the trigger phrase enrollment process:
receiving, while the speech recognition-enabled electronic device is in a sleep mode, a second audio signal including an utterance of the trigger phrase spoken by the user; and
detecting, using the trigger phrase model trained during the trigger phrase enrollment process, the utterance of the trigger phrase in the second audio signal, the trigger phrase when detected in the second audio signal causing the speech recognition-enabled electronic device to wake from the sleep mode, the sleep mode comprising a power-saving mode of operation in which one or more parts of the speech recognition-enabled electronic device are in a low-power state or powered off.
16. The computer-readable medium of claim 15 , wherein the operations further comprise, when the first number of frames in the first audio signal exceed the threshold value and when the second number of frames in the first audio signal is outside the frame count threshold range:
prompting the user to speak the trigger phrase again; and
rejecting the first audio signal corresponding to the user speaking the trigger phrase for use in training the trigger phrase model.
17. The computer-readable medium of claim 15 , wherein the operations further comprise:
identifying audio characteristics of the user speaking the trigger phrase and audio characteristics of background noise for the corresponding frame in the received first audio signal;
comparing the identified audio characteristics of the user speaking the trigger phrase to predetermined criteria; and
determining a voice activity detection flag for the corresponding frame in the received first audio signal in response to comparing the identified audio characteristics of the user speaking the trigger phrase to the predetermined criteria.
18. The computer-readable medium of claim 17 , wherein determining the voice activation detection flag for the corresponding frame in the received first audio signal comprises:
generating an accept enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being less than predetermined threshold values; and
generating a reject enrollment flag in response to the identified audio characteristics of the user speaking the trigger phrase being greater than the predetermined threshold values.
19. The computer-readable medium of claim 18 , wherein the operations further comprise determining a length of the user speaking the trigger phrase in the received first audio signal by counting a number of frames in the received first audio signal that comprise the accept enrollment flag.
20. The computer-readable medium of claim 19 , wherein the operations further comprise:
comparing the length of the user speaking the trigger phrase to a lower phrase length threshold and to a higher phrase length threshold; and
when the length of the user speaking the hotword is less than the lower phrase length threshold or greater than the higher phrase threshold, prompting the user to speak the trigger phrase in a second attempt.