Granted Patent B2

US 10,199,051 · App. 14/175,864 · Granted Feb 5, 2019

Inventors: Justin Binder (Oakland, CA); Samuel D. Post (San Jose, CA); Onur Tackin (Sunnyvale, CA); Thomas R. Gruber (Emerald Hills, CA)

Assignee: APPLE INC.

G10L21/16G06F3/167G10L15/265G10L17/24G10L15/22G10L15/30

Abstract

Claims (88)

1. A method for operating a voice trigger, performed at an electronic device including one or more processors and memory storing instructions for execution by the one or more processors, the method comprising:

determining, based on comparing an amount of light detected on at least a front surface of the electronic device to a threshold amount of light, whether to operate the voice trigger in a standby mode or in a listening mode;

in accordance with a determination to operate the voice trigger in the listening mode:

receiving a sound input;

generating an input representation of the sound input, wherein the input representation represents audio signatures of the sound input;

determining whether at least a portion of the sound input corresponds to a predetermined type of sound;

upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content based on comparing of the input representation of the sound input to one or more reference representations;

upon a determination that the sound input includes the predetermined content, generating a control signal comprising instructions to initiate a speech-based service; and

initiating the speech-based service based on the control signal; and

in accordance with a determination to operate the voice trigger in the standby mode, forgoing initiating the speech-based service based on received sound input.

2. The method of claim 1 , wherein said determining whether the sound input corresponds to a predetermined type of sound is performed by a first sound detector, wherein said determining whether the sound input includes predetermined content is performed by a second sound detector, and wherein the first sound detector consumes less power while operating than the second sound detector.

3. The method of claim 2 , wherein the second sound detector is initiated in response to a determination by the first sound detector that the sound input corresponds to the predetermined type.

4. The method of claim 2 , wherein the second sound detector is operated for at least a predetermined amount of time after the determination by the first sound detector that the sound input corresponds to the predetermined type.

5. The method of claim 1 , wherein the predetermined type is a human voice and the predetermined content is one or more words.

6. The method of claim 1 , wherein the predetermined content is one or more predetermined phonemes.

7. The method of claim 6 , wherein the one or more predetermined phonemes constitute at least one word.

8. The method of claim 1 , further comprising, prior to determining whether the sound input corresponds to the predetermined type of sound, determining whether the sound input satisfies a predetermined condition.

9. The method of claim 8 , wherein the predetermined condition is an amplitude threshold.

10. The method of claim 8 , wherein said determining whether the sound input satisfies a predetermined condition is performed by a third sound detector, wherein the third sound detector consumes less power while operating than a first sound detector, the first sound detector being configured to determine whether the sound input corresponds to the predetermined type of sound.

11. The method of claim 1 , further comprising:

storing at least a portion of the sound input in memory; and

providing the portion of the sound input to the speech-based service once the speech-based service is initiated.

12. The method of claim 1 , further comprising determining whether the sound input corresponds to a voice of a particular user.

13. The method of claim 12 , wherein the speech-based service is initiated upon a determination that the sound input includes the predetermined content and that the sound input corresponds to the voice of the particular user.

14. The method of claim 13 , wherein the speech-based service is initiated in a limited access mode upon a determination that the sound input includes the predetermined content and that the sound input does not correspond to the voice of the particular user.

15. The method of claim 13 , further comprising, upon a determination that the sound input corresponds to the voice of the particular user, outputting a voice prompt including a name of the particular user.

16. The method of claim 1 , further comprising:

determining whether the electronic device is in a predetermined orientation; and

upon a determination that the electronic device is in the predetermined orientation, activating a predetermined mode of the voice trigger.

17. A non-transitory computer-readable storage medium, storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for:

determining, based on comparing an amount of light detected on at least a front surface of the electronic device to a threshold amount of light, whether to operate a voice trigger in a standby mode or in a listening mode;

in accordance with a determination to operate the voice trigger in the listening mode:

receiving a sound input;

generating an input representation of the sound input, wherein the input representation represents audio signatures of the sound input;

determining whether at least a portion of the sound input corresponds to a predetermined type of sound;

upon a determination that at least a portion of the sound input corresponds to the predetermined type, determining whether the sound input includes predetermined content based on comparing of the input representation of the sound input to one or more reference representations;

upon a determination that the sound input includes the predetermined content, generating a control signal comprising instructions to initiate a speech based service; and

initiating the speech-based service based on the control signal; and

in accordance with a determination to operate the voice trigger in the standby mode, forgoing initiating the speech-based service based on received sound input.

18. An electronic device, comprising:

a sound receiving unit configured to receive sound input; and

a processing unit coupled to the sound receiving unit, the processing unit configured to:

determine, based on comparing an amount of light detected on at least a front surface of the electronic device to a threshold amount of light, whether to operate a voice trigger in a standby mode or in a listening mode;

in accordance with a determination to operate the voice trigger in the listening mode:

generate an input representation of the sound input, wherein the input representation represents audio signatures of the sound input;

determine whether at least a portion of the sound input corresponds to a predetermined type of sound;

upon a determination that at least a portion of the sound input corresponds to the predetermined type, determine whether the sound input includes predetermined content based on comparing of the input representation of the sound input to one or more reference representations;

upon a determination that the sound input includes the predetermined content, generate a control signal comprising instructions to initiate a speech-based service; and

initiate the speech-based service based on the control signal; and

in accordance with a determination to operate the voice trigger in the standby mode, forgo initiating the speech-based service based on received sound input.

19. The electronic device of claim 18 , the processing unit further configured to, prior to determining whether the sound input corresponds to the predetermined type of sound, determine whether the sound input satisfies a predetermined condition.

20. The non-transitory computer-readable storage medium of claim 17 , wherein said determining whether the sound input corresponds to a predetermined type of sound is performed by a first sound detector, wherein said determining whether the sound input includes predetermined content is performed by a second sound detector, and wherein the first sound detector consumes less power while operating than the second sound detector.

21. The non-transitory computer-readable storage medium of claim 20 , wherein the second sound detector is initiated in response to a determination by the first sound detector that the sound input corresponds to the predetermined type.

22. The non-transitory computer-readable storage medium of claim 20 , wherein the second sound detector is operated for at least a predetermined amount of time after the determination by the first sound detector that the sound input corresponds to the predetermined type.

23. The non-transitory computer-readable storage medium of claim 17 , wherein the predetermined type is a human voice and the predetermined content is one or more words.

24. The non-transitory computer-readable storage medium of claim 17 , wherein the predetermined content is one or more predetermined phonemes.

25. The non-transitory computer-readable storage medium of claim 24 , wherein the one or more predetermined phonemes constitute at least one word.

26. The non-transitory computer-readable storage medium of claim 17 , the one or more programs further including instructions for, prior to determining whether the sound input corresponds to the predetermined type of sound, determining whether the sound input satisfies a predetermined condition.

27. The non-transitory computer-readable storage medium of claim 26 , wherein the predetermined condition is an amplitude threshold.

28. The non-transitory computer-readable storage medium of claim 26 , wherein said determining whether the sound input satisfies a predetermined condition is performed by a third sound detector, wherein the third sound detector consumes less power while operating than a first sound detector, the first sound detector being configured to determine whether the sound input corresponds to the predetermined type of sound.

29. The non-transitory computer-readable storage medium of claim 17 , the one or more programs further including instructions for:

storing at least a portion of the sound input in memory; and

providing the portion of the sound input to the speech-based service once the speech-based service is initiated.

30. The non-transitory computer-readable storage medium of claim 17 , the one or more programs further including instructions for determining whether the sound input corresponds to a voice of a particular user.

31. The non-transitory computer-readable storage medium of claim 30 , wherein the speech-based service is initiated upon a determination that the sound input includes the predetermined content and that the sound input corresponds to the voice of the particular user.

32. The non-transitory computer-readable storage medium of claim 31 , wherein the speech-based service is initiated in a limited access mode upon a determination that the sound input includes the predetermined content and that the sound input does not correspond to the voice of the particular user.

33. The non-transitory computer-readable storage medium of claim 31 , wherein the one or more programs include further instructions for, upon a determination that the sound input corresponds to the voice of the particular user, outputting a voice prompt including a name of the particular user.

34. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more programs include further instructions for:

determining whether the electronic device is in a predetermined orientation; and

upon a determination that the electronic device is in the predetermined orientation, activating a predetermined mode of the voice trigger.

35. The electronic device of claim 19 , wherein the predetermined condition is an amplitude threshold.

36. The electronic device of claim 19 , wherein said determining whether the sound input satisfies a predetermined condition is performed by a third sound detector, wherein the third sound detector consumes less power while operating than a first sound detector, the first sound detector being configured to determine whether the sound input corresponds to the predetermined type of sound.

37. The electronic device of claim 18 , wherein said determining whether the sound input corresponds to a predetermined type of sound is performed by a first sound detector, wherein said determining whether the sound input includes predetermined content is performed by a second sound detector, and wherein the first sound detector consumes less power while operating than the second sound detector.

38. The electronic device of claim 37 , wherein the second sound detector is initiated in response to a determination by the first sound detector that the sound input corresponds to the predetermined type.

39. The electronic device of claim 37 , wherein the second sound detector is operated for at least a predetermined amount of time after the determination by the first sound detector that the sound input corresponds to the predetermined type.

40. The electronic device of claim 18 , wherein the predetermined type is a human voice and the predetermined content is one or more words.

41. The electronic device of claim 18 , wherein the predetermined content is one or more predetermined phonemes.

42. The electronic device of claim 41 , wherein the one or more predetermined phonemes constitute at least one word.

43. The electronic device of claim 18 , the processing unit further configured to:

store at least a portion of the sound input in memory; and

provide the portion of the sound input to the speech-based service once the speech-based service is initiated.

44. The electronic device of claim 18 , the processing unit further configured to determine whether the sound input corresponds to a voice of a particular user.

45. The electronic device of claim 44 , wherein the speech-based service is initiated upon a determination that the sound input includes the predetermined content and that the sound input corresponds to the voice of the particular user.

46. The electronic device of claim 45 , wherein the speech-based service is initiated in a limited access mode upon a determination that the sound input includes the predetermined content and that the sound input does not correspond to the voice of the particular user.

47. The electronic device of claim 45 , the processing unit further configured to: upon a determination that the sound input corresponds to the voice of the particular user, output a voice prompt including a name of the particular user.

48. The electronic device of claim 18 , the processing unit further configured to:

determine whether the electronic device is in a predetermined orientation; and

upon a determination that the electronic device is in the predetermined orientation, activate a predetermined mode of the voice trigger.

Assignments (1)

Continuity (2)

Cited By (76)

US 12,189,748 US 12,197,712 US 12,197,817 US 12,198,413 US 12,198,430 US 12,200,297 US 12,204,932 US 12,210,603 US 12,211,502 US 12,216,754 US 12,216,894 US 12,219,314 US 12,223,282 US 12,230,264 US 12,236,952 US 12,243,522 US 12,249,014 US 12,254,887 US 12,260,234 US 12,262,111 US 12,277,954 US 12,293,203 US 12,293,763 US 12,293,764 US 12,299,755 US 12,301,635 US 12,314,527 US 12,327,552 US 12,333,404 US 12,353,897 US 12,361,943 US 12,367,879 US 12,374,097 US 12,374,330 US 12,386,434 US 12,386,491 US 12,387,619 US 12,406,316 US 12,406,490 US 12,417,596 US 12,423,917 US 12,431,128 US 12,443,797 US 12,462,005 US 12,475,170 US 12,475,698 US 12,477,470 US 12,495,098 US 12,518,323 US 12,549,203 US 12,554,815 US 12,556,890 US 12,561,032 US 12,567,404 US 12,567,415 US 12,572,198 US 12,573,391 US 12,574,627 US 12,592,017 US 12,608,171 US 12,613,621 US 12,613,730 US 12,619,304 US 12,619,452 US 12,619,701 US 12,620,179 US 12,625,546 US 12,625,557 US 12,640,151 US 12,640,154 US 12,645,778 US 12,664,969 US 12,670,639 US 12,670,671 US 12,675,839 US 12,694,221