IP Library › Granted Patent US 11,935,168
Granted Patent B1
US 11,935,168 · App. 17/714,590 · Granted Mar 19, 2024

Selective amplification of voice and interactive language simulator

Inventors: Shiraz Akmal (Playa Vista, CA); Aaron M. Burns (Sunnyvale, CA); Brad K. Herman (Culver City, CA)
Assignee: Apple Inc.
G06T11/60G10L15/005G10L15/187G10L15/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,935,168
App. No.
17/714,590
Granted
Mar 19, 2024
Kind
B1
Abstract

Systems and processes for operating a digital assistant are provided. An example method includes, at an electronic device having one or more processors and memory, receiving an audio input including an utterance, determining, based on a speaker profile, an identity of a speaker of the utterance, determining whether the identity of the speaker matches a predetermined identity, and in accordance with a determination that the identity of the speaker matches the predetermined identity selectively adjusting a volume of the utterance relative to a volume of other sound of the audio input and providing an output of the adjusted utterance.

Claims (109)

1. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

receiving an audio input including a user request in a first language;

determining a location included in the user request and a second language responsive to the request based on the location included in the user request;

procedurally generating a virtual environment including one or more virtual objects based on a seed corresponding to the location and a predetermined set of rules;

providing the procedurally generated virtual environment including the one or more objects labeled in the second language; and

providing a spoken output using the second language.

2. The non-transitory computer-readable storage medium of claim 1 , the one or more programs further including instructions for:

determining the second language from the user request with natural language processing; and

generating the virtual environment based on the determined location.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the predetermined set of rules includes a rule specifying types of objects to be generated.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the predetermined set of rules includes a rule specifying deterministic parameters for the one or more virtual objects.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more virtual objects are determined based on the user request.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more virtual objects are determined based on the second language.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more virtual objects are determined based on a user profile.

8. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more virtual objects are determined based on an interaction history between the user and a digital assistant.

9. The non-transitory computer-readable storage medium of claim 1 , wherein the one or more virtual objects are determined based on contextual data associated with a user.

10. The non-transitory computer-readable storage medium of claim 1 , the one or more programs further including instructions for:

detecting an input referencing a first object of the one or more virtual objects; and

in response to detecting the input referencing the first object, providing an output corresponding to the first object.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the output corresponding to the first object includes providing a label corresponding to the first object with the first language.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the input referencing the first object includes a pronunciation of the first object in the second language.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the output corresponding to the first object includes an evaluation of the pronunciation of the first object.

14. The non-transitory computer-readable storage medium of claim 1 , the one or more programs further including instructions for:

generating a virtual representation of a digital assistant; and

providing the virtual representation of the digital assistant in the virtual environment.

15. The non-transitory computer-readable storage medium of claim 14 , the one or more programs further including instructions for:

receiving a second audio input in the second language; and

providing a response, in the second language, to the second audio input with the virtual representation of the digital assistant.

16. The non-transitory computer-readable storage medium of claim 1 , the one or more programs further including instructions for:

while providing the virtual environment:

receiving a third audio input including a request to change the virtual environment;

determining one or more properties of the virtual environment to change based on the request; and

providing an updated virtual environment after changing the one or more properties of the virtual environment.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more properties of the virtual environment to change are one or more properties of the one or more objects.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the one or more properties of the virtual environment to change are the location of the virtual environment.

19. An electronic device comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving an audio input including a user request in a first language;

determining a location included in the user request and a second language responsive to the request based on the location included in the user request;

procedurally generating a virtual environment including one or more virtual objects based on a seed corresponding to the location and a predetermined set of rules;

providing the procedurally generated virtual environment including the one or more objects labeled in the second language; and

providing a spoken output using the second language.

20. The electronic device of claim 19 , the one or more programs further including instructions for:

determining the second language from the user request with natural language processing; and

generating the virtual environment based on the determined location.

21. The electronic device of claim 19 , wherein the predetermined set of rules includes a rule specifying types of objects to be generated.

22. The electronic device of claim 19 , wherein the predetermined set of rules includes a rule specifying deterministic parameters for the one or more virtual objects.

23. The electronic device of claim 19 , wherein the one or more virtual objects are determined based on the user request.

24. The electronic device of claim 19 , wherein the one or more virtual objects are determined based on the second language.

25. The electronic device of claim 19 , wherein the one or more virtual objects are determined based on a user profile.

26. The electronic device of claim 19 , wherein the one or more virtual objects are determined based on an interaction history between the user and a digital assistant.

27. The electronic device of claim 19 , wherein the one or more virtual objects are determined based on contextual data associated with a user.

28. The electronic device of claim 19 , the one or more programs further including instructions for:

detecting an input referencing a first object of the one or more virtual objects; and

in response to detecting the input referencing the first object, providing an output corresponding to the first object.

29. The electronic device of claim 28 , wherein the output corresponding to the first object includes providing a label corresponding to the first object with the first language.

30. The electronic device of claim 28 , wherein the input referencing the first object includes a pronunciation of the first object in the second language.

31. The electronic device of claim 30 , wherein the output corresponding to the first object includes an evaluation of the pronunciation of the first object.

32. The electronic device of claim 19 , the one or more programs further including instructions for:

generating a virtual representation of a digital assistant; and

providing the virtual representation of the digital assistant in the virtual environment.

33. The electronic device of claim 32 , the one or more programs further including instructions for:

receiving a second audio input in the second language; and

providing a response, in the second language, to the second audio input with the virtual representation of the digital assistant.

34. The electronic device of claim 19 , the one or more programs further including instructions for:

while providing the virtual environment:

receiving a third audio input including a request to change the virtual environment;

determining one or more properties of the virtual environment to change based on the request; and

providing an updated virtual environment after changing the one or more properties of the virtual environment.

35. The electronic device of claim 34 , wherein the one or more properties of the virtual environment to change are one or more properties of the one or more objects.

36. The electronic device of claim 34 , wherein the one or more properties of the virtual environment to change are the location of the virtual.

37. A method, comprising:

at an electronic device with one or more processors and memory:

receiving an audio input including a user request in a first language

determining a location included in the user request and a second language responsive to the request based on the location included in the user request;

procedurally generating a virtual environment including one or more virtual objects based on a seed corresponding to the location and a predetermined set of rules;

providing the procedurally generated virtual environment including the one or more objects labeled in the second language; and

providing a spoken output using the second language.

38. The method of claim 37 , further comprising:

determining the second language from the user request with natural language processing; and

generating the virtual environment based on the determined location.

39. The method of claim 37 , wherein the predetermined set of rules includes a rule specifying types of objects to be generated.

40. The method of claim 37 , wherein the predetermined set of rules includes a rule specifying deterministic parameters for the one or more virtual objects.

41. The method of claim 37 , wherein the one or more virtual objects are determined based on the user request.

42. The method of claim 37 , wherein the one or more virtual objects are determined based on the second language.

43. The method of claim 37 , wherein the one or more virtual objects are determined based on a user profile.

44. The method of claim 37 , wherein the one or more virtual objects are determined based on an interaction history between the user and a digital assistant.

45. The method of claim 37 , wherein the one or more virtual objects are determined based on contextual data associated with a user.

46. The method of claim 37 , the one or more programs further including instructions for:

detecting an input referencing a first object of the one or more virtual objects; and

in response to detecting the input referencing the first object, providing an output corresponding to the first object.

47. The method of claim 46 , wherein the output corresponding to the first object includes providing a label corresponding to the first object with the first language.

48. The method of claim 46 , wherein the input referencing the first object includes a pronunciation of the first object in the second language.

49. The method of claim 48 , wherein the output corresponding to the first object includes an evaluation of the pronunciation of the first object.

50. The method of claim 37 , further comprising:

generating a virtual representation of a digital assistant; and

providing the virtual representation of the digital assistant in the virtual environment.

51. The method of claim 50 , further comprising:

receiving a second audio input in the second language; and

providing a response, in the second language, to the second audio input with the virtual representation of the digital assistant.

52. The method of claim 37 , further comprising:

while providing the virtual environment:

receiving a third audio input including a request to change the virtual environment;

determining one or more properties of the virtual environment to change based on the request; and

providing an updated virtual environment after changing the one or more properties of the virtual environment.

53. The method of claim 52 , wherein the one or more properties of the virtual environment to change are one or more properties of the one or more objects.

54. The method of claim 52 , wherein the one or more properties of the virtual environment to change are the location of the virtual.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2022
From: AKMAL, SHIRAZ; BURNS, AARON M.; HERMAN, BRAD K.
To: APPLE INC.
Reel/Frame 059701/0629 →
Continuity (1)
Provisional Application 63188848 · May 14, 2021
Cited By (3)
US 12,632,590 US 12,701,023 US 12,749,497