IP Library › Granted Patent US 11,562,729
Granted Patent B2
US 11,562,729 · App. 17/355,047 · Granted Jan 24, 2023

Devices, methods, and user interfaces for adaptively providing audio outputs

Inventors: Mitchell R. Lerner (San Francisco, CA); Pedro Mari (Santa Cruz, CA)
Assignee: APPLE INC.
G10K11/17827G10K11/17873G10K11/17885G10K2210/1081G10K2210/3056
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,562,729
App. No.
17/355,047
Filed
Jun 22, 2021
Granted
Jan 24, 2023
Kind
B2
Art Unit
2653
USPC
381/71.6
Abstract

An electronic device includes one or more pose sensors for detecting a pose of a user of the electronic device relative to a first physical environment and is in communication with one or more audio output devices. While a first pose of the user meets first presentation criteria, the electronic device provides audio content at a first simulated spatial location relative to the user. The electronic device detects a change in the pose of the user from the first pose to a second pose. In response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria, the electronic device provides audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location.

Claims (100)

1. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by an electronic device that includes one or more pose sensors for detecting a pose of a user of the electronic device relative to a first physical environment and that is in communication with one or more audio output devices, cause the electronic device to:

while a first pose of the user meets first presentation criteria:

provide audio content at a first simulated spatial location relative to the user;

detect a change in the pose of the user from the first pose to a second pose; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

provide audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location, including moving a simulated spatial location of the audio content relative to the first physical environment.

2. The non-transitory computer readable storage medium of claim 1 , wherein the second simulated spatial location is above the user.

3. The non-transitory computer readable storage medium of claim 1 , wherein the first pose of the user is one of a lying down pose, a sitting pose, and a standing pose, and the second pose of the user is a different one of a lying down pose, a sitting pose, and a standing pose.

4. The non-transitory computer readable storage medium of claim 1 , wherein detecting the change in the pose of the user includes detecting a change in position of the user from the first physical environment to a second physical environment that is different from the first physical environment.

5. The non-transitory computer readable storage medium of claim 1 , wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to detect, via one or more microphones, sound in the first physical environment, wherein providing the audio content at the second simulated spatial location includes reducing an output level of the audio content relative to the detected sound in the first physical environment.

6. The non-transitory computer readable storage medium of claim 1 , wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to:

determine an event associated with the change in the pose of the user from the first pose to the second pose; and

in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

perform an operation to assist the user to respond to the event.

7. The non-transitory computer readable storage medium of claim 1 , wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to:

detect a change in the pose of the user from the second pose to a third pose; and

in response to detecting the change in the pose of the user from the second pose to the third pose, and in accordance with a determination that the third pose of the user meets the first presentation criteria:

provide audio content at the first simulated spatial location relative to the user.

8. The non-transitory computer readable storage medium of claim 1 , wherein the electronic device includes a display generation component, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to:

while the first pose of the user meets the first presentation criteria:

display, via the display generation component, video content corresponding to the audio content; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

cease to display at least a portion of the video content.

9. The non-transitory computer readable storage medium of claim 1 , wherein the electronic device includes a display generation component, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to:

while the first pose of the user meets the first presentation criteria:

display, via the display generation component, a simulated three-dimensional environment that includes one or more virtual objects; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

cease to display at least a portion of one or more virtual objects in the simulated three-dimensional environment.

10. The non-transitory computer readable storage medium of claim 9 , wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

display, via the display generation component, a representation of at least a portion of the first physical environment.

11. The non-transitory computer readable storage medium of claim 9 , wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

change respective positions of one or more virtual objects in the simulated three-dimensional environment.

12. An electronic device, comprising:

one or more pose sensors for detecting a pose of a user of the electronic device relative to a first physical environment, wherein the electronic device is in communication with one or more audio output devices;

one or more processors; and

memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:

while a first pose of the user meets first presentation criteria:

providing audio content at a first simulated spatial location relative to the user;

detecting a change in the pose of the user from the first pose to a second pose; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

providing audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location, including moving a simulated spatial location of the audio content relative to the first physical environment.

13. The electronic device of claim 12 , wherein the second simulated spatial location is above the user.

14. The electronic device of claim 12 , wherein the first pose of the user is one of a lying down pose, a sitting pose, and a standing pose, and the second pose of the user is a different one of a lying down pose, a sitting pose, and a standing pose.

15. The electronic device of claim 12 , wherein detecting the change in the pose of the user includes detecting a change in position of the user from the first physical environment to a second physical environment that is different from the first physical environment.

16. The electronic device of claim 12 , wherein the one or more programs include instructions for detecting, via one or more microphones, sound in the first physical environment, wherein providing the audio content at the second simulated spatial location includes reducing an output level of the audio content relative to the detected sound in first physical environment.

17. The electronic device of claim 12 , wherein the one or more programs include instructions for:

determining an event associated with the change in the pose of the user from the first pose to the second pose; and

in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

performing an operation to assist the user to respond to the event.

18. The electronic device of claim 12 , wherein the one or more programs include instructions for:

detecting a change in the pose of the user from the second pose to a third pose; and

in response to detecting the change in the pose of the user from the second pose to the third pose, and in accordance with a determination that the third pose of the user meets the first presentation criteria:

providing audio content at the first simulated spatial location relative to the user.

19. The electronic device of claim 12 , wherein the electronic device includes a display generation component, and the one or more programs include instructions for:

while the first pose of the user meets the first presentation criteria:

displaying, via the display generation component, video content corresponding to the audio content; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

ceasing to display at least a portion of the video content.

20. The electronic device of claim 12 , wherein the electronic device includes a display generation component, and the one or more programs include instructions for:

while the first pose of the user meets the first presentation criteria:

displaying, via the display generation component, a simulated three-dimensional environment that includes one or more virtual objects; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

ceasing to display at least a portion of one or more virtual objects in the simulated three-dimensional environment.

21. The electronic device of claim 20 , wherein the one or more programs include instructions for, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

displaying, via the display generation component, a representation of at least a portion of the first physical environment.

22. The electronic device of claim 20 , wherein the one or more programs include instructions for, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

changing respective positions of one or more virtual objects in the simulated three-dimensional environment.

23. A method, comprising:

at an electronic device including one or more pose sensors for detecting a pose of a user of the electronic device relative to a first physical environment, wherein the electronic device is in communication with one or more audio output devices:

while a first pose of the user meets first presentation criteria:

providing audio content at a first simulated spatial location relative to the user;

detecting a change in the pose of the user from the first pose to a second pose; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

providing audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location, including moving a simulated spatial location of the audio content relative to the first physical environment.

24. The method of claim 23 , wherein the second simulated spatial location is above the user.

25. The method of claim 23 , wherein the first pose of the user is one of a lying down pose, a sitting pose, and a standing pose, and the second pose of the user is a different one of a lying down pose, a sitting pose, and a standing pose.

26. The method of claim 23 , wherein detecting the change in the pose of the user includes detecting a change in position of the user from the first physical environment to a second physical environment that is different from the first physical environment.

27. The method of claim 23 , wherein the method includes detecting, via one or more microphones, sound in the first physical environment, wherein providing the audio content at the second simulated spatial location includes reducing an output level of the audio content relative to the detected sound in the first physical environment.

28. The method of claim 23 , wherein the method includes:

determining an event associated with the change in the pose of the user from the first pose to the second pose; and

in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

performing an operation to assist the user to respond to the event.

29. The method of claim 23 , wherein the method includes:

detecting a change in the pose of the user from the second pose to a third pose; and

in response to detecting the change in the pose of the user from the second pose to the third pose, and in accordance with a determination that the third pose of the user meets the first presentation criteria:

providing audio content at the first simulated spatial location relative to the user.

30. The method of claim 23 , wherein the electronic device includes a display generation component, and the method includes:

while the first pose of the user meets the first presentation criteria:

displaying, via the display generation component, video content corresponding to the audio content; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

ceasing to display at least a portion of the video content.

31. The method of claim 23 , wherein the electronic device includes a display generation component, and the method includes:

while the first pose of the user meets the first presentation criteria:

displaying, via the display generation component, a simulated three-dimensional environment that includes one or more virtual objects; and

in response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not meet the first presentation criteria:

ceasing to display at least a portion of one or more virtual objects in the simulated three-dimensional environment.

32. The method of claim 31 , wherein the method includes, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

displaying, via the display generation component, a representation of at least a portion of the first physical environment.

33. The method of claim 31 , wherein the method includes, in response to detecting the change in the pose of the user, and in accordance with the determination that the second pose of the user does not meet the first presentation criteria:

changing respective positions of one or more virtual objects in the simulated three-dimensional environment.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2021
From: MARI, PEDRO
To: APPLE INC.
Reel/Frame 057219/0087 →
Continuity (3)
Continuation 16862376 · Apr 29, 2020
Provisional Application 62843891 · May 6, 2019
Related Publication 20210319778A1 · Oct 14, 2021