Granted Patent B2

US 12,198,280 · App. 18/068,711 · Granted Jan 14, 2025

Inventors: Timothy R. Pease (Emerald Hills, CA); Alexandre Da Veiga (San Francisco, CA); David H. Huang (San Mateo, CA); Peng Liu (San Jose, CA); Robert K. Molholm (Scotts Valley, CA)

Assignee: Apple Inc.

G06T19/006G06F3/013G06F3/14G06F3/165G06V40/20

Abstract

Claims (118)

1. An electronic device, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

obtaining a set of parameters corresponding to characteristics of a physical setting of a user;

determining, based on the set of parameters, at least one display placement value and a fixed boundary location corresponding to the physical setting, wherein the display placement value includes a virtual display size; and

in accordance with a determination that the at least one display placement value satisfies a display placement criterion including a threshold virtual display size, wherein the display placement criterion is satisfied when the virtual display size satisfies the threshold virtual display size:

displaying a virtual display at the fixed boundary location corresponding to the physical setting.

2. The electronic device of claim 1 , the one or more programs further including instructions for:

in accordance with a determination that the at least one display placement value does not satisfy the display placement criterion:

displaying the virtual display at a fixed location corresponding to a predefined distance in front of the user in the physical setting.

3. The electronic device of claim 1 , the one or more programs further including instructions for:

detecting a user input corresponding to an intent to adjust the virtual display;

in accordance with a determination that the intent to adjust the virtual display includes an intent to move the virtual display, displaying movement of the virtual display to an alternative location; and

in accordance with a determination that the intent to adjust the virtual display includes an intent to modify a size of the virtual display, displaying a size modification of the virtual display.

4. The electronic device of claim 1 , the one or more programs further including instructions for:

in accordance with detecting a virtual space preference:

identifying at least one boundary corresponding to the physical setting;

displaying a virtual room;

wherein the virtual room at least partially replaces a representation of the at least one boundary; and

wherein the virtual display is displayed within the virtual room.

5. The electronic device of claim 1 , the one or more programs further including instructions for:

in accordance with detecting at least one light parameter exceeding a threshold:

displaying a virtual sky; and

wherein the virtual sky at least partially replaces a representation of at least one ceiling boundary corresponding to the physical setting.

6. The electronic device of claim 1 , the one or more programs further including instructions for:

displaying media on the virtual display;

identifying an attribute associated with the displayed media;

identifying at least one boundary corresponding to the physical setting; and

displaying a virtual room based on the identified attribute, wherein the virtual room at least partially replaces a representation of the at least one boundary.

7. The electronic device of claim 1 , the one or more programs further including instructions for:

detecting a user head movement; and

in accordance with a determination that the user head movement corresponds to movement away from the virtual display, decreasing a curvature of the virtual display.

8. The electronic device of claim 7 , the one or more programs further including instructions for:

in accordance with a determination that the user head movement corresponds to movement towards the virtual display, increasing a curvature of the virtual display.

9. The electronic device of claim 1 , the one or more programs further including instructions for:

identifying audio associated with media displayed on the virtual display;

receiving, from a media delivery model, a result based on the set of parameters;

in accordance with a determination that the result includes an audio rendering setting:

obtaining, from the result, audio output parameters based on the characteristics of the physical setting; and

outputting the audio consistent with the audio output parameters; and

in accordance with a determination that the result does not include an audio rendering setting;

obtaining default audio output parameters; and

outputting the audio consistent with the default audio output parameters.

10. The electronic device of claim 1 , the one or more programs further including instructions for:

in accordance with detecting an amount of movement of a user gaze towards the virtual display:

replacing at least a portion of a representation of the physical setting with a portion of a virtual setting.

11. The electronic device of claim 1 , the one or more programs further including instructions for:

while detecting movement of the user gaze towards the virtual display:

gradually replacing, in proportion to the detected movement of the user gaze towards the virtual display, a representation of the physical setting with a virtual setting; and

while detecting the movement of the user gaze away from the virtual display:

gradually replacing, in proportion to the detected movement of the user gaze away from the virtual display, a virtual setting with the representation of physical setting.

12. The electronic device of claim 1 , the one or more programs further including instructions for:

providing the set of parameters to a media delivery model; and

receiving, from the media delivery model, a result based on the provided set of parameters, wherein the result includes the at least one display placement value and the fixed boundary location corresponding to the physical setting.

13. The electronic device of claim 1 , the one or more programs further including instructions for:

providing a first audio output including audio associated with the physical setting;

while initiating display of the virtual display, providing a second audio output including audio associated with media playing on the virtual display; and

while displaying the virtual display at the fixed boundary location, continuing to provide the first audio output and the second audio output.

14. The electronic device of claim 13 , the one or more programs further including instructions for:

while initiating display of the virtual display:

lowering a first volume level corresponding to the first audio output, wherein the first volume level is lowered to a first predefined volume level; and

raising a second volume level corresponding to the second audio output, wherein the second volume level is raised to a second predefined volume level.

15. The electronic device of claim 14 , the one or more programs further including instructions for:

while lowering the first volume level corresponding to the first audio output, applying a third audio output to the first audio output, wherein the third audio output corresponds to a noise control signal.

16. The electronic device of claim 1 , the one or more programs further including instructions for:

providing a first audio output including audio associated with the physical setting;

providing a second audio output including audio associated with media playing on the virtual display;

in accordance with detecting an amount of movement of a user gaze away from the virtual display:

adjusting, based on the amount of movement of the user gaze away from the virtual display, a first volume level of the first audio output and a second volume level of the second audio output; and

in accordance with detecting an amount of movement of a user gaze towards the virtual display:

adjusting, based on the amount of movement of the user gaze towards from the virtual display, the first volume level of the first audio output and the second volume level of the second audio output.

17. A method, comprising:

at an electronic device:

obtaining a set of parameters corresponding to characteristics of a physical setting of a user;

determining, based on the set of parameters, at least one display placement value and a fixed boundary location corresponding to the physical setting, wherein the display placement value includes a virtual display size; and

in accordance with a determination that the at least one display placement value satisfies a display placement criterion including a threshold virtual display size, wherein the display placement criterion is satisfied when the virtual display size satisfies the threshold virtual display size:

displaying a virtual display at the fixed boundary location corresponding to the physical setting.

18. The method of claim 17 , comprising:

in accordance with a determination that the at least one display placement value does not satisfy the display placement criterion:

displaying the virtual display at a fixed location corresponding to a predefined distance in front of the user in the physical setting.

19. The method of claim 17 , comprising:

detecting a user input corresponding to an intent to adjust the virtual display;

in accordance with a determination that the intent to adjust the virtual display includes an intent to move the virtual display, displaying movement of the virtual display to an alternative location; and

in accordance with a determination that the intent to adjust the virtual display includes an intent to modify a size of the virtual display, displaying a size modification of the virtual display.

20. The method of claim 17 , comprising:

providing a first audio output including audio associated with the physical setting;

while initiating display of the virtual display, providing a second audio output including audio associated with media playing on the virtual display; and

while displaying the virtual display at the fixed boundary location, continuing to provide the first audio output and the second audio output.

21. The method of claim 20 , comprising:

while initiating display of the virtual display:

lowering a first volume level corresponding to the first audio output, wherein the first volume level is lowered to a first predefined volume level; and

raising a second volume level corresponding to the second audio output, wherein the second volume level is raised to a second predefined volume level.

22. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

obtaining a set of parameters corresponding to characteristics of a physical setting of a user;

determining, based on the set of parameters, at least one display placement value and a fixed boundary location corresponding to the physical setting, wherein the display placement value includes a virtual display size; and

in accordance with a determination that the at least one display placement value satisfies a display placement criterion including a threshold virtual display size, wherein the display placement criterion is satisfied when the virtual display size satisfies the threshold virtual display size:

displaying a virtual display at the fixed boundary location corresponding to the physical setting.

23. The computer readable medium of claim 22 , the one or more programs further including instructions for:

in accordance with a determination that the at least one display placement value does not satisfy the display placement criterion:

displaying the virtual display at a fixed location corresponding to a predefined distance in front of the user in the physical setting.

24. The computer readable medium of claim 22 , the one or more programs further including instructions for:

detecting a user input corresponding to an intent to adjust the virtual display;

in accordance with a determination that the intent to adjust the virtual display includes an intent to move the virtual display, displaying movement of the virtual display to an alternative location; and

in accordance with a determination that the intent to adjust the virtual display includes an intent to modify a size of the virtual display, displaying a size modification of the virtual display.

25. The computer readable medium of claim 22 , the one or more programs further including instructions for:

in accordance with detecting a virtual space preference:

identifying at least one boundary corresponding to the physical setting;

displaying a virtual room;

wherein the virtual room at least partially replaces a representation of the at least one boundary; and

wherein the virtual display is displayed within the virtual room.

26. The computer readable medium of claim 22 , the one or more programs further including instructions for:

providing a first audio output including audio associated with the physical setting;

while initiating display of the virtual display, providing a second audio output including audio associated with media playing on the virtual display; and

while displaying the virtual display at the fixed boundary location, continuing to provide the first audio output and the second audio output.

27. The computer readable medium of claim 22 , the one or more programs further including instructions for:

while initiating display of the virtual display:

lowering a first volume level corresponding to the first audio output, wherein the first volume level is lowered to a first predefined volume level; and

raising a second volume level corresponding to the second audio output, wherein the second volume level is raised to a second predefined volume level.

Continuity (4)

References Cited (178)