IP Library › Granted Patent US 11,842,449

Granted Patent B2

US 11,842,449 · App. 17/002,225 · Granted Dec 12, 2023

Presenting an environment based on user movement

Inventors: Alexis Palangie (Palo Alto, CA); Aaron Mackay Burns (Sunnyvale, CA)

Assignee: Apple Inc.

G06T19/006 G06F3/011 G06F3/017

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,842,449
App. No.: 17/002,225
Granted: Dec 12, 2023
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

In an exemplary process, a computer-generated reality environment comprising a virtual object is presented and user movement that occurs in a physical environment is detected. In response to determining that the detected user movement is toward the virtual object and that the virtual object obstructs a real object from the physical environment, a determination is made whether the detected user movement is directed to the virtual object or the real object. In accordance with a determination that the detected user movement is directed to the real object, a visual appearance of the virtual object is modified, where modifying the visual appearance of the virtual object comprises displaying presenting at least a portion of the real object. In accordance with a determination that the detected user movement is directed to the virtual object, the presentation of the virtual object is maintained to obstruct the real object.

Claims (64)

1. An electronic device, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

presenting, via a display device, a computer-generated reality environment comprising a virtual object;

detecting, via one or more sensors, user movement that occurs in a physical environment;

determining that the detected user movement is toward the virtual object;

determining that the virtual object obstructs a real object from the physical environment; and

in response to determining that the detected user movement is toward the virtual object and determining that the virtual object obstructs a real object from the physical environment:

determining whether the detected user movement is directed to the virtual object or the real object;

in accordance with a determination that the detected user movement is directed to the real object, wherein a distance between the user and the real object is a non-zero distance during the detected user movement, modifying a visual appearance of the virtual object, wherein modifying the visual appearance of the virtual object comprises presenting at least a portion of the real object; and

in accordance with a determination that the detected user movement is directed to the virtual object, wherein a distance between the user and the virtual object is a non-zero distance during the detected user movement, maintaining the presentation of the virtual object to obstruct the real object.

2. The electronic device of claim 1 , wherein determining that the detected user movement is toward the virtual object includes determining that a distance between the virtual object and a location of the user movement does not exceed a threshold distance.

3. The electronic device of claim 1 , wherein the one or more programs further include instructions for:

detecting, via the one or more sensors, a user pose that occurs in the physical environment, wherein the determination that the detected user movement is directed to the real object includes a determination that the detected user pose corresponds to a feature of the real object.

4. The electronic device of claim 1 , wherein the one or more programs further include instructions for:

5. The electronic device of claim 1 , wherein the determination that the detected user movement is directed to the real object includes a determination that a speed associated with the detected user movement exceeds a threshold speed.

6. The electronic device of claim 1 , wherein the one or more programs further include instructions for:

detecting, via the one or more sensors, a user gaze, wherein the determination that the detected user movement is directed to the real object includes a determination that the detected user gaze is directed to the real object.

7. The electronic device of claim 1 , wherein modifying the visual appearance of the virtual object includes:

in accordance with a determination that the detected user movement is directed to the real object with a first level of confidence, modifying the visual appearance of the virtual object by a first magnitude; and

in accordance with a determination that the detected user movement is directed to the real object with a second level of confidence different from the first level of confidence, modifying the visual appearance of the virtual object by a second magnitude different from the first magnitude.

8. The electronic device of claim 1 , wherein determining that the virtual object obstructs the real object from the physical environment includes determining that the virtual object at least partially overlaps the real object in the computer-generated reality environment.

9. The electronic device of claim 1 , wherein determining that the virtual object obstructs the real object from the physical environment includes determining that the virtual object at least partially blocks a view of the real object from a user perspective of the computer-generated reality environment.

10. The electronic device of claim 1 , wherein determining whether the detected user movement is directed to the virtual object or the real object includes predicting where the detected user movement will stop.

11. The electronic device of claim 1 , wherein modifying the visual appearance of the virtual object includes ceasing to present at least a portion of the virtual object.

12. The electronic device of claim 1 , wherein determining whether the detected user movement is directed to the virtual object or the real object is performed using a machine learning algorithm.

13. The electronic device of claim 1 , wherein presenting the computer-generated reality environment includes presenting a second real object from the physical environment concurrently with the virtual object, and wherein at least a portion of the second real object is not obstructed by the virtual object.

14. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:

presenting, via a display device, a computer-generated reality environment comprising a virtual object;

detecting, via one or more sensors, user movement that occurs in a physical environment;

determining that the detected user movement is toward the virtual object;

determining that the virtual object obstructs a real object from the physical environment; and

in response to determining that the detected user movement is toward the virtual object and determining that the virtual object obstructs a real object from the physical environment:

determining whether the detected user movement is directed to the virtual object or the real object;

15. The non-transitory computer-readable storage medium of claim 14 , wherein determining that the detected user movement is toward the virtual object includes determining that a distance between the virtual object and a location of the user movement does not exceed a threshold distance.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:

17. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:

18. The non-transitory computer-readable storage medium of claim 14 , wherein the determination that the detected user movement is directed to the real object includes a determination that a speed associated with the detected user movement exceeds a threshold speed.

19. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more programs further include instructions for:

20. The non-transitory computer-readable storage medium of claim 14 , wherein presenting the computer-generated reality environment includes presenting a second real object from the physical environment concurrently with the virtual object, and wherein at least a portion of the second real object is not obstructed by the virtual object.

21. A method, comprising:

presenting, via a display device, a computer-generated reality environment comprising a virtual object;

detecting, via one or more sensors, user movement that occurs in a physical environment;

determining that the detected user movement is toward the virtual object;

determining that the virtual object obstructs a real object from the physical environment; and

in response to determining that the detected user movement is toward the virtual object and determining that the virtual object obstructs a real object from the physical environment:

determining whether the detected user movement is directed to the virtual object or the real object;

22. The method of claim 21 , wherein determining that the detected user movement is toward the virtual object includes determining that a distance between the virtual object and a location of the user movement does not exceed a threshold distance.

23. The method of claim 21 , further comprising:

24. The method of claim 21 , further comprising:

25. The method of claim 21 , wherein the determination that the detected user movement is directed to the real object includes a determination that a speed associated with the detected user movement exceeds a threshold speed.

26. The method of claim 21 , further comprising:

27. The method of claim 21 , wherein presenting the computer-generated reality environment includes presenting a second real object from the physical environment concurrently with the virtual object, and wherein at least a portion of the second real object is not obstructed by the virtual object.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2020

From: PALANGIE, ALEXIS; BURNS, AARON MACKAY

To: APPLE INC.

Reel/Frame 053628/0397 →

Continuity (2)

Provisional Application 62906667 · Sep 26, 2019

Related Publication 20210097766A1 · Apr 1, 2021