IP Library Granted Patent US 12,056,293
Granted Patent B2
US 12,056,293 · App. 18/346,722 · Granted Aug 6, 2024

Selecting virtual objects in a three-dimensional space

Inventors: James M. Powderly (Ft. Lauderdale, FL); Savannah Niles (Ft. Lauderdale, FL); Frank Hamilton (Martinsburg, WV); Marshal A. Fontaine (St. Augustine, FL); Rony Abovitz (Weston, FL); Alysha Naples (London, GB)
Assignee: MAGIC LEAP, INC.
G06F3/0346G06F1/163G06F3/011G06F3/012G06F3/013G06F3/016G06F3/017G06F3/04815G06F3/0482G06F3/04883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,056,293
App. No.
18/346,722
Granted
Aug 6, 2024
Kind
B2
Abstract

Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.

Claims (40)

1. A wearable system for selecting a virtual object located in a three-dimensional (3D) space, the wearable system comprising:

a display system configured to present virtual objects in a 3D space;

a non-transitory data store configured to store interactable objects in the 3D space;

a sensor configured to determine a pose of a user; and

a hardware processor programmed to communicate with the display system, the data store, and the sensor to:

determine a location and a pose of a user;

identify, based on the pose of the user, a physical object in a field of view (FOV) of the user, the FOV comprising a portion of an environment of the user that is perceivable at a given time by the user;

access, based at least in part on the location or the identified physical object, information related to the identified physical object;

generate virtual content based on the information; and

display the virtual content such that the information appears to the user to be associated with the physical object.

2. The system of claim 1 , wherein the hardware processor is programmed further to initiate a selection event associated with the virtual content.

3. The system of claim 2 , wherein the hardware processor is configured to initiate the selection event in response to receiving an input from a user input device or detecting a change of the pose of the user.

4. The system of claim 1 , wherein the hardware processor is programmed further to present, in a two-dimensional (2D) interface, the virtual content within a threshold range of the physical object in the 3D space.

5. The system of claim 1 , wherein the hardware processor is programmed further to index the virtual content based on a coordinate associated with a distance from a position of the physical object.

6. The system of claim 5 , wherein the coordinate comprises one or more of: a lateral distance, vertical distance, and depth distance relative to the position of the physical object.

7. The system of claim 5 , wherein the hardware processor is programmed further to determine the position of the physical object based on the pose of the user.

8. The system of claim 1 , wherein the virtual content comprises text, an image, a graphic, or a video.

9. The system of claim 1 , wherein the hardware processor is programmed further to play audio associated with the displayed virtual content.

10. The system of claim 1 , wherein the hardware processor is programmed further to display additional virtual content based at least in part on a user input.

11. The system of claim 1 , wherein the processor is configured to perform at least one of to identify the physical object in the FOV:

determine a path of a gaze of the user based on the pose of the user and select a possible physical object that intersects the path of the gaze as the physical object; or

select a leftmost or a rightmost possible physical object in the user's FOV as the physical object, wherein the leftmost or the rightmost physical object is selected based at least partly on based on a coordinate associated with a position of the physical object.

12. The system of claim 1 , wherein the hardware processor is further configured to present a focus indicator associated with the virtual content.

13. The system of claim 1 , further comprising a geolocation sensor configured to acquire data associated with the location of the user, and wherein the virtual content comprises a weather application which is programmed to:

determine the location of the user based on the data acquired by the geolocation sensor;

communicate with a remote computing device to acquire weather data based on the location of the user; and

superimpose the virtual content in the 3D space of the user.

14. A method of selecting a virtual object located in a three-dimensional (3D) space, the method comprising:

under control of a hardware processor programmed to communicate with a display system, a data store, and a sensor configured to determine a pose of a user:

determining a location and a pose of a user;

identifying, based on the pose of the user, a physical object in a field of view (FOV) of the user, the FOV comprising a portion of an environment of the user that is perceived at a given time by the user;

accessing, based at least in part on the location or the identified physical object, information related to the identified physical object;

generating virtual content based on the information; and

displaying the virtual content such that the information appears to the user to be associated with the physical object.

15. The method of claim 14 , further comprising initiating a selection event associated with the virtual content.

16. The method of claim 15 , wherein initiating the selection event is in response to receiving an input from a user input device or detecting a change of the pose of the user.

17. The method of claim 14 , further comprising presenting, in a two-dimensional (2D) interface, the virtual content within a threshold range of the physical object in the 3D space.

18. The method of claim 14 , further comprising determining a position of the physical object based on the pose of the user.

19. The method of claim 14 , wherein the virtual content comprises text, an image, a graphic, or a video.

20. The method of claim 14 , further comprising playing audio associated with the displayed virtual content.

Assignments (3)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073387/0487 →
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0238 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2023
From: POWDERLY, JAMES M.; NILES, SAVANNAH; HAMILTON, FRANK; FONTAINE, MARSHAL A.; ABOVITZ, RONY; NAPLES, ALYSHA
To: MAGIC LEAP, INC.
Reel/Frame 065273/0762 →