IP Library Granted Patent US 11,650,709
Granted Patent B2
US 11,650,709 · App. 17/185,362 · Granted May 16, 2023

3D models for displayed 2D elements

Inventors: Christopher Richard Williams (Miami, FL); Damian Franco (Miami, FL)
Assignee: Magic Leap, Inc.
G06F3/04815G06F3/04812G06F3/04842G06F3/167G06F9/451G06T17/00G06T19/006G06T19/20G06T2200/24G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,650,709
App. No.
17/185,362
Granted
May 16, 2023
Kind
B2
Abstract

A system includes: a screen configured for wear by a user, the screen configured to display a 2-dimensional (2D) element; a processing unit coupled to the display; and a user input device configured to generate a signal in response to a user input for selecting the 2D element displayed by the screen; wherein the processing unit is configured to obtain a 3-dimensional (3D) model associated with the 2D element in response to the generated signal.

Claims (45)

1. A system, comprising:

a screen configured for wear by a user, the screen configured to display a 2-dimensional (2D) element;

a processing unit coupled to the display; and

a user input device configured to generate a signal in response to a user input selecting, or indicating an interest in, the 2D element displayed by the screen;

wherein the processing unit is configured to obtain model information of a 3-dimensional (3D) model associated with the 2D element from a storage device;

wherein the 2D element comprises a graphical object, and the 3D model is a three-dimensional version of the graphical object; and

wherein the system is configured to provide an indication informing the user that the 3D model associated with the 2D element is available for access, wherein the system is configured to provide the indication in association with the 2D element before the 3D model is provided for manipulation by the user; and

wherein the indication comprises a changing graphic displayed in association with the 2D element, the changing graphic indicating that the 3D model is accessible, and wherein the system is configured to provide the changing graphics based on the signal.

2. The system of claim 1 , wherein the user input device is configured to generate the signal in response to a double-clicking action, a press-and-hold action, a long hold action, a pressing of a button while a cursor is pointing at the 2D element, a hovering of the cursor over the 2D element for a threshold period of time, or an audio input.

3. The system of claim 1 , wherein the 3D model is a 3D rendering or 3D depiction of the 2D element.

4. The system of claim 1 , wherein the changing graphic comprises a change in a color of the 2D element.

5. The system of claim 1 , wherein the changing graphic is displayed next to the 2D element.

6. The system of claim 1 , wherein the changing graphic comprises a change in a configuration of a cursor.

7. The system of claim 1 , wherein the changing graphic comprises a display of the 3D model beginning to grow out of the 2D element.

8. The system of claim 1 , wherein the 2D element is presented within a browser or a viewer.

9. The system of claim 1 , wherein the 3D model is stored in association with a page descriptor information.

10. The system of claim 1 , wherein the 2D element is a part of a page, and wherein metadata for the page indicates a location from which the 3D model is to be retrieved.

11. The system of claim 1 , wherein the 2D element is part of a page, and wherein markup language of the page comprises metadata that designates the 2D element as being associated with the 3D model.

12. The system of claim 1 , wherein the 3D model is dynamically generated.

13. The system of claim 1 , wherein the 2D element is a part of a page presented in a browser or viewer, and wherein the 3D model is loaded and/or made available when or after the page is loaded in the browser or the viewer.

14. The system of claim 1 , wherein the processing unit is configured to retrieve the 3D model through a request made to an application programming interface (API).

15. The system of claim 1 , wherein the processing unit is configured to obtain the 3D model associated with the 2D element by receiving the 3D model transmitted via a network.

16. The system of claim 1 , further comprising a memory storing the 3D model, and wherein the processing unit is configured to obtain the 3D model by retrieving the 3D model from the memory.

17. The system of claim 1 , wherein the 3D model is for addition to an environment that is presented by the system.

18. The system of claim 1 , wherein the processing unit is configured to receive a command for placing the 3D model at a certain location with respect to an environment as viewed through the screen.

19. The system of claim 18 , wherein the command comprises a cursor position generated based on a movement of a cursor to place the 3D model in association with a virtual object or a real-world object with respect to the environment.

20. The system of claim 1 , wherein the screen is at least partially transparent for allowing the user to view a surrounding environment through the screen.

21. The system of claim 1 , wherein the system is configured to provide a MR, AR, or VR environment to the user.

22. A method, comprising:

displaying, by a screen that is configured for wear by a user, a 2-dimensional (2D) element;

receiving a signal generated by a user input device in response to a user input selecting, or indicating an interest in, the 2D element displayed by the screen; and

obtaining model information of a 3-dimensional (3D) model associated with the 2D element from a storage device;

wherein the 2D element comprises a graphical object, and the 3D model is a three-dimensional version of the graphical object;

wherein the method further comprises providing an indication informing the user that the 3D model associated with the 2D element is available for access, wherein the indication is provided in association with the 2D element before the 3D model is provided for manipulation by the user; and

wherein the indication comprises a changing graphic displayed in association with the 2D element, the changing graphic indicating that the 3D model is accessible, and wherein the changing graphics is provided based on the signal.

23. A product comprising a non-transitory medium storing a set of instructions, an execution of which by a processing unit will cause a method to be performed, the method comprising:

displaying, by a screen that is configured for wear by a user, a 2-dimensional (2D) element;

receiving a signal generated by a user input device in response to a user input selecting, or indicating an interest in, the 2D element displayed by the screen; and

obtaining model information of a 3-dimensional (3D) model associated with the 2D element from a storage device;

wherein the 2D element comprises a graphical object, and the 3D model is a three-dimensional version of the graphical object;

wherein the method further comprises providing an indication that the 3D model associated with the 2D element is available for access, wherein the indication is provided in association with the 2D element before the 3D model is provided for manipulation by the user; and

wherein the indication comprises a changing graphic displayed in association with the 2D element, the changing graphics indicating that the 3D model is accessible, and wherein the changing graphic is provided based on the signal.

24. The system of claim 1 , wherein the indication also comprises an audio signal.

25. The system of claim 1 , wherein the changing graphic is different from the 2D element.

26. The system of claim 1 , wherein the changing graphic comprises a change in the 2D element.

Assignments (4)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0238 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 17, 2021
From: WILLIAMS, CHRISTOPHER RICHARD; FRANCO, DAMIAN
To: MAGIC LEAP, INC.
Reel/Frame 057520/0735 →
Continuity (2)
Provisional Application 62983196 · Feb 28, 2020
Related Publication 20210271370A1 · Sep 2, 2021