IP Library Granted Patent US 12,079,938
Granted Patent B2
US 12,079,938 · App. 17/723,245 · Granted Sep 3, 2024

Dynamic colocation of virtual content

Inventors: Paul Andrew Baier (Bundoora, AU); Domingo Noriega-Padilla (Miami, FL)
Assignee: Magic Leap, Inc.
G06T19/006G02B27/017G06F17/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,079,938
App. No.
17/723,245
Granted
Sep 3, 2024
Kind
B2
Abstract

Disclosed herein are systems and methods for colocating virtual content. A method may include receiving first persistent coordinate data, second persistent coordinate data, and relational data. A third persistent coordinate data and a fourth persistent coordinate data may be determined based on input received via one or more sensors of a head-wearable device. It can be determined whether the first persistent coordinate data corresponds to the third persistent coordinate data. In accordance with a determination that the first persistent coordinate data corresponds to the third persistent coordinate data, it can be determined whether the second persistent coordinate data corresponds to the fourth persistent coordinate data. In accordance with a determination that the second persistent coordinate data corresponds to the fourth persistent coordinate data, a virtual object can be displayed using the relational data and the second persistent coordinate data via a display of the head-wearable device. In accordance with a determination that the second persistent coordinate data does not correspond to the fourth persistent coordinate data, the virtual object can be displayed using the relational data and the first persistent coordinate data via the display of the head-wearable device. In accordance with a determination that the first persistent coordinate data does not correspond to the third persistent coordinate data, the method may forgo displaying the virtual object via the display of the head-wearable device.

Claims (49)

1. A system comprising:

a wearable head device comprising a display and one or more sensors; and

one or more processors configured to perform a method comprising:

receiving first persistent coordinate data, wherein the first persistent coordinate data comprises a transformation matrix relating a virtual coordinate space of a physical environment to a real coordinate space of the physical environment;

receiving relational data, wherein the relational data relates a position to the first persistent coordinate data;

determining, based on input received via the one or more sensors, first location data identifying a first environment of the wearable head device;

determining whether the first persistent coordinate data corresponds to the first environment of the wearable head device;

in accordance with a determination that the first persistent coordinate data corresponds to the first environment of the wearable head device, displaying a virtual object at the position via the display of the wearable head device based on the first persistent coordinate data and based further on the relational data; and

in accordance with a determination that the first persistent coordinate data does not correspond to the first environment of the wearable head device, forgoing displaying the virtual object via the display of the wearable head device,

wherein the first persistent coordinate data is determined based on a combination of two or more sets of second persistent coordinate data.

2. The system of claim 1 , wherein the wearable head device is a first wearable head device, and wherein the method further comprises transmitting the first location data to a second wearable head device.

3. The system of claim 1 , wherein the wearable head device is a first wearable head device, and wherein said receiving the first persistent coordinate data comprises receiving the first persistent coordinate data from a second wearable head device.

4. The system of claim 1 , wherein the wearable head device is a first wearable head device, and wherein said receiving the relational data comprises receiving the relational data from a second wearable head device.

5. The system of claim 1 , the method further comprising receiving data corresponding to the virtual object.

6. The system of claim 1 , wherein the first environment of the wearable head device corresponds to a room.

7. The system of claim 1 , wherein the relational data comprises matrix data.

8. The system of claim 1 , wherein the combination of the two or more sets of second persistent coordinate data comprises a weighted average of the two or more sets of second persistent coordinate data.

9. The system of claim 8 , wherein the weighted average is determined by applying one or more confidence measurements to the two or more sets of second persistent coordinate data.

10. The system of claim 1 , wherein the two or more sets of second persistent coordinate data are determined based on at least one observation of the physical environment by one or more of: the wearable head device and one or more second wearable head devices.

11. The system of claim 1 , wherein said receiving the first persistent coordinate data comprises receiving the first persistent coordinate data from a remote server.

12. The system of claim 11 , wherein the first persistent coordinate data is determined at the remote server.

13. The system of claim 12 , wherein the two or more sets of second persistent coordinate data are transmitted to the remote server by one or more of: the wearable head device and one or more second wearable head devices.

14. A method comprising:

receiving, at a wearable head device, first persistent coordinate data, wherein the first persistent coordinate data comprises a transformation matrix relating a virtual coordinate space of a physical environment to a real coordinate space of the physical environment;

receiving, at the wearable head device, relational data, wherein the relational data relates a position to the first persistent coordinate data;

determining, based on input received via one or more sensors of the wearable head device, first location data identifying a first environment of the wearable head device;

determining whether the first persistent coordinate data corresponds to the first environment of the wearable head device;

in accordance with a determination that the first persistent coordinate data corresponds to the first environment of the wearable head device, displaying a virtual object at the position via a display of the wearable head device based on the first persistent coordinate data and based further on the relational data; and

in accordance with a determination that the first persistent coordinate data does not correspond to the first environment of the wearable head device, forgoing displaying the virtual object via the display of the wearable head device,

wherein the first persistent coordinate data is determined based on a combination of two or more sets of second persistent coordinate data.

15. The method of claim 14 , wherein the wearable head device is a first wearable head device, and wherein the method further comprises transmitting the first location data to a second wearable head device.

16. The method of claim 14 , wherein the wearable head device is a first wearable head device, and wherein said receiving the first persistent coordinate data comprises receiving the first persistent coordinate data from a second wearable head device.

17. The method of claim 14 , wherein the wearable head device is a first wearable head device, and wherein said receiving the relational data comprises receiving the relational data from a second wearable head device.

18. The method of claim 14 , further comprising receiving data corresponding to the virtual object.

19. The method of claim 14 , wherein the first environment of the wearable head device corresponds to a room.

20. The method of claim 14 , wherein the relational data comprises matrix data.

21. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

receiving, at a wearable head device, first persistent coordinate data, wherein the first persistent coordinate data comprises a transformation matrix relating a virtual coordinate space of a physical environment to a real coordinate space of the physical environment;

receiving, at the wearable head device, relational data, wherein the relational data relates a position to the first persistent coordinate data;

determining, based on input received via one or more sensors of the wearable head device, first location data identifying a first environment of the wearable head device;

determining whether the first persistent coordinate data corresponds to the first environment of the wearable head device;

in accordance with a determination that the first persistent coordinate data corresponds to the first environment of the wearable head device, displaying a virtual object at the position via a display of the wearable head device based on the first persistent coordinate data and based further on the relational data; and

in accordance with a determination that the first persistent coordinate data does not correspond to the first environment of the wearable head device, forgoing displaying the virtual object via the display of the wearable head device,

wherein the first persistent coordinate data is determined based on a combination of two or more sets of second persistent coordinate data.

22. The non-transitory computer-readable medium of claim 21 , wherein the wearable head device is a first wearable head device, and wherein the method further comprises transmitting the first location data to a second wearable head device.

23. The non-transitory computer-readable medium of claim 21 , wherein the wearable head device is a first wearable head device, and wherein said receiving the first persistent coordinate data comprises receiving the first persistent coordinate data from a second wearable head device.

24. The non-transitory computer-readable medium of claim 21 , wherein the wearable head device is a first wearable head device, and wherein said receiving the relational data comprises receiving the relational data from a second wearable head device.

25. The non-transitory computer-readable medium of claim 21 , the method further comprising receiving data corresponding to the virtual object.

26. The non-transitory computer-readable medium of claim 21 , wherein the first environment of the wearable head device corresponds to a room.

Assignments (3)
SECURITY INTEREST Recorded Oct 29, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073430/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2023
From: BAIER, PAUL ANDREW; NORIEGA-PADILLA, DOMINGO
To: MAGIC LEAP, INC.
Reel/Frame 064487/0979 →
SECURITY INTEREST Recorded Jul 1, 2022
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060560/0438 →
Continuity (3)
Continuation 17173125 · Feb 10, 2021
Provisional Application 62972541 · Feb 10, 2020
Related Publication 20220245905A1 · Aug 4, 2022
Cited By (3)
US 12,211,145 US 12,315,094 US 12,495,094