IP Library Granted Patent US 11,704,874
Granted Patent B2
US 11,704,874 · App. 16/987,144 · Granted Jul 18, 2023

Spatial instructions and guides in mixed reality

Inventors: Tushar Arora (Hollywood, FL); Scott Kramarich (Venice, CA)
Assignee: Magic Leap, Inc.
G06T19/006G06F3/011H04W4/024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,704,874
App. No.
16/987,144
Granted
Jul 18, 2023
Kind
B2
Abstract

Exemplary systems and methods for creating spatial contents in a mixed reality environment are disclosed. In an example, a location associated with a first user in a coordinate space is determined. A persistent virtual content is generated. The persistent virtual content is associated with the first user's associated location. The first user's associated location is determined and is associated with the persistent virtual content. A location of a second user at a second time in the coordinate space is determined. The persistent virtual content is presented to the second user via a display at a location in the coordinate space corresponding to the first user's associated location.

Claims (71)

1. A method comprising:

receiving, in a recording session at a first time, a first input from a first user, the first input indicative of a locomotion of the first user during the recording session;

determining, using a sensor of a first wearable head device, a location associated with the first user at the first time in a coordinate space of a virtual environment;

generating a persistent virtual content corresponding to the first input, the persistent virtual content comprising video of the locomotion of the first user, wherein the persistent virtual content is associated with the location associated with the first user at the first time;

determining, at a second time, a first location of a second user at the second time in the coordinate space;

determining, based on the location associated with the first user at the first time and the first location of the second user at the second time, whether to present the persistent virtual content to the second user;

in accordance with a determination to present the persistent virtual content to the second user, presenting, via a display, the persistent virtual content to the second user;

in accordance with a determination to not present the persistent virtual content to the second user, forgoing presenting the persistent virtual content to the second user,

wherein the presenting the persistent virtual content to the second user comprises presenting, in a playback session later than the first time, the video of the locomotion of the first user at a location in the coordinate space corresponding to the location associated with the first user at the first time;

determining, at a third time, whether a second location of the second user at the third time in the coordinate space is within a threshold distance from the first location of the second user;

in accordance with a determination that the second location of the second user is not within the threshold distance, pausing the presenting the persistent virtual content; and

in accordance with a determination that the second location of the second user is within the threshold distance, forgoing pausing the presenting the persistent virtual content.

2. The method of claim 1 , wherein the first input comprises at least one of a head movement of the first user, a hand movement of the first user, and a handheld controller input.

3. The method of claim 1 , wherein the sensor comprises at least one of a position sensor and a camera.

4. The method of claim 1 , wherein the virtual content comprises a virtual marking.

5. The method of claim 1 , wherein determining whether to present the persistent virtual content to the second user comprises determining, using a sensor of a second wearable head device, whether a field of view of the second user includes the location associated with the first user at the first time.

6. The method of claim 1 , further comprising receiving a filter selection from the second user, wherein determining whether to present the persistent virtual content to the second user comprises determining whether the persistent virtual content is associated with the filter selection.

7. The method of claim 1 , further comprising:

determining a new location, wherein the new location is based on the location associated with the first user at the first time and based further on the first location of the second user at the second time; and

associating the new location with the persistent virtual content.

8. The method of claim 7 , further comprising:

determining, at a fourth time, a location of a third user at the fourth time; and

in accordance with a determination that the location of the third user is at the new location, presenting, in a second playback session, the persistent virtual content to the third user.

9. The method of claim 7 , wherein the new location is further based on an accuracy of the location associated with the first user at the first time and an accuracy of the first location of the second user at the second time.

10. The method of claim 1 , wherein the method further comprises, in accordance with the determination to not present the persistent virtual content to the second user:

determining whether a field of view of the second user includes the location associated with the first user at the first time; and

in accordance with a determination that the field of view of the second user includes the location associated with the first user at the first time, presenting, in the playback session, a second content associated with the persistent virtual content.

11. The method of claim 1 , further comprising:

while the persistent virtual content is presented to the second user in the playback session:

determining, at a fourth time, a location of a third user at the fourth time in the coordinate space; and

in accordance with a determination that the location of the third user is at the first location of the second user at the second time:

receiving an input from the third user;

and

in response to receiving the input from the third user, presenting, in a second playback session, the persistent virtual content to the third user concurrently with presenting, in the playback session, the persistent virtual content to the second user.

12. The method of claim 1 , wherein the location associated with the first user at the first time is a physical location of the first user in the coordinate space.

13. The method of claim 1 , wherein the location associated with the first user at the first time is a virtual location of the first user in the coordinate space, the virtual location different from a physical location of the first wearable head device.

14. A system comprising:

a sensor; and

one or more processors configured to execute a method comprising:

receiving, in a recording session at a first time, a first input from a first user, the first input indicative of a locomotion of the first user during the recording session;

determining, using the sensor, a location associated with the first user at the first time in a coordinate space of a virtual environment;

generating a persistent virtual content corresponding to the first input, the persistent virtual content comprising video of the locomotion of the first user, wherein the persistent virtual content is associated with the location associated with the first user at the first time;

determining, at a second time, a first location of a second user at the second time in the coordinate space;

determining, based on the location associated with the first user at the first time and the first location of the second user at the second time, whether to present the persistent virtual content to the second user;

in accordance with a determination to present the persistent virtual content to the second user, presenting, via a display, the persistent virtual content to the second user;

in accordance with a determination to not present the persistent virtual content to the second user, forgoing presenting the persistent virtual content to the second user,

wherein the presenting the persistent virtual content to the second user comprises presenting, in a playback session later than the first time, the video of the locomotion of the first user at a location in the coordinate space corresponding to the location associated with the first user at the first time;

determining, at a third time, whether a second location of the second user at the third time in the coordinate space is within a threshold distance from the first location of the second user;

in accordance with a determination that the second location of the second user is not within the threshold distance, pausing the presenting the persistent virtual content; and

in accordance with a determination that the second location of the second user is within the threshold distance, forgoing pausing the presenting the persistent virtual content.

15. The system of claim 14 , wherein the method further comprises:

determining a new location, wherein the new location is based on the location associated with the first user at the first time and based further on the first location of the second user at the second time; and

associating the new location with the persistent virtual content.

16. The system of claim 14 , wherein the location associated with the first user at the first time is a physical location of the first user in the coordinate space.

17. The system of claim 14 , wherein the location associated with the first user at the first time is a virtual location of the first user in the coordinate space, the virtual location different from a physical location of the first wearable head device.

18. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute a method comprising:

receiving, in a recording session at a first time, a first input from a first user, the first input indicative of a locomotion of the first user during the recording session;

determining, using a sensor of a first wearable head device, a location associated with the first user at the first time in a coordinate space of a virtual environment;

generating a persistent virtual content corresponding to the first input, the persistent virtual content comprising video of the locomotion of the first user, wherein the persistent virtual content is associated with the location associated with the first user at the first time;

determining, at a second time, a first location of a second user at the second time in the coordinate space;

determining, based on the location associated with the first user at the first time and the first location of the second user at the second time, whether to present the persistent virtual content to the second user;

in accordance with a determination to present the persistent virtual content to the second user, presenting, via a display, the persistent virtual content to the second user;

in accordance with a determination to not present the persistent virtual content to the second user, forgoing presenting the persistent virtual content to the second user,

wherein the presenting the persistent virtual content to the second user comprises presenting, in a playback session later than the first time, the video of the locomotion of the first user at a location in the coordinate space corresponding to the location associated with the first user at the first time;

determining, at a third time, whether a second location of the second user at the third time in the coordinate space is within a threshold distance from the first location of the second user;

in accordance with a determination that the second location of the second user is not within the threshold distance, pausing the presenting the persistent virtual content; and

in accordance with a determination that the second location of the second user is within the threshold distance, forgoing pausing the presenting the persistent virtual content.

19. The non-transitory computer-readable medium of claim 18 , wherein the method further comprises:

determining a new location, wherein the new location is based on the location associated with the first user at the first time and based further on the first location of the second user at the second time; and

associating the new location with the persistent virtual content.

20. The non-transitory computer-readable medium of claim 18 , wherein the location associated with the first user at the first time is a physical location of the first user in the coordinate space.

Assignments (3)
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073008/0696 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2021
From: ARORA, TUSHAR; KRAMARICH, SCOTT
To: MAGIC LEAP, INC.
Reel/Frame 055895/0966 →
Continuity (3)
Provisional Application 62894448 · Aug 30, 2019
Provisional Application 62884153 · Aug 7, 2019
Related Publication 20210043005A1 · Feb 11, 2021
Cited By (1)
US 12,430,860