IP Library Granted Patent US 12,056,268
Granted Patent B2
US 12,056,268 · App. 17/404,667 · Granted Aug 6, 2024

Platformization of mixed reality objects in virtual reality environments

Inventors: Christopher Richard Tanner (San Carlos, CA); Amir Mesguich Havilio (San Francisco, CA); Michelle Pujals (Los Gatos, CA)
Assignee: META PLATFORMS TECHNOLOGIES, LLC
G06F3/011G06F3/03G06T17/20G06T19/006G06T2210/04G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,056,268
App. No.
17/404,667
Granted
Aug 6, 2024
Kind
B2
Abstract

In one embodiment, a method includes receiving an instruction to transition to a first virtual reality (VR) environment of a first VR application, accessing attribute information of one or more real-world objects in a real-world environment proximate to a VR display device, determining, for each of the one or more real-world objects, one or more first mixed reality (MR) functionalities applicable to the real-world object based on its attribute information, and rendering, for one or more displays of the VR display device, the first VR environment with one or more MR objects corresponding to the one or more real-world objects, wherein each MR object comprises one of the first MR functionalities applicable to the corresponding real-world object.

Claims (68)

1. A method comprising, by one or more computing systems:

receiving an instruction to transition to a virtual reality (VR) environment of a VR application;

accessing attribute information of one or more real-world objects in a real-world environment within a threshold distance of a VR display device;

determining, for each of the one or more real-world objects, one or more mixed reality (MR) functionalities applicable to the real-world object based on its attribute information;

generating one or more first MR objects, corresponding to at least one first real-world object of the one or more real-world objects, and one or more second MR objects, corresponding to at least one second real-world object of the one or more real-world objects;

rendering, for one or more displays of the VR display device, the VR environment with the one or more first MR objects corresponding to the at least one first real-world object of the one or more real-world objects,

wherein each first MR object comprises one of the MR functionalities applicable to the corresponding first real-world object, and

wherein the rendered VR environment omits the one or more second MR objects corresponding to the at least one second real-world object of the one or more real-world objects; and

displaying the at least one second real-world object, via a passthrough view of the real-world environment, while rendering the VR environment.

2. The method of claim 1 , further comprising:

receiving an instruction to transition to a second VR environment of the VR application;

determining, for each of the one or more real world objects, one or more second MR functionalities based on the second VR environment of the first VR application and applicable to the real-world object based on its attribute information; and

rendering, for the one or more displays of the VR display device, the second VR environment with the one or more first MR objects corresponding to the at least one first real-world object, wherein each first MR object comprises one of the second MR functionalities applicable to the corresponding first real-world object.

3. The method of claim 1 , wherein the attribute information of the one or more real-world objects comprises one or more spatial dimensions of the one or more real-world objects.

4. The method of claim 1 , wherein the attribute information of the one or more real-world objects comprises one or more volumetric shapes of the one or more real-world objects.

5. The method of claim 1 , wherein the attribute information of the one or more real-world objects comprises one or more real-world functionalities of the one or more real-world objects.

6. The method of claim 1 , wherein one or more of the MR functionalities applicable to the real-world object corresponds to one or more real-world functionalities of the real-world object.

7. The method of claim 1 , wherein one or more of the MR functionalities applicable to the real-world object do not correspond to any real-world functionality of the real-world object.

8. The method of claim 1 , further comprising:

identifying that the at least one first real-world object in the real-world environment is within the threshold distance of the VR display device.

9. The method of claim 8 , further comprising:

identifying a planar surface of one of the at least one first real-world object in the real-world environment; and

applying one of the MR functionalities to the planar surface of the one of the at least one first real-world object.

10. The method of claim 8 , further comprising:

generating a bounding box containing one of the at least one first real-world object in the real-world environment; and

applying one of the MR functionalities to the bounding box containing the one of the at least one first real-world object.

11. The method of claim 8 , further comprising:

generating a three-dimensional (3D) mesh of one of the at least one first real-world object in the real-world environment; and

applying one of the MR functionalities to the 3D mesh of the one of the at least one first real-world object.

12. The method of claim 1 , further comprising:

rendering, for the one or more displays of the VR display device, one or more suggestions for applying one or more of the MR functionalities to the at least one first real-world object;

receiving an indication of a user selection corresponding to one of the suggestions for applying one of the MR functionalities to the at least one first real-world object; and

applying the one of the MR functionalities to the at least one first real-world object responsive to the user selection.

13. The method of claim 1 , further comprising:

accessing floorplan information of the real-world environment;

rendering, for the one or more displays of the VR display device, the VR environment, wherein the VR environment corresponds to the floorplan information.

14. The method of claim 13 , further comprising:

receiving an indication the VR display device has moved to a new area of the real-world environment corresponding to the floorplan information, the new area of the real-world environment comprising one or more new real-world objects;

accessing attribute information of one or more new real-world objects in the new real-world environment within the threshold distance of the VR display device;

determining, for each of the one or more new real-world objects, one or more new MR functionalities applicable to the new real-world object based on its attribute information; and

rendering, for the one or more displays of the VR display device, a second VR environment with one or more new MR objects corresponding to the one or more new real-world objects, wherein each new MR object comprises one or more determined new MR functionalities applicable to the corresponding new real-world object.

15. The method of claim 1 , further comprising:

rendering, for the one or more displays of the VR display device, an avatar corresponding to a user within the VR environment, wherein rendering the avatar comprises placing the avatar within the VR environment relative to a first MR object, of the at least one first MR object, based on the determined MR functionality of the first MR object.

16. The method of claim 1 , wherein the one or more first MR objects are highlighted, outlined or made apparent based on a proximity of the VR display device to the one or more first MR objects.

17. One or more computer-readable non-transitory storage media embodying software that is operable when executed by a mobile computing device to:

receive an instruction to transition to a virtual reality (VR) environment of a VR application;

access attribute information of one or more real-world objects in a real-world environment within a threshold distance of a VR display device;

determine, for each of the one or more real-world object, one or more mixed reality (MR) functionalities applicable to the real-world object based on its attribute information;

generate one or more first MR objects, corresponding to at least one first real-world object of the one or more real-world objects, and one or more second MR objects, corresponding to at least one second real-world object of the one or more real-world objects;

render, for one or more displays of the VR display device, the VR environment with the one or more first MR objects corresponding to the at least one first real-world object of the one or more real-world objects,

wherein each first MR object comprises one of the determined MR functionalities applicable to the corresponding first real-world object, and

wherein the rendered VR environment omits the one or more second MR objects corresponding to the at least one second real-world object of the one or more real-world objects; and

displaying the at least one second real-world object, via a passthrough view of the real-world environment, while rendering the VR environment.

18. The media of claim 17 , further comprising:

receive an instruction to transition to a second VR environment of the VR application;

determine, for each of the one or more real-world objects, one or more second MR functionalities based on the second VR environment of the VR application and applicable to the real-world object based on its attribute information; and

render, for one or more displays of the VR display device, the second VR environment with the one or more first MR objects corresponding to the at least one first real-world object, wherein each first MR object comprises one of the second MR functionalities applicable to the corresponding first real-world object.

19. A mobile computing device comprising:

one or more processors; and

a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

receive an instruction to transition to a virtual reality (VR) environment of a VR application;

access attribute information of one or more real-world objects in a real-world environment proximate to a VR display device;

determine, for each of the one or more real-world object, one or more mixed reality (MR) functionalities applicable to the real-world object based on its attribute information;

generate one or more first MR objects, corresponding to at least one first real-world object of the one or more real-world objects, and one or more second MR objects, corresponding to at least one second real-world object of the one or more real-world objects;

render, for one or more displays of the VR display device, the VR environment with the one or more first MR objects corresponding to the at least one first real-world object of the one or more real-world objects,

wherein each first MR object comprises one of the determined MR functionalities applicable to the corresponding first real-world object, and

wherein the rendered VR environment omits the one or more second MR objects corresponding to the at least one second real-world object of the one or more real-world objects; and

display the at least one second real-world object, via a passthrough view of the real-world environment, while rendering the VR environment.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2022
From: TANNER, CHRISTOPHER RICHARD; HAVILIO, AMIR MESGUICH; PUJALS, MICHELLE
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 061224/0201 →
CHANGE OF NAME Recorded Jul 6, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060591/0848 →