IP Library Granted Patent US 11,704,879
Granted Patent B2
US 11,704,879 · App. 17/551,926 · Granted Jul 18, 2023

Systems and methods for providing spatial awareness in virtual reality

Inventor: Eugene Lee (Mountain View, CA)
Assignee: Meta Platforms Technologies, LLC
G06T19/006G02B27/0093G02B27/017G06F3/04815
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,704,879
App. No.
17/551,926
Granted
Jul 18, 2023
Kind
B2
Abstract

In one embodiment, a method includes rendering, for a VR display device and based on a field of view of a user, a first output image of a VR environment comprising a virtual boundary corresponding to a real-world environment; determining, responsive to the user approaching within a first threshold distance of the virtual boundary, a speed of movement and the field of view of the user; accessing one or more images of the real-world environment captured by one or more cameras of the VR display device; and rendering, for the VR display device, a second output image comprising a portion of the VR environment and a portion of a passthrough view of the real-world environment based on the accessed images, where an area of the portion of the passthrough view is based on the determined speed of movement and the field of view of the user.

Claims (42)

1. A method comprising, by one or more computing systems:

rendering, for a virtual reality (VR) display device and based on a field of view of a user, a first output image of a VR environment comprising a virtual boundary corresponding to a real-world environment;

determining, responsive to the user approaching within a first threshold distance of the virtual boundary, a speed of movement and the field of view of the user;

accessing one or more images of the real-world environment captured by one or more cameras of the VR display device; and

rendering, for the VR display device, a second output image comprising a portion of the VR environment and a portion of a passthrough view of the real-world environment based on the accessed images, wherein an area of the portion of the passthrough view is based on the determined speed of movement and the field of view of the user.

2. The method of claim 1 , further comprising:

rendering, for the one or more displays of the VR display device, a third output image comprising one or more real-world objects beyond the virtual boundary as one or more mixed reality (MR) objects.

3. The method of claim 2 , further comprising:

determining whether the user is approaching within a second threshold distance of the virtual boundary, wherein the second threshold distance is greater than the first threshold distance;

determining, responsive to the user approaching within the second threshold distance of the virtual boundary, a direction of movement and the field of view of the user; and

accessing one or more additional images of the real-world environment containing the one or more real-world objects captured by cameras of the VR display device;

wherein the third output image comprises the one or more real-world objects in the accessed additional images.

4. The method of claim 2 , wherein the one or more MR objects are rendered as an outline of the one or more objects, a semi-opaque rendering of the one or more objects, or a fully opaque rendering of the one or more objects.

5. The method of claim 1 , wherein the area of the portion of the passthrough view is based on the determined speed of the user, wherein the portion is a spherical cap of a spherical second output image.

6. The method of claim 5 , wherein the area of the portion of the passthrough view is relatively larger for a faster determined speed of the movement of the user, and wherein the area of the portion of the passthrough view is relatively smaller for a slower determined speed of the movement of the user.

7. The method of claim 1 , wherein a sharpness of a transition from the VR environment to the portion of the passthrough view is based on the determined speed of the movement of the user, wherein the transition is a fade from the VR environment into the portion of the passthrough view.

8. The method of claim 7 , wherein the sharpness of the transition from the VR environment to the portion of the passthrough view is relatively sharper for a faster determined speed of the movement of the user, and wherein the sharpness of the transition from the VR environment to the portion of the passthrough view is relatively less sharp for a slower determined speed of the movement of the user.

9. The method of claim 1 , wherein the rendered portion of the passthrough view corresponds to the direction of movement of the user.

10. The method of claim 9 , wherein the rendered portion of the passthrough view is in the field of view of the user when the direction of movement of the user is determined to be toward the field of view.

11. The method of claim 9 , wherein the rendered portion of the passthrough view is in a peripheral view of the user when the direction of movement is determined to be perpendicular to the field of view.

12. The method of claim 9 , wherein the rendered portion of the passthrough view is in a peripheral view of and behind the user when the direction of movement is determined to be away from the field of view.

13. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

render, for a virtual reality (VR) display device and based on a field of view of a user, a first output image of a VR environment comprising a virtual boundary corresponding to a real-world environment;

determine, responsive to the user approaching within a first threshold distance of the virtual boundary, a speed of movement and the field of view of the user;

access one or more images of the real-world environment captured by one or more cameras of the VR display device; and

render, for the VR display device, a second output image comprising a portion of the VR environment and a portion of a passthrough view of the real-world environment based on the accessed images, wherein an area of the portion of the passthrough view is based on the determined speed of movement and the field of view of the user.

14. The media of claim 13 , wherein the software is further operable when executed to: render, for the one or more displays of the VR display device, a third output image comprising one or more real-world objects beyond the virtual boundary as one or more mixed reality (MR) objects.

15. The media of claim 14 , wherein the software is further operable when executed to:

determine whether the user is approaching within a second threshold distance of the virtual boundary, wherein the second threshold distance is greater than the first threshold distance;

determine, responsive to the user approaching within the second threshold distance of the virtual boundary, a direction of movement and the field of view of the user; and

access one or more additional images of the real-world environment containing the one or more real-world objects captured by cameras of the VR display device;

wherein the third output image comprises the one or more real-world objects in the accessed additional images.

16. The media of claim 14 , wherein the one or more MR objects are rendered as an outline of the one or more objects, a semi-opaque rendering of the one or more objects, or a fully opaque rendering of the one or more objects.

17. The media of claim 13 , wherein the area of the portion of the passthrough view is based on the determined speed of the user, wherein the portion is a spherical cap of a spherical second output image.

18. The media of claim 17 , wherein the area of the portion of the passthrough view is relatively larger for a faster determined speed of the movement of the user, and wherein the area of the portion of the passthrough view is relatively smaller for a slower determined speed of the movement of the user.

19. The media of claim 13 , wherein the software is further operable when executed to:

determining a speed of the movement of the user, wherein a sharpness of a transition from the VR environment to the portion of the passthrough view is based on the determined speed of the movement of the user, wherein the transition is a fade from the VR environment into the portion of the passthrough view.

20. A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

render, for a virtual reality (VR) display device and based on a field of view of a user, a first output image of a VR environment comprising a virtual boundary corresponding to a real-world environment;

determine, responsive to the user approaching within a first threshold distance of the virtual boundary, a speed of movement and the field of view of the user;

access one or more images of the real-world environment captured by one or more cameras of the VR display device; and

render, for the VR display device, a second output image comprising a portion of the VR environment and a portion of a passthrough view of the real-world environment based on the accessed images, wherein an area of the portion of the passthrough view is based on the determined speed of movement and the field of view of the user.

Assignments (1)
CHANGE OF NAME Recorded Jul 6, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060591/0848 →
Continuity (2)
Continuation 17139434 · Dec 31, 2020
Related Publication 20220207837A1 · Jun 30, 2022
Cited By (4)
US 12,283,002 US 12,315,092 US 12,400,409 US 12,573,100