IP Library › Granted Patent US 10,078,218
Granted Patent B2
US 10,078,218 · App. 14/986,645 · Granted Sep 18, 2018

Non-overlapped stereo imaging for virtual reality headset tracking

Inventors: Dov Katz (Irvine, CA); Oskar Linde (Belmont, CA); Kjell Erik Alexander Sjoholm (San Francisco, CA); Rasmus Karl Emil Simander (San Francisco, CA)
Assignee: Oculus VR, LLC
G02B27/017G06T19/006G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,078,218
App. No.
14/986,645
Granted
Sep 18, 2018
Kind
B2
Abstract

A virtual reality (VR) headset includes a first camera and a second camera capturing image data of an environment of the VR headset. Each camera has a field of view, and a portion of the fields of view of the first and second cameras overlap while a portion of the fields of view do not overlap. A processor receiving the image data from the first and second cameras is configured to identify a first observation of a position of the VR headset in the environment and positions of a plurality of features based on the image data captured by the first camera. The processor also identifies a second observation of the position of the VR headset in the environment and the positions of the features based on the image data captured by the second camera. Based on the first and second observations, the processor determines a model of the environment.

Claims (52)

1. A system comprising:

a virtual reality headset;

a first camera and a second camera on the virtual reality headset, each camera having a field of view and capturing image data of an environment of the virtual reality headset, wherein a portion of the fields of view of the first and second cameras overlap and a portion of the fields of view do not overlap; and

a processor receiving the image data from the first and second cameras and configured to:

identify a first observation of a position of the virtual reality headset in the environment and positions of a plurality of features in the environment based on the image data captured by the first camera;

identify a second observation of the position of the virtual reality headset in the environment and the positions of the plurality of features in the environment based on the image data captured by the second camera;

determine, based on features appearing in the overlapping portion of the fields of view, a scale for objects in the environment;

determine a size of an object in the environment based on features appearing in the non-overlapping portion of the fields of view and further based on the scale; and

generate a three-dimensional model of the environment, the three-dimensional model comprising a representation of the object.

2. The system of claim 1 , wherein the virtual reality headset comprises an electronic display configured to display media to a user, and wherein the processor is further configured to:

display media to the user by the electronic display based at least in part on the model of the environment.

3. The system of claim 2 , wherein displaying the media comprises:

displaying one or more virtual objects positioned in the environment based on the model of the environment.

4. The system of claim 2 , wherein displaying the media comprises:

detecting changes in positions of one or more features in the environment; and

modifying the displayed media responsive to the detected changes.

5. The system of claim 1 , wherein the processor is further configured to identify an initial position of the virtual reality headset in the environment by identifying distances between a reference point of the headset at the initial position and each of the identified features.

6. The system of claim 5 , further comprising:

one or more motion sensors on the virtual reality headset, each motion sensor generating motion data corresponding to movement of the virtual reality headset;

wherein the processor is further configured to, after the virtual reality headset has been moved from the initial position to a new position in the environment:

generate a first estimate of the new position based on the motion data;

generate a second estimate of the new position based on observations of the features in image data captured by the first and second cameras;

determine a difference between the second estimate and the first estimate; and

identify the new position of the virtual reality headset based on the first estimate, the second estimate, and the determined difference.

7. The system of claim 6 , wherein generating the second estimate comprises correlating features observed in image data captured by the first camera with features observed in image data captured by the second camera.

8. The system of claim 6 , wherein the virtual reality headset comprises an electronic display configured to display media to a user, and wherein the processor is further configured to display media to the user based at least in part on the motion data.

9. The system of claim 5 , wherein the processor is further configured to:

identify one or more additional features in the environment based on the image data captured by the first camera and the image data captured by the second camera at a new position of the virtual reality headset.

10. A method comprising:

receiving image data of an environment of a virtual reality headset, the image data captured by a first camera and a second camera coupled to the virtual reality headset and each having a field of view, wherein a portion of the fields of view of the first and second cameras overlap and a portion of the fields of view do not overlap;

identifying a first observation of a position of the virtual reality headset in the environment and positions of a plurality of features in the environment based on the image data captured by the first camera;

identifying a second observation of the position of the virtual reality headset in the environment and the positions of the plurality of features in the environment based on the image data captured by the second camera;

determining, based on features appearing in the overlapping portion of the fields of view, a scale for objects in the environment;

determining a size of an object in the environment based on features appearing in the non-overlapping portion of the fields of view and further based on the scale; and

generating a three-dimensional model of the environment, the three-dimensional model comprising a representation of the object.

11. The method of claim 10 , wherein the virtual reality headset comprises an electronic display configured to display media to a user, and wherein the method further comprises:

displaying media to the user by the electronic display based at least in part on the model of the environment.

12. The method of claim 11 , wherein displaying the media comprises:

displaying one or more virtual objects positioned in the environment based on the model of the environment.

13. The method of claim 11 , wherein displaying the media comprises:

detecting changes in positions of one or more features in the environment; and

modifying the displayed media responsive to the detected changes.

14. The method of claim 10 , further comprising:

identifying an initial position of the virtual reality headset in the environment by identifying distances between a reference point of the headset at the initial position and each of the identified features.

15. The method of claim 14 , wherein the virtual reality headset further comprises one or more motion sensors each generating motion data corresponding to movement of the virtual reality headset, the method further comprising, after the virtual reality headset has been moved from the initial position to a new position in the environment:

generating a first estimate of the new position based on the motion data;

generating a second estimate of the new position based on observations of the features in image data captured by the first and second cameras; and

determining a difference between the second estimate and the first estimate.

16. The method of claim 15 , wherein generating the second estimate comprises correlating features observed in image data captured by the first camera with features observed in image data captured by the second camera.

17. The method of claim 15 , wherein the virtual reality headset comprises an electronic display configured to display media to a user, and wherein the method further comprises displaying media to the user based at least in part on the motion data.

18. The method of claim 14 , further comprising:

identifying one or more additional features in the environment based on the image data captured by the first camera and the image data captured by the second camera at a new position of the virtual reality headset.

Assignments (4)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
CHANGE OF NAME Recorded Sep 12, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047178/0616 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2016
From: SIMANDER, RASMUS KARL EMIL
To: OCULUS VR, LLC
Reel/Frame 038941/0856 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2016
From: KATZ, DOV; LINDE, OSKAR; SJOHOLM, KJELL ERIK ALEXANDER
To: OCULUS VR, LLC
Reel/Frame 038911/0420 →
Continuity (1)
Related Publication 20170192232A1 · Jul 6, 2017
Cited By (1)
US 12,326,971