Generating content for presentation by a head mounted display based on data captured by a light field camera positioned on the head mounted display

1. A method comprising:

capturing information about light from an environment external to a head mounted display (HMD) using a light field camera positioned on the HMD so a field of view of the light field camera is external to the HMD, the light field camera having a specific position on the HMD relative to an eye of a user wearing the HMD;

determining a virtual position of the light field camera based on the specific position of the light field camera on the HMD relative to the eye of the user wearing the HMD by

obtaining an image of the user's eye from a camera included in the front rigid body that is directed towards the user's eye and that has specific position relative to a portion of the HMD, and

determining a position of the user's eye from the image and the specific position relative to the portion of the HMD, the virtual position of the light field camera comprising the determined position of the user's eye;

selecting rays of light captured by the light field camera projected to intersect a plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera; and

generating one or more images representing a portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera.

2. The method of claim 1 , wherein determining the position of the user's eye from the image and the specific position relative to the portion of the HMD comprises:

obtaining a specific horizontal distance from a center of an eyecup assembly included in the HMD from the light field camera;

obtaining a specific vertical distance from the center of the eyecup assembly included in the HMD from the light field camera; and

determining the position of the user's eye within the HMD from the one or more images of the eye of the user obtained from the camera, the specific position relative to the portion of the HMD, the specific horizontal distance, and the specific vertical distance.

3. The method of claim 1 , wherein the information about light from the environment external to the HMD comprises intensity and direction of rays of light external to the HMD captured by the light field camera.

4. The method of claim 3 , wherein selecting rays of light captured by the light field camera projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

projecting rays of light captured by the light field camera to the virtual position based on the direction of rays of light external to the HMD captured by the light field camera; and

selecting rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera.

5. The method of claim 4 , wherein generating one or more images representing the portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

retrieving intensity information corresponding to each of the selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera; and

generating an image so an intensity at a position within the image corresponds to intensities of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position.

6. The method of claim 5 , wherein generating the image so the intensity at the position within the image corresponds to intensities of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position comprises:

retrieving color information captured by the light field camera corresponding to selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera; and

generating the image so a color at the position within the image corresponds to color information of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position.

7. A system comprising:

a head mounted display (HMD) including:

a light field camera coupled to the HMD configured to capture information about light from an environment external to the HMD and within a field of view of the light field camera, the light field camera having a specific position on the HMD relative to an eye of a user wearing the HMD and positioned on the HMD so the field of view of the light field camera is external to the HMD; and

a camera configured to capture one or more images of a portion of a face of the user wearing the HMD, the portion of the face of the user including the eye of the user; and

a console configured to receive information about light from the environment external to the HMD from the light field camera and to receive the one or more images of the portion of the face of the user wearing the HMD from the camera, the console configured to:

determine a virtual position of the light field camera based on the specific position of the light field camera on the HMD relative to the eye of the user wearing the HMD and the one or more images of the portion of the face of the user wearing the HMD by

obtaining an image of the user's eye from a camera included in the front rigid body that is directed towards the user's eye and that has specific position relative to a portion of the HMD, and

determining a position of the user's eye from the image and the specific position relative to the portion of the HMD, the virtual position of the light field camera comprising the determined position of the user's eye;

select rays of light captured by the light field camera projected to intersect a plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera; and

generate one or more images representing a portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera.

8. The system of claim 7 , wherein determine the virtual position of the light field camera based on the specific position of the light field camera on the HMD relative to the eye of the user wearing the HMD and the one or more images of the portion of the face of the user wearing the HMD comprises:

obtain a specific horizontal distance from a center of an eyecup assembly included in the HMD from the light field camera;

obtain a specific vertical distance from the center of the eyecup assembly included in the HMD from the light field camera; and

determine the position of the eye of the user within the HMD from the one or more images of the eye of the user obtained from the camera, the specific position of the camera relative to the one or more portions of the HMD, the specific horizontal distance, and the specific vertical distance.

9. The system of claim 7 , wherein the information about light from the environment external to the HMD comprises intensity and direction of rays of light external to the HMD captured by the light field camera.

10. The system of claim 9 , wherein select rays of light captured by the light field camera projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

project rays of light captured by the light field camera to the virtual position based on the direction of rays of light external to the HMD captured by the light field camera; and

select rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera.

11. The system of claim 10 , wherein generate one or more images representing the portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

retrieve intensity information corresponding to each of the selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera; and

generate an image so an intensity at a position within the image corresponds to intensities of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position.

12. The system of claim 11 , wherein generate the image so the intensity at the position within the image corresponds to intensities of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position comprises:

retrieve color information captured by the light field camera corresponding to selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera; and

generate the image so a color at the position within the image corresponds to color information of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position.

13. A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

capture information about light from an environment external to a head mounted display (HMD) using a light field camera positioned on the HMD so a field of view of the light field camera is external to the HMD, the light field camera having a specific position on the HMD relative to an eye of a user wearing the HMD;

determine a virtual position of the light field camera based on the specific position of the light field camera on the HMD relative to the eye of the user wearing the HMD by

obtaining an image of the user's eye from a camera included in the front rigid body that is directed towards the user's eye and that has specific position relative to a portion of the HMD, and

determining a position of the user's eye from the image and the specific position relative to the portion of the HMD, the virtual position of the light field camera comprising the determined position of the user's eye;

select rays of light captured by the light field camera projected to intersect a plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera; and

generate one or more images representing a portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera.

14. The computer program product of claim 13 , wherein the information about light from the environment external to the HMD comprises intensity and direction of rays of light external to the HMD captured by the light field camera.

15. The computer program product of claim 14 , wherein select rays of light captured by the light field camera projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

project rays of light captured by the light field camera to the virtual position based on the direction of rays of light external to the HMD captured by the light field camera; and

select rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera.

16. The computer program product of claim 15 , wherein generate one or more images representing the portion of the environment external to the HMD from the selected rays of light captured by the light field camera that are projected to intersect the plane in the virtual position of the light field camera and corresponding to the field of view of the light field camera comprises:

retrieve intensity information corresponding to each of the selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera; and

generate an image so an intensity at a position within the image corresponds to intensities of selected rays of light projected to intersect the plane in the virtual position of the light field camera corresponding to the field of view of the light field camera at the position.