Non-uniform stereo rendering
Examples of the disclosure describe systems and methods for recording augmented reality and mixed reality experiences. In an example method, an image of a real environment is received via a camera of a wearable head device. A pose of the wearable head device is estimated, and a first image of a virtual environment is generated based on the pose. A second image of the virtual environment is generated based on the pose, wherein the second image of the virtual environment comprises a larger field of view than a field of view of the first image of the virtual environment. A combined image is generated based on the second image of the virtual environment and the image of the real environment.
1. A method comprising:
receiving, via one or more sensors of a wearable head device, an image of a real environment;
estimating a pose of the wearable head device, wherein the pose is estimated via performing, based on an output of the one or more sensors, one or more of Simultaneous Localization and Mapping (SLAM) and visual-inertial odometry;
generating a first image of a virtual environment based on the pose;
generating a second image of the virtual environment based on the pose, wherein the second image of the virtual environment comprises a larger field of view than a field of view of the first image of the virtual environment; and
generating a combined image based on the second image of the virtual environment and the image of the real environment, wherein:
generating the combined image comprises performing a planar projection, the planar projection comprising a mapping of a point of the second image of the virtual environment to a corresponding point of the image of the real environment.
2. The method of claim 1 , further comprising:
generating a cropped image based on the second image of the virtual environment; and
presenting, via a display of the wearable head device, a stereoscopic image based on the first image of the virtual environment and the cropped image.
3. The method of claim 2 , wherein the cropped image comprises a center of the cropped image, wherein the second image of the virtual environment comprises a center of the second image, and wherein the center of the cropped image is the same as the center of the second image.
4. The method of claim 2 , wherein the cropped image comprises a cropped field of view, and wherein the field of view of the second image of the virtual environment is larger than the cropped field of view.
5. The method of claim 1 , wherein the second image of the virtual environment comprises an aspect ratio, and wherein the image of the real environment comprises the same aspect ratio.
6. The method of claim 1 , wherein the second image of the virtual environment comprises an aspect ratio, and wherein the first image of the virtual environment comprises the same aspect ratio.
7. The method of claim 1 , wherein the first image of the virtual environment comprises a first framebuffer size, wherein the second image of the virtual environment comprises a second framebuffer size, and wherein the second framebuffer size is larger than the first framebuffer size.
8. The method of claim 1 , wherein the first image of the virtual environment comprises a framebuffer size, and wherein the cropped image comprises the same framebuffer size.
9. The method of claim 1 , wherein the first image of the virtual environment comprises a first resolution, wherein the second image of the virtual environment comprises a second resolution, and wherein the second resolution is larger than the first resolution.
10. The method of claim 1 , wherein the one or more sensors comprises a camera, and the camera is configured to be placed to a left side of a user's left eye.
11. The method of claim 1 , wherein the one or more sensors comprises a camera, and the camera is configured to be placed to a right side of a user's right eye.
12. The method of claim 1 , wherein the one or more sensors comprises a camera, and the camera is configured to be placed in between a user's left eye and a user's right eye.
13. The method of claim 1 , further comprising presenting, via a display, the combined image to a second device, different from the wearable head device.
14. The method of claim 1 , wherein a focal point of the image of the real environment, the point of the second image of the virtual environment, and the corresponding point of the image of the real environment are collinear.
15. A system comprising:
one or more sensors of a wearable head device;
one or more processors configured to execute a method comprising:
receiving, via the one or more sensors of the wearable head device, an image of a real environment;
estimating a pose of the wearable head device, wherein the pose is estimated via performing, based on an output of the one or more sensors, one or more of Simultaneous Localization and Mapping (SLAM) and visual-inertial odometry;
generating a first image of a virtual environment based on the pose;
generating a second image of the virtual environment based on the pose, wherein the second image of the virtual environment comprises a larger field of view than a field of view of the first image of the virtual environment; and
generating a combined image based on the second image of the virtual environment and the image of the real environment, wherein:
generating the combined image comprises performing a planar projection, the planar projection comprising a mapping of a point of the second image of the virtual environment to a corresponding point of the image of the real environment.
16. The system of claim 15 , further comprising:
generating a cropped image based on the second image of the virtual environment; and
presenting, via a display of the wearable head device, a stereoscopic image based on the first image of the virtual environment and the cropped image.
17. The system of claim 16 , wherein the cropped image comprises a center of the cropped image, wherein the second image of the virtual environment comprises a center of the second image, and wherein the center of the cropped image is the same as the center of the second image.
18. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute a method comprising:
receiving, via one or more sensors of a wearable head device, an image of a real environment;
estimating a pose of the wearable head device, wherein the pose is estimated via performing, based on an output of the one or more sensors, one or more of Simultaneous Localization and Mapping (SLAM) and visual-inertial odometry;
generating a first image of a virtual environment based on the pose;
generating a second image of the virtual environment based on the pose, wherein the second image of the virtual environment comprises a larger field of view than a field of view of the first image of the virtual environment; and
generating a combined image based on the second image of the virtual environment and the image of the real environment, wherein:
generating the combined image comprises performing a planar projection, the planar projection comprising a mapping of a point of the second image of the virtual environment to a corresponding point of the image of the real environment.
19. The non-transitory computer-readable medium of claim 18 , further comprising:
generating a cropped image based on the second image of the virtual environment; and
presenting, via a display of the wearable head device, a stereoscopic image based on the first image of the virtual environment and the cropped image.
20. The non-transitory computer-readable medium of claim 19 , wherein the cropped image comprises a center of the cropped image, wherein the second image of the virtual environment comprises a center of the second image, and wherein the center of the cropped image is the same as the center of the second image.