Reconstruction of three-dimensional video
In one embodiment, a method includes accessing multiple images representing a 360° view of a scene, each image representing a portion of the 360° scene and including a first overlap area and a second overlap area, the first overlap area corresponding to an overlap with a first adjacent image, and the second overlap area corresponding to an overlap with a second adjacent image. The method also includes, calculating, for each overlap area, a set of homographies based on one or more feature points in the overlap area. The method further includes creating a set of left images from the multiple images, the set of left images representing a first 360° view of the scene. The method also includes creating a set of right images from the multiple images, the set of right images representing a second 360° view of the scene.
1. A method comprising:
accessing a plurality of images representing a 360° view of a scene, each image representing a portion of the 360° view of the scene and comprising a first overlap area and a second overlap area, the first overlap area corresponding to an overlap with a first adjacent image, and the second overlap area corresponding to an overlap with a second adjacent image, wherein the plurality of accessed images without modification do not produce a three-dimensional view of the scene;
calculating, for each of the overlap areas, a set of homographies based on one or more feature points in the overlap area;
creating a set of left images from the plurality of images, the set of left images representing a first 360° view of the scene;
creating a set of right images from the plurality of images, the set of right images representing a second 360° view of the scene, wherein the set of right images is distinct from the set of left images;
stitching the set of left images together using the calculated homographies to produce a stitched 360° left image;
stitching the set of right images together using the calculated homographies to produce a stitched 360° right image that is distinct from the stitched 360° left image;
providing for display to a left eye of a user at least a portion of the stitched 360° left image; and
providing for display to a right eye of the user at least a portion of the stitched 360° right image.
2. The method of claim 1 , wherein the plurality of images are captured by a plurality of cameras, wherein each camera of the plurality of cameras has greater than or equal to 50% angular overlap with an adjacent camera.
3. The method of claim 2 , wherein each camera has a different orientation.
4. The method of claim 1 , wherein:
the first overlap area of an image of the plurality of images comprises greater than or equal to 50% of an area of the image; and
the second overlap area of the image comprises greater than or equal to 50% of the area of the image.
5. The method of claim 1 , wherein:
the set of left images is formed from the first overlap area of each of the plurality of images; and
the set of right images is formed from the second overlap area of each of the plurality of images.
6. The method of claim 1 , further comprising, stitching the plurality of images together to form a stitched image, the stitching based at least in part on the feature points in overlap areas of adjacent images.
7. The method of claim 1 , further comprising:
displaying to the left eye of the user at least a portion of the stitched 360° left image; and
displaying to the right eye of the user at least a portion of the stitched 360° right image.
8. The method of claim 1 , wherein:
providing for display to the left eye of the user at least the portion of the stitched 360° left image comprises sending, to a client computing device of the user, at least the portion of the stitched 360° left image; and
providing for display to the right eye of the user at least the portion of the stitched 360° right image comprises sending, to the client computing device of the user, at least the portion of the stitched 360° right image.
9. The method of claim 1 , wherein one or both of the set of left images and the set of right images is created using one or more masks.
10. The method of claim 1 , wherein one or both of the set of left images and the set of right images is created using direct image segmentation.
11. One or more non-transitory computer-readable storage media comprising instructions that are operable when executed to:
access a plurality of images representing a 360° view of a scene, each image representing a portion of the 360° view of the scene and comprising a first overlap area and a second overlap area, the first overlap area corresponding to an overlap with a first adjacent image, and the second overlap area corresponding to an overlap with a second adjacent image, wherein the plurality of accessed images without modification do not produce a three-dimensional view of the scene;
calculate, for each of the overlap areas, a set of homographies based on one or more feature points in the overlap area;
create a set of left images from the plurality of images, the set of left images representing a first 360° view of the scene;
create a set of right images from the plurality of images, the set of right images representing a second 360° view of the scene, wherein the set of right images is distinct from the set of left images;
stitch the set of left images together using the calculated homographies to produce a stitched 360° left image;
stitch the set of right images together using the calculated homographies to produce a stitched 360° right image that is distinct from the stitched 360° left image;
provide for display to a left eye of a user at least a portion of the stitched 360° left image; and
provide for display to a right eye of the user at least a portion of the stitched 360° right image.
12. The media of claim 11 , wherein the plurality of images are captured by a plurality of cameras, wherein each camera of the plurality of cameras has greater than or equal to 50% angular overlap with an adjacent camera.
13. The media of claim 12 , wherein each camera has a different orientation.
14. The media of claim 11 , wherein:
the first overlap area of an image of the plurality of images comprises greater than or equal to 50% of an area of the image; and
the second overlap area of the image comprises greater than or equal to 50% of the area of the image.
15. The media of claim 11 , wherein:
the set of left images is formed from the first overlap area of each of the plurality of images; and
the set of right images is formed from the second overlap area of each of the plurality of images.
16. The media of claim 11 , wherein the instructions are further operable when executed to stitch the plurality of images together to form a stitched image, the stitching based at least in part on the feature points in overlap areas of adjacent images.
17. The media of claim 11 , wherein the instructions are further operable when executed to:
display to the left eye of the user at least the portion of the stitched 360° left image; and
display to the right eye of the user at least the portion of the stitched 360° right image.
18. The media of claim 11 , wherein:
providing for display to the left eye of the user at least the portion of the stitched 360° left image comprises sending, to a client computing device of the user, at least the portion of the stitched 360° left image; and
providing for display to the right eye of the user at least the portion of the stitched 360° right image comprises sending, to the client computing device of the user, at least the portion of the stitched 360° right image.
19. An apparatus comprising:
one or more non-transitory computer-readable storage media embodying instructions; and
one or more processors coupled to the storage media and configured to execute the instructions to:
access a plurality of images representing a 360° view of a scene, each image representing a portion of the 360° view of the scene and comprising a first overlap area and a second overlap area, the first overlap area corresponding to an overlap with a first adjacent image, and the second overlap area corresponding to an overlap with a second adjacent image, wherein the plurality of accessed images without modification do not produce a three-dimensional view of the scene;
calculate, for each of the overlap areas, a set of homographies based on one or more feature points in the overlap area;
create a set of left images from the plurality of images, the set of left images representing a first 360° view of the scene;
create a set of right images from the plurality of images, the set of right images representing a second 360° view of the scene, wherein the set of right images is distinct from the set of left images;
stitch the set of left images together using the calculated homographies to produce a stitched 360° left image;
stitch the set of right images together using the calculated homographies to produce a stitched 360° right image that is distinct from the stitched 360° left image;
provide for display to a left eye of a user at least a portion of the stitched 360° left image; and
provide for display to a right eye of the user at least a portion of the stitched 360° right image.
20. The apparatus of claim 19 , wherein the plurality of images are captured by a plurality of cameras, wherein each camera of the plurality of cameras has greater than or equal to 50% angular overlap with an adjacent camera.
21. The apparatus of claim 20 , wherein each camera has a different orientation.
22. The apparatus of claim 19 , wherein:
the first overlap area of an image of the plurality of images comprises greater than or equal to 50% of an area of the image; and
the second overlap area of the image comprises greater than or equal to 50% of the area of the image.
23. The apparatus of claim 19 , wherein:
the set of left images is formed from the first overlap area of each of the plurality of images; and
the set of right images is formed from the second overlap area of each of the plurality of images.
24. The apparatus of claim 19 , wherein the processors are further configured to execute the instructions to stitch the plurality of images together to form a stitched image, the stitching based at least in part on the feature points in overlap areas of adjacent images.
25. The apparatus of claim 19 , wherein the processors are further configured to execute the instructions to:
display to the left eye of the user at least the portion of the stitched 360° left image; and
display to the right eye of the user at least the portion of the stitched 360° right image.
26. The apparatus of claim 19 , wherein:
providing for display to the left eye of the user at least the portion of the stitched 360° left image comprises sending, to a client computing device of the user, at least the portion of the stitched 360° left image; and
providing for display to the right eye of the user at least the portion of the stitched 360° right image comprises sending, to the client computing device of the user, at least the portion of the stitched 360° right image.