System and method for generating a three-dimensional photographic image
A method includes receiving, from a camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame comprises a raster of pixels of image data; detecting layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels in the one or more frames; and determining a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising first and second discrete layers, wherein each discrete layer is associated with a unique depth value relative to the camera. The method further includes mapping the pixels to the layers of the plurality of discrete layers; rendering the pixels as a first image of the scene as viewed from a first perspective; and rendering the pixels as a second image of the scene as viewed from a second perspective.
1. A method of generating a two-dimensional image with parallax shift effects providing an appearance of three-dimensional depth, the method comprising:
receiving, from a camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame of image data comprises a raster of pixels of image data;
detecting, based on the one or more frames of image data, layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels of image data in the one or more frames of image data;
determining, based on the layer information of the scene, a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising a first discrete layer and a second discrete layer, wherein each discrete layer is associated with a unique depth value relative to the camera;
mapping the pixels of image data to the discrete layers of the plurality of discrete layers;
rendering the pixels of image data as a first image of the scene as viewed from a first perspective; and
rendering the pixels of image data as a second image of the scene as viewed from a second perspective, the second image comprising a parallax shift effect of content in the first image based on repositioning the first discrete layer relative to the second discrete layer;
wherein one or more pixels of the first discrete layer occlude a first set of pixels of the second discrete layer in the first image;
wherein the one or more pixels of the first discrete layer occlude a second set of pixels of the second discrete layer in the second image; and
wherein the first and second discrete layers of the multi-layer model are selected from a set of possible layers as satisfying depth and size criteria for providing the parallax shift effect.
2. The method of claim 1 , wherein the one or more frames of image data comprise a single frame of image data obtained from a single camera perspective.
3. The method of claim 2 , further comprising:
passing the single frame of image data to a machine model to reconstruct a depth map; and
detecting the layer information of the scene based on the reconstructed depth map.
4. The method of claim 1 , further comprising:
detecting the one or more three-dimensional objects of the scene; and
determining the multi-layer model of the scene based on a depth-based distribution of the one or more detected three-dimensional objects of the scene.
5. The method of claim 4 , wherein a number of discrete layers of the multi-layer model is based on a number of detected three-dimensional objects in the scene whose size exceeds a threshold size.
6. The method of claim 1 , further comprising:
determining the depth-based distribution of the pixels of image data in the one or more frames of image data;
identifying peaks in the depth-based distribution of the pixels of image data exceeding an amplitude threshold value and a depth separation threshold value; and
creating one or more discrete layers of the multi-layer model based on the identified peaks exceeding the amplitude threshold value and the depth separation threshold value;
wherein the depth and size criteria for providing the parallax shift effect comprise the amplitude threshold value and the depth separation threshold value.
7. The method of claim 1 , wherein the plurality of discrete layers comprises:
a foreground layer at a front of the scene;
a background layer at a back of the scene; and
one or more intermediate layers, wherein each of the one or more intermediate layers is associated with a unique depth value between a depth value of the foreground layer and a depth value of the background layer.
8. An apparatus for generating a two-dimensional image with parallax shift effects providing an appearance of three-dimensional depth, the apparatus comprising:
a camera; and
a controller configured to:
receive, from the camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame of image data comprises a raster of pixels of image data;
detect, based on the one or more frames of image data, layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels of image data in the one or more frames of image data;
determine, based on the layer information of the scene, a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising a first discrete layer and a second discrete layer, wherein each discrete layer is associated with a unique depth value relative to the camera;
map the pixels of image data to the discrete layers of the plurality of discrete layers;
render the pixels of image data as a first image of the scene as viewed from a first perspective; and
render the pixels of image data as a second image of the scene as viewed from a second perspective, the second image comprising a parallax shift effect of content in the first image based on repositioning the first discrete layer relative to the second discrete layer;
wherein one or more pixels of the first discrete layer occlude a first set of pixels of the second discrete layer in the first image;
wherein the one or more pixels of the first discrete layer occlude a second set of pixels of the second discrete layer in the second image; and
wherein the first and second discrete layers of the multi-layer model are selected from a set of possible layers as satisfying depth and size criteria for providing the parallax shift effect.
9. The apparatus of claim 8 , wherein the one or more frames of image data comprise a single frame of image data obtained from a single camera perspective.
10. The apparatus of claim 9 , wherein the controller is further configured to:
pass the single frame of image data to a machine model to reconstruct a depth map; and
detect the layer information of the scene based on the reconstructed depth map.
11. The apparatus of claim 8 , wherein the controller is further configured to:
detect the one or more three-dimensional objects of the scene; and
determine the multi-layer model of the scene based on a depth-based distribution of the one or more detected three-dimensional objects of the scene.
12. The apparatus of claim 11 , wherein a number of discrete layers of the multi-layer model is based on a number of detected three-dimensional objects in the scene whose size exceeds a threshold size.
13. The apparatus of claim 8 , wherein the controller is further configured to:
determine the depth-based distribution of the pixels of image data in the one or more frames of image data;
identify peaks in the depth-based distribution of the pixels of image data exceeding an amplitude threshold value and a depth separation threshold value; and
create one or more discrete layers of the multi-layer model based on the identified peaks exceeding the amplitude threshold value and the depth separation threshold value;
wherein the depth and size criteria for providing the parallax shift effect comprise the amplitude threshold value and the depth separation threshold value.
14. The apparatus of claim 8 , wherein the plurality of discrete layers comprises:
a foreground layer at a front of the scene;
a background layer at a back of the scene; and
one or more intermediate layers, wherein each of the one or more intermediate layers is associated with a unique depth value between a depth value of the foreground layer and a depth value of the background layer.
15. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause an apparatus to:
receive, from a camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame of image data comprises a raster of pixels of image data;
detect, based on the one or more frames of image data, layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels of image data in the one or more frames of image data;
determine, based on the layer information of the scene, a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising a first discrete layer and a second discrete layer, wherein each discrete layer is associated with a unique depth value relative to the camera;
map the pixels of image data to the discrete layers of the plurality of discrete layers;
render the pixels of image data as a first image of the scene as viewed from a first perspective; and
render the pixels of image data as a second image of the scene as viewed from a second perspective, the second image comprising a parallax shift effect of content in the first image based on repositioning the first discrete layer relative to the second discrete layer;
wherein one or more pixels of the first discrete layer occlude a first set of pixels of the second discrete layer in the first image;
wherein the one or more pixels of the first discrete layer occlude a second set of pixels of the second discrete layer in the second image; and
wherein the first and second discrete layers of the multi-layer model are selected from a set of possible layers as satisfying depth and size criteria for providing the parallax shift effect.
16. The non-transitory computer-readable medium of claim 15 , wherein the one or more frames of image data comprise a single frame of image data obtained from a single camera perspective.
17. The non-transitory computer-readable medium of claim 16 , further comprising instructions that, when executed, cause the apparatus to:
pass the single frame of image data to a machine model to reconstruct a depth map; and
detect the layer information of the scene based on the reconstructed depth map.
18. The non-transitory computer-readable medium of claim 15 , further comprising instructions that, when executed, cause the apparatus to:
detect the one or more three-dimensional objects of the scene; and
determine the multi-layer model of the scene based on a depth-based distribution of the one or more detected three-dimensional objects of the scene.
19. The non-transitory computer-readable medium of claim 18 , wherein a number of discrete layers of the multi-layer model is based on a number of detected three-dimensional objects in the scene whose size exceeds a threshold size.
20. The non-transitory computer-readable medium of claim 15 , further comprising instructions that, when executed, cause the apparatus to:
determine the depth-based distribution of the pixels of image data in the one or more frames of image data;
identify peaks in the depth-based distribution of the pixels of image data exceeding an amplitude threshold value and a depth separation threshold value; and
create one or more discrete layers of the multi-layer model based on the identified peaks exceeding the amplitude threshold value and the depth separation threshold value;
wherein the depth and size criteria for providing the parallax shift effect comprise the amplitude threshold value and the depth separation threshold value.