Method and device for processing image content
A method and system are provided for processing image content. In one embodiment the method comprises receiving a plurality of captured contents showing same scene as captured by one or more cameras having a different focal length and depth maps and generating a consensus cube by obtaining depth map estimations from said received contents. The visibility of different objects in then analysed to create a soft visibility cube that provides visibility information about each content. A color cube is then generated by using information from the consensus and soft visibility cube. The color cube is then used to combine different received contents and generate a single image for the plurality of contents received.
1. A method comprising:
obtaining a virtual color cube representing a volumetric scene from the point of view of a virtual camera, the virtual color cube comprising slices of the volumetric scene at different depths, each slice encoding a 2D array of pixels, wherein each pixel in each slice comprises a color specification and an alpha value;
partitioning each slice of the virtual color cube into tiles, each tile comprising a rectangular subset of the pixels of the slice; and
constructing an atlas of significant tiles, wherein constructing the atlas further comprises storing the pixels of an excluded tile in a residual image at a position corresponding to the position occupied by the tile in its original slice of the virtual color cube.
2. The method of claim 1 , further comprising for each tile in each slice of the virtual color cube, comparing the alpha values associated with the pixels of the tile to a threshold alpha value.
3. The method of claim 2 , further comprising comparing and then determining based on said comparison when each tile is to be included or excluded in the atlas of significant tiles.
4. The method of claim 3 , wherein constructing the atlas further comprises:
when processing a subsequent slice of the virtual color cube, adding the stored pixels of the excluded tile to the pixels of a corresponding tile of the subsequent slice to produce modified pixels, and
using the modified pixels when performing the comparing, determining, including and excluding steps of constructing the atlas for the corresponding tile.
5. The method of claim 1 , wherein constructing the atlas further comprises processing the slices of the virtual color cube in order of their depth values.
6. The method of claim 1 , wherein the color specification comprises R, G, and B color values.
7. The method of claim 1 , wherein the alpha value for a pixel reflects a confidence that the pixel represents a portion of the volumetric scene accurately positioned at the depth corresponding to the slice in which the pixel resides.
8. The method of claim 1 , wherein the alpha value for a pixel represents a probability.
9. The method of claim 1 , wherein the alpha value for a pixel is a value in the range from 0 to 1.
10. The method of claim 1 , wherein the alpha value for a pixel reflects the degree to which multiple cameras capturing the scene agree that a surface of an object exists at the pixel position within the slice.
11. The method of claim 1 , wherein each slice of the virtual color cube is partitioned using the same pattern of tiles.
12. An apparatus comprising:
a processor configured to:
obtain a virtual color cube representing a volumetric scene from the point of view of a virtual camera, the virtual color cube comprising slices of the volumetric scene at different depths, each slice encoding a 2D array of pixels, wherein each pixel in each slice comprises a color specification and an alpha value;
partition each slice of the virtual color cube into tiles, each tile comprising a rectangular subset of the pixels of the slice; and
construct an atlas of significant tiles, comprising storing the pixels of an excluded tile in a residual image at a position corresponding to the position occupied by the tile in its original slice of the virtual color cube.
13. The apparatus of claim 12 , wherein the processor is further configured to, for each tile in each slice of the virtual color cube, compare the alpha values associated with the pixels of the tile to a threshold alpha value.
14. The apparatus of claim 13 , wherein the processor is further configured to compare and determine based on said comparison when each tile is to be included or excluded in the atlas of significant tiles.
15. The apparatus of claim 13 , wherein the processor is further configured to:
when processing a subsequent slice of the virtual color cube, add the stored pixels of the excluded tile to the pixels of a corresponding tile of the subsequent slice to produce modified pixels; and
use the modified pixels when performing the comparing, determining, including and excluding steps of constructing the atlas for the corresponding tile.
16. The apparatus of claim 12 , wherein the processor is further configured to:
process the slices of the virtual color cube in order of their depth values.
17. The apparatus of claim 12 , wherein the color specification comprises R, G, and B color values.
18. The apparatus of claim 12 , wherein the alpha value for a pixel reflects a confidence that the pixel represents a portion of the volumetric scene accurately positioned at the depth corresponding to the slice in which the pixel resides.
19. The apparatus of claim 12 , wherein the alpha value for a pixel reflects the degree to which multiple cameras capturing the scene agree that a surface of an object exists at the pixel position within the slice.
20. The apparatus of claim 12 , wherein each slice of the virtual color cube is partitioned using the same pattern of tiles.