Multi-resolution multi-view video rendering

1. A device for video rendering, the device comprising:

a memory; and

an electronic processor configured to

receive, from a source device, video data including multiple reference viewpoints,

determine a target image plane corresponding to a target viewpoint,

determine, within the target image plane, one or more target image regions,

determine, for each target image region within the target image plane, a proxy image region, the proxy image region being larger than the corresponding target image region by a margin surrounding the corresponding target image region,

determine, for each target image region within the target image plane, a plurality of reference pixels that fit within the corresponding proxy image region, from one or more of the multiple reference viewpoints,

perform, for each target image region within the target image plane, a forward warping operation on the plurality of reference pixels,

perform, for each target image region within the target image plane, a backwards warping operation on the plurality of reference pixels,

project, for each target image region within the target image plane, the plurality of reference pixels that fit within the corresponding proxy image region, from the one or more multiple reference viewpoints, to the target image region within the target image plane, producing a rendered target region from each target image region, and

composite one or more of the rendered target regions to create video rendering for the target image plane.

2. The device of claim 1 , wherein the at least one target image region is a plurality of target image regions, and respective ones of the plurality of target image regions are rendered with different resolutions.

3. The device of claim 1 , wherein the at least one target image region is a plurality of target image regions, and a number of reference viewpoints is different for each target image region.

4. The device of claim 1 , wherein the proxy image region is determined based on at least one of a depth range, a virtual camera array layout, or real camera array layout.

5. The device of claim 1 , wherein the margin is predetermined.

6. The device of claim 1 , wherein at least one target image region is determined based on a focus area of a viewer.

7. The device of claim 6 , wherein the focus area is a foveation area having a resolution greater than a resolution of a peripheral area within the target image region.

8. A method for video rendering, the method comprising:

receiving, from a source device, video data including multiple reference viewpoints;

determining a target image plane corresponding to a target viewpoint,

determining, within the target image plane, one or more target image regions,

determining, for each target image region within the target image plane, a proxy image region, the proxy image region being larger than the corresponding target image region by a margin surrounding the corresponding target image region;

determining, for each target image region within the target image plane, a plurality of reference pixels that fit within the corresponding proxy image region, from one or more of the multiple reference viewpoints;

performing, for each target image region within the target image plane, a forward warping operation on the plurality of reference pixels,

performing, for each target image region within the target image plane, a backwards warping operation on the plurality of reference pixels,

projecting, for each target image region within the target image plane, the plurality of reference pixels that fit within the corresponding proxy image region, from the one or more multiple reference viewpoints, to the target image region within the target image plane, producing a rendered target region from each target image region; and

compositing one or more of the rendered target regions to create video rendering for the target image plane.

9. The method of claim 8 , wherein the at least one target image region is a plurality of target image regions, and respective ones of the plurality of target image regions are is a plurality of target image regions, and respective ones of the plurality of target image regions are rendered with different resolutions.

10. The method of claim 8 , wherein the at least one target image region is a plurality of target image regions, and a number of reference viewpoints is different for each target image region.

11. The method of claim 8 , wherein the proxy image region is determined based on at least one of a depth range, a virtual camera array layout, or real camera array layout.

12. The method of claim 8 , wherein the margin is predetermined.

13. The method of claim 8 , wherein at least one target image region is determined based on a focus area of a viewer.

14. The method of claim 13 , wherein the focus area is a foveation area having a resolution greater than a resolution of a peripheral area within the target image region.

15. A device for video rendering, the device comprising: a memory; and

an electronic processor configured to

receive source content including multiple reference viewpoints,

map the source content using a projective mapping technique,

partition the mapped source content into a plurality of tiles,

determine a subset of the plurality of regions for synthesis of a target view based on a proxy image region surrounding each target image region, wherein the proxy image region is

determined based on at least one of a depth range, a virtual camera array layout, or real camera array layout,

determine, for each target image region, a plurality of reference pixels that fit within the corresponding proxy image region, from one or more of the multiple reference viewpoints,

render the subset of the plurality of regions by performing a forward warping operation and performing a backwards warping operation on pixels within the proxy image region surrounding each target image region,

project, for each target image region, the plurality of reference pixels that fit within the corresponding proxy image region, from the one or more multiple reference viewpoints, to the target image region, producing a rendered target region from each target image region, and

composite one or more of the rendered target regions to create video rendered for the target view.

16. The device of claim 15 , wherein a respective size of a respective tile is based on the resolution of the tile.

17. The device of claim 15 , wherein a number of reference viewpoints is different for each target image region.

18. The device of claim 15 , wherein the projective mapping technique is an equirectangular mapping or a cube mapping.