Generation of images for three-dimensional video for different viewpoints
An apparatus comprises a receiver ( 601 ) receiving captured video data for a real world scene and being linked with a capture pose region. A store ( 615 ) stores a 3D mesh model of the real world scene. A renderer ( 605 ) generates an output image for a viewport for a viewing pose. The renderer ( 605 ) comprises a first circuit ( 607 ) arranged to generate first image data for the output image by projection of captured video data to the viewing pose and second circuit ( 609 ) arranged to determine second image data for a first region of the output image in response to the three-dimensional mesh model. A third circuit ( 611 ) generates the output image to include at least some of the first image data and to include the second image data for the first region. A fourth circuit ( 613 ) determines the first region based on a deviation of the viewing pose relative to the capture pose region.
1 . An apparatus, comprising:
a first receiver circuit,
wherein the first receiver circuit is arranged to receive captured video data,
wherein the captured video data provides a dynamic representation of a real world scene,
wherein the video data is linked with a capture pose region;
a storage circuit,
wherein the storage circuit is arranged to store a three-dimensional mesh model,
wherein the three-dimensional mesh provides a static representation of a portion of the real world scene;
a second receiver circuit,
wherein the second receiver circuit is arranged to receive a viewing pose; and
a renderer circuit,
wherein the renderer circuit is arranged to generate an output image for a viewport of the viewing pose;
wherein the renderer circuit comprises a first portion, a second portion, a third portion and a fourth portion,
wherein the first portion is arranged to generate first image data for a portion of the viewport of a portion of the output image by view-shifting the captured video data from a capture pose to the viewing pose,
wherein the second portion is arranged to generate second image data for a portion of the first viewport for at least a first region of the output image using the three-dimensional mesh model,
wherein the third portion is arranged to generate the output image so as to comprise at least a portion of the first image data,
wherein the third portion is arranged to generate the output image so as to comprise the second image data of the first region,
wherein the fourth portion is arranged to determine the first region based of a deviation of the viewing pose relative to the capture pose region.
2 . The apparatus of claim 1 , wherein the renderer circuit is arranged to determine whether a quality of first image data generated by the first portion does not meet a quality criterion.
3 . The apparatus of claim 1 , wherein the third portion is arranged to determine if the first region based on a difference between the viewing pose and the capture pose region.
4 . The apparatus of claim 3 , wherein the difference is an angular difference.
5 . The apparatus of claim 1 , wherein the renderer circuit is arranged to change the second image data based on the captured video data.
6 . The apparatus of claim 1 , wherein the renderer circuit is arranged to change adapt the first data based on the three-dimensional mesh model.
7 . The apparatus of claim 1 , wherein the renderer circuit is arranged to change the second image data based on-to the first image data.
8 . The apparatus of claim 1 , wherein the renderer circuit is arranged to change the first image data based on the second image data.
9 . The apparatus of claim 1 , wherein the renderer circuit is arranged to change the three dimensional mesh model based on the first image data.
10 . The apparatus of claim 1 , further comprising a model generator circuit, wherein the model generator circuit is arranged to generate the three dimensional mesh model based on the captured video data.
11 . The apparatus of claim 1 ,
wherein the first receiver circuit is arranged to receive the video data from a remote source,
wherein the first receiver circuit is arranged to receive the three dimensional mesh model from the remote source.
12 . The apparatus of claim 1 , wherein the second portion is arranged to vary a detail level of the first region based on the deviation of the viewing pose relative to the capture zone.
13 . The apparatus of claim 1 ,
wherein the first receiver circuit is arranged to receive second captured video data of the real world scene,
wherein the second captured video data is linked with a second capture pose region,
wherein the first portion is arranged to determine third image data for a portion of the output image by projection of the second captured video data to the viewing pose,
wherein the third portion is arranged to determine the first region based on a deviation of the viewing pose with respect to the second capture pose region.
14 . A method, comprising:
receiving captured video data,
wherein the captured video data provides a dynamic representation of a real world scene,
wherein the video data is linked with a capture pose region;
storing a three-dimensional mesh model,
wherein the three-dimensional mesh model provides a static representation of a portion of the real world scene;
receiving a viewing pose; and
generating an output image for a viewport of the viewing pose; the generating comprising:
generating first image data of the viewport for a portion of the output image by view-shifting the captured video data from a capture pose to the viewing pose;
generating second image data of the viewport for at least a first region of the output image using the three-dimensional mesh model;
generating the output image so as to comprise at least a portion of the first image data and the second image data of the first region; and
determining the first region based on a deviation of the viewing pose relative to the capture pose region.
15 . A computer program stored on a non-transitory medium, wherein the computer program when executed on a processor performs the method of claim 14 .
16 . The method of claim 14 , further comprising determining whether a quality of first image data does not meet a quality criterion.
17 . The method of claim 14 , further comprising determining if the first region based on a difference between the viewing pose and the capture pose region.
18 . The method of claim 17 , wherein the difference is an angular difference.
19 . The method of claim 14 , further comprising changing the second image data based on the captured video data.
20 . The method of claim 14 , further comprising changing the first data based on the three-dimensional mesh model.