Information processing device, information processing method, video distribution method, and information processing system
There is provided an information processing device to generate a video to which a wide range of renditions are applied from a three-dimensional object generated by a volumetric technology. The information processing device includes a first generation unit ( 134 ) that generates, based on a three-dimensional model of a subject generated by using a plurality of captured images obtained by imaging the subject and based on a two-dimensional image, a video in which a subject generated from the three-dimensional model, and the two-dimensional image, are simultaneously present.
1. An information processing device comprising:
circuitry configured to
receive, based on a three-dimensional model of a subject generated by using a plurality of captured images obtained by imaging the subject, a packed image including
a texture image obtained by converting three-dimensional texture information based on a virtual viewpoint set in a virtual space in which the three-dimensional model is disposed, and
a depth image obtained by converting depth information from the virtual viewpoint to the three-dimensional model of the subject into a two-dimensional image, and
generate a video in which the subject generated from the three-dimensional model, and generated from the two-dimensional image, are simultaneously present,
wherein the texture image and the depth image are packed in one frame of the packed image.
2. The information processing device according to claim 1 ,
wherein the two-dimensional image uses at least one captured video out of the plurality of captured images used to generate the three-dimensional model of the subject.
3. The information processing device according to claim 2 ,
wherein the circuitry generates the video based on the three-dimensional model of the subject and based on the two-dimensional image including the subject temporally corresponding to the subject used to generate the three-dimensional model.
4. The information processing device according to claim 1 ,
wherein the packed image further includes at least one of the plurality of captured images.
5. The information processing device according to claim 4 ,
wherein the texture image and the captured image included in the one frame of the packed image temporally correspond to each other.
6. The information processing device according to claim 1 ,
wherein the texture image included in the packed image includes:
a first texture image obtained by converting the three-dimensional model of the subject into the two-dimensional texture information based on the virtual viewpoint set in the virtual space in which the three-dimensional model is disposed; and
a second texture image including the subject from a same viewpoint as the virtual viewpoint, the second texture image being different from the first texture image.
7. The information processing device according to claim 1 ,
wherein the circuitry is further configured to
reconstruct the three-dimensional model based on the received packed image,
render the three-dimensional model at the virtual viewpoint set in the virtual space in which the three-dimensional model is disposed, and
generate, by the reconstruction and rendering, the two-dimensional image including the subject generated from the three-dimensional model.
8. The information processing device according to claim 1 ,
wherein the circuitry generates the video by using the three-dimensional model reconstructed from the received packed image.
9. The information processing device according to claim 1 ,
wherein the circuitry is further configured to apply a shadow to a region of the subject included in the video.
10. The information processing device according to claim 1 ,
wherein the circuitry is further configured to enhance image quality of the video by using at least one captured image among the plurality of captured images.
11. The information processing device according to claim 1 ,
wherein the circuitry is further configured to perform effect processing on the video.
12. The information processing device according to claim 1 ,
wherein the circuitry is further configured to transmit the video to one or more user terminals via a predetermined network.
13. The information processing device according to claim 1 ,
wherein the circuitry is further configured to
dispose the three-dimensional model of the subject generated using the plurality of captured images obtained by imaging the subject in a three-dimensional space, and
arrange the two-dimensional image based on at least one image of the plurality of captured images in the three-dimensional space, and
wherein, by the arrangement of the two-dimensional image, the circuitry generates the video including a combined three-dimensional model including the three-dimensional model and the two-dimensional image.
14. The information processing device according to claim 13 ,
wherein the circuitry generates the video by rendering the combined three-dimensional model based on the virtual viewpoint set in the virtual space in which the three-dimensional model is disposed.
15. An information processing method comprising:
receiving, by using a computer, based on a three-dimensional model of a subject generated by using a plurality of captured images obtained by imaging the subject, a packed image including
a texture image obtained by converting three-dimensional texture information based on a virtual viewpoint set in a virtual space in which the three-dimensional model is disposed, and
a depth image obtained by converting depth information from the virtual viewpoint to the three-dimensional model of the subject into a two-dimensional image; and
generating a video in which the subject generated from the three-dimensional model, and generated from the two-dimensional image, are simultaneously present.
16. A video distribution method comprising:
receiving, based on a three-dimensional model of a subject generated by using a plurality of captured images obtained by imaging the subject, a packed image including
a texture image obtained by converting three-dimensional texture information based on a virtual viewpoint set in a virtual space in which the three-dimensional model is disposed, and
a depth image obtained by converting depth information from the virtual viewpoint to the three-dimensional model of the subject into a two-dimensional image;
generating a video in which the subject generated from the three-dimensional model, and generated from the two-dimensional image, are simultaneously present; and
distributing the video to a user terminal via a predetermined network.
17. An information processing system comprising:
an imaging device configured to image a subject to generate a plurality of captured images of the subject;
an information processing device configured to
receive, based on a three-dimensional model of a subject generated by using a plurality of captured images obtained by imaging the subject, a packed image including
a texture image obtained by converting three-dimensional texture information based on a virtual viewpoint set in a virtual space in which the three-dimensional model is disposed, and
a depth image obtained by converting depth information from the virtual viewpoint to the three-dimensional model of the subject into a two-dimensional image, and
generate a video in which the subject generated from the three-dimensional model generated by using the plurality of captured images, and generated from the two-dimensional image, are simultaneously present; and
a user terminal configured to display the video generated by the information processing device to a user.