Information processing apparatus, information processing method, and storage medium
An object is to output data with which it is possible to generate a virtual viewpoint image of high image quality while suppressing the amount of data. The information processing apparatus according to the present disclosure obtains virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint, generates color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices, and outputs the three-dimensional shape data and the color information.
1 . An information processing apparatus comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to perform:
(1) obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
(2) generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
(3) outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data, and
wherein color information for another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data is color information that does not depend on the virtual viewpoint.
2 . The information processing apparatus according to claim 1 , wherein the element corresponding to the partial area displayed in the virtual viewpoint image is determined based on the position of the virtual viewpoint and the line-of-sight direction from the virtual viewpoint, both being identified by the virtual viewpoint information.
3 . The information processing apparatus according to claim 1 , wherein the color information is generated based on an image selected from among the plurality of images based on the position of the virtual viewpoint and the line-of-sight direction from the virtual viewpoint, both being identified by the virtual viewpoint information.
4 . The information processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to perform control so that the plurality of images is not output.
5 . The information processing apparatus according to claim 1 , wherein the virtual viewpoint information further includes information for identifying a viewing angle of the virtual viewpoint image.
6 . The information processing apparatus according to claim 1 , wherein the three-dimensional shape data is generated based on the virtual viewpoint information.
7 . The information processing apparatus according to claim 1 , wherein the three-dimensional shape data is one of point cloud, mesh data, and voxel data.
8 . The information processing apparatus according to claim 1 , wherein the virtual viewpoint information is obtained by reception from an external device via a communication unit.
9 . The information processing apparatus according to claim 8 , wherein the three-dimensional shape data and the color information are output to the external device.
10 . The information processing apparatus according to claim 9 , wherein the external device generates a virtual viewpoint image representing an appearance from the virtual viewpoint based on the three-dimensional shape data and the color information.
11 . The information processing apparatus according to claim 10 , wherein the one or more processors further execute the instructions to perform:
outputting the virtual viewpoint information, the three-dimensional shape data, and the color information to an external device different from a device that designates the virtual viewpoint.
12 . An information processing method comprising the steps of:
obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data, and
wherein color information for another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data is color information that does not depend on the virtual viewpoint.
13 . A non-transitory computer-readable storage medium storing a program for causing a computer to perform an information processing method comprising the steps of:
obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data, and
wherein color information for another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data is color information that does not depend on the virtual viewpoint.
14 . An information processing apparatus comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to perform:
(1) obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
(2) generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
(3) outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data,
wherein the generating deletes another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data, and
wherein the three-dimensional shape data that is output is three-dimensional shape data from which the another element is deleted.
15 . An information processing method comprising the steps of:
obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data,
wherein the generating deletes another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data, and
wherein the three-dimensional shape data that is output is three-dimensional shape data from which the another element is deleted.
16 . A non-transitory computer-readable storage medium storing a program for causing a computer to perform an information processing method comprising the steps of:
obtaining virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint;
generating color information for an element configuring three-dimensional shape data for identifying a three-dimensional shape of an object based on the virtual viewpoint information and a plurality of images captured from a plurality of imaging devices; and
outputting the three-dimensional shape data and the color information,
wherein the color information is generated based on the virtual viewpoint information and the plurality of images for an element corresponding to a partial area displayed in a virtual viewpoint image representing an appearance from the virtual viewpoint among elements configuring the three-dimensional shape data,
wherein the generating deletes another element different from the element corresponding to the partial area displayed in the virtual viewpoint image representing the appearance from the virtual viewpoint among the elements configuring the three-dimensional shape data, and
wherein the three-dimensional shape data that is output is three-dimensional shape data from which the another element is deleted.