Image processing apparatus that obtains model data, control method of image processing apparatus, and storage medium
A technique of the present disclosure includes: an obtaining unit configured to obtain model data indicating a three-dimensional shape of an object in a space; a receiving unit configured to receive viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint; a generating unit configured to generate image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and a transmitting unit configured to transmit the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus.
1. An image processing apparatus comprising:
an obtaining unit configured to obtain model data indicating a three-dimensional shape of an object in a space;
a receiving unit configured to receive viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint;
a generating unit configured to generate image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and
a transmitting unit configured to transmit the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus,
wherein the generated image data and the second portion of the obtained model data are transmitted in a ratio based, at least in part, on a bandwidth of a communication channel between the image processing apparatus and the external apparatus.
2. The image processing apparatus according to claim 1 , wherein the first portion of the model data is data indicating a three-dimensional shape of at least one of a plurality of objects in the space.
3. The image processing apparatus according to claim 2 , wherein the at least one object is selected based on a type of the object.
4. The image processing apparatus according to claim 2 , wherein the at least one object is selected based on a distance between the virtual viewpoint and the object.
5. The image processing apparatus according to claim 2 , wherein the at least one object is selected based on a distance between a specific object among the plurality of objects and each of the other objects.
6. The image processing apparatus according to claim 2 , wherein
the receiving unit receives interest information indicating a point of interest of a user from the external apparatus, and
the at least one object is selected based on the interest information.
7. The image processing apparatus according to claim 2 , wherein the at least one object is selected based on a bandwidth of a communication channel to which the image processing apparatus is connected, such that a size of data transmitted from the image processing apparatus does not exceed the bandwidth of the communication channel.
8. The image processing apparatus according to claim 1 , wherein the obtaining unit generates the model data based on a plurality of captured images obtained by capturing images of the space from different directions with a plurality of image capturing apparatuses.
9. The image processing apparatus according to claim 1 , wherein the model data is point cloud data, voxel data, or mesh data.
10. The image processing apparatus according to claim 1 , wherein the virtual viewpoint image is generated by synthesizing the image data transmitted from the image processing apparatus and other image data generated by rendering the second portion of the model data based on the viewpoint information.
11. The image processing apparatus according to claim 10 , wherein the transmitting unit further transmits distance information to the external apparatus, the distance information used for the synthesizing and indicating a distance between the virtual viewpoint and the object.
12. The image processing apparatus according to claim 1 , wherein
the receiving unit receives scene information from the external apparatus, the scene information specifying a scene corresponding to the virtual viewpoint image to be generated, and
the obtaining unit obtains the model data indicating a three-dimensional shape of an object in the scene specified by the received scene information.
13. The image processing apparatus according to claim 1 , wherein the image data transmitted from the image processing apparatus includes partial image data subjected to lossless compression and partial image data subjected to lossy compression.
14. A control method of an image processing apparatus, comprising:
obtaining model data indicating a three-dimensional shape of an object in a space;
receiving viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint;
generating image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and
transmitting the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus,
wherein the generated image data and the second portion of the obtained model data are transmitted in a ratio based, at least in part, on a bandwidth of a communication channel between the image processing apparatus and the external apparatus.
15. The control method according to claim 14 , wherein the first portion of the model data is data indicating a three-dimensional shape of at least one of a plurality of objects in the space.
16. The control method according to claim 15 , wherein the at least one object is selected based on a bandwidth of a communication channel to which the image processing apparatus is connected, such that a size of data transmitted from the image processing apparatus does not exceed the bandwidth of the communication channel.
17. A non-transitory computer-readable storage medium storing a program that causes an image processing apparatus to execute a control method comprising:
obtaining model data indicating a three-dimensional shape of an object in a space;
receiving viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint;
generating image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and
transmitting the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus,
wherein the generated image data and the second portion of the obtained model data are transmitted in a ratio based, at least in part, on a bandwidth of a communication channel between the image processing apparatus and the external apparatus.