Generation apparatus, generation method, and storage medium
A generation apparatus includes following units. A first acquisition unit acquires a first virtual viewpoint image generated based on a plurality of images captured by a plurality of image capturing apparatuses, and a first virtual viewpoint in a three-dimensional space including an image capturing space captured by the plurality of image capturing apparatuses. A second acquisition unit acquires a second virtual viewpoint image generated based on the plurality of captured images, and a second virtual viewpoint arranged with respect to a predetermined surface in the three-dimensional space and the first virtual viewpoint. A generation unit generates a third virtual viewpoint image corresponding to the first virtual viewpoint and including an image indicating a state where an object in the image capturing space is reflected on the predetermined surface, based on the first virtual viewpoint image, and the second virtual viewpoint image.
1. A generation apparatus comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to:
acquire a first virtual viewpoint image generated based on a plurality of images captured by a plurality of image capturing apparatuses and a first virtual viewpoint in a three-dimensional space;
acquire a second virtual viewpoint image generated based on the plurality of captured images and a second virtual viewpoint arranged with respect to a predetermined surface in the three-dimensional space with the first virtual viewpoint,
wherein a position of the second virtual viewpoint and a line-of-sight direction from the second virtual viewpoint are symmetrical to a position of the first virtual viewpoint and a line-of-sight direction from the first virtual viewpoint with respect to the predetermined surface; and
generate a third virtual viewpoint image corresponding to the first virtual viewpoint and including an image indicating a state where an object in the image capturing space is reflected on the predetermined surface, based on the first virtual viewpoint image and the second virtual viewpoint image.
2. The generation apparatus according to claim 1 , wherein the third virtual viewpoint image includes an image indicating a state where an object in the image capturing space is reflected on the predetermined surface.
3. The generation apparatus according to claim 1 , wherein the third virtual viewpoint image is generated based on combination of the first virtual viewpoint image and the second virtual viewpoint image.
4. The generation apparatus according to claim 3 , wherein the first virtual viewpoint image and the second virtual viewpoint image are combined in an area corresponding to the predetermined surface in the third virtual viewpoint image.
5. The generation apparatus according to claim 3 , wherein the first virtual viewpoint image and the second virtual viewpoint image are combined based on a combining ratio determined based on at least one of a reflectance of the predetermined surface, and a luminance identified by a pixel value of an area corresponding to the predetermined surface in the first virtual viewpoint image.
6. The generation apparatus according to claim 1 , wherein the second virtual viewpoint image is not generated in a case where the first virtual viewpoint is in a specific space, and the second virtual viewpoint image is generated in a case where the first virtual viewpoint is not in the specific space.
7. The generation apparatus according to claim 6 , wherein the specific space is a space different from the image capturing space captured by the plurality of image capturing apparatus.
8. The generation apparatus according to claim 1 , wherein the first virtual viewpoint image is generated based on three-dimensional shape data representing a three-dimensional shape of a first object in the image capturing space, and three-dimensional shape data representing a three-dimensional shape of a second object having the predetermined surface.
9. The generation apparatus according to claim 8 , wherein the second virtual viewpoint image is generated based on the three-dimensional shape data representing the three-dimensional shape of the first object, but not based on the three-dimensional shape data representing the three-dimensional shape of the second object.
10. The generation apparatus according to claim 1 , wherein the third virtual viewpoint image is generated based on the first virtual viewpoint image, and the second virtual viewpoint image subjected to predetermined processing based on surface information about the predetermined surface.
11. The generation apparatus according to claim 10 ,
wherein the surface information includes information representing a reflectance of the predetermined surface, and
wherein the predetermined processing changes a pixel value of the second virtual viewpoint image, based on the reflectance included in the surface information.
12. The generation apparatus according to claim 11 , wherein the predetermined processing further changes a pixel value of the first virtual viewpoint image, based on the reflectance included in the surface information.
13. The generation apparatus according to claim 11 , wherein the predetermined processing changes the pixel value of the second virtual viewpoint image based on a first ratio in a case where the reflectance is a first value, and changes the pixel value of the second virtual viewpoint image based on a second ratio greater than the first ratio in a case where the reflectance is a second value greater than the first value.
14. The generation apparatus according to claim 10 ,
wherein the surface information includes information representing a shape of the predetermined surface, and
wherein the predetermined processing transforms the second virtual viewpoint image, based on the information representing the shape of the predetermined surface.
15. The generation apparatus according to claim 10 , wherein the predetermined processing changes a pixel value of the second virtual viewpoint image, based on a luminance identified by a pixel value of an area corresponding to the predetermined surface in the first virtual viewpoint image.
16. The generation apparatus according to claim 1 , wherein a pixel value of an area corresponding to the predetermined surface of the third virtual viewpoint image is determined based on a pixel value of an area corresponding to the predetermined surface of the second virtual viewpoint image, and surface information about the predetermined surface.
17. The generation apparatus according to claim 1 ,
wherein the one or more processors execute the instructions further to, in a case where the predetermined surface is each of a plurality of predetermined surfaces, acquire a plurality of second virtual viewpoint images each corresponding to a different one of the plurality of predetermined surfaces, and
wherein the third virtual viewpoint image corresponding to the first virtual viewpoint is generated, based on the first virtual viewpoint image and the plurality of second virtual viewpoint images.
18. A generation method comprising:
acquiring a first virtual viewpoint image generated based on a plurality of images captured by a plurality of image capturing apparatuses and a first virtual viewpoint in a three-dimensional space;
acquiring a second virtual viewpoint image generated based on the plurality of captured images and a second virtual viewpoint arranged with respect to a predetermined surface in the three-dimensional space with the first virtual viewpoint,
wherein a position of the second virtual viewpoint and a line-of-sight direction from the second virtual viewpoint are symmetrical to a position of the first virtual viewpoint and a line-of-sight direction from the first virtual viewpoint with respect to the predetermined surface; and
generating a third virtual viewpoint image corresponding to the first virtual viewpoint and including an image indicating a state where an object in the image capturing space is reflected on the predetermined surface, based on the first virtual viewpoint image.
19. A non-transitory computer-readable storage medium that stores a computer program for causing a computer to:
acquiring a first virtual viewpoint image generated based on a plurality of images captured by a plurality of image capturing apparatuses and a first virtual viewpoint in a three-dimensional space;
acquiring a second virtual viewpoint image generated based on the plurality of captured images and a second virtual viewpoint arranged with respect to a predetermined surface in the three-dimensional space with the first virtual viewpoint,
wherein a position of the second virtual viewpoint and a line-of-sight direction from the second virtual viewpoint are symmetrical to a position of the first virtual viewpoint and a line-of-sight direction from the first virtual viewpoint with respect to the predetermined surface; and
generating a third virtual viewpoint image corresponding to the first virtual viewpoint and including an image indicating a state where an object in the image capturing space is reflected on the predetermined surface, based on the first virtual viewpoint image.