Image processing apparatus, method for image processing, and storage medium
An image processing apparatus for generating a virtual viewpoint video includes a first viewpoint determination unit arranged to generate a camera parameter of a first virtual camera based on an input value from a first input device, a playback-position determination unit arranged to determine a playback position of a virtual viewpoint image based on the input value from the first input device, a first-video generation unit arranged to generate a first virtual viewpoint video based on the first virtual camera and the playback position, a second viewpoint determination unit arranged to generate a camera parameter of a second virtual camera based on the input value from the first input device and an input value from a second input device, and a second video generation unit arranged to generate a second virtual viewpoint video based on the second virtual camera and the playback position.
1 . An image processing apparatus comprising
one or more memories storing instructions; and
one or more processors executing the instructions to:
obtain a plurality of images captured by a plurality of image capturing apparatuses;
generate a camera parameter of a first virtual camera based on an input value from a first input device;
determine a playback position of a virtual viewpoint image to be played back based on the input value from the first input device;
generate a first virtual viewpoint video based on the plurality of images, the camera parameter of the first virtual camera, and the playback position;
generate a camera parameter of a second virtual camera based on the input value from the first input device and an input value from a second input device; and
generate a second virtual viewpoint video based on the plurality of images, the camera parameter of the second virtual camera, and the playback position.
2 . The image processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
determine a position and an orientation of the first virtual camera based on the input value from the first input device;
determine a position of the second virtual camera based on the input value from the first input device; and
determine an orientation of the second virtual camera based on the input value from the second input device.
3 . The image processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
output the first virtual viewpoint video and the second virtual viewpoint video to the first input device; and
output the second virtual viewpoint video to the second input device.
4 . The image processing apparatus according to claim 3 , wherein the one or more processors further execute the instructions to:
switch the virtual viewpoint image to be output to the second input device from the second virtual viewpoint video to the first virtual viewpoint video based on an instruction from the first input device.
5 . The image processing apparatus according to claim 3 , wherein the one or more processors further execute the instructions to;
change the parameter of the second virtual camera to the parameter of the first virtual camera based on an instruction from the first input device.
6 . The image processing apparatus according to claim 3 , wherein the one or more processors further execute the instructions to:
exchange the parameter of the first virtual camera and the parameter of the second virtual camera based on an instruction from the first input device.
7 . The image processing apparatus according to claim 3 , wherein the one or more processors further execute the instructions to:
change playback speeds of the first virtual viewpoint video and the second virtual viewpoint video based on an instruction from the first input device.
8 . The image processing apparatus according to claim 3 , wherein the one or more processors further execute the instructions to:
generate a third virtual viewpoint video from the plurality of images, the third virtual viewpoint video providing an overhead view of a captured space based on the plurality of images and the playback position; and
output the third virtual viewpoint video to the first input device.
9 . The image processing apparatus according to claim 8 , wherein the third virtual viewpoint video includes an icon representing a position of the first virtual camera and an icon representing a position of the second virtual camera.
10 . The image processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
determine a position and an orientation of the first virtual camera based on the input value from the first input device; and
determine an orientation and a height of a position of the second virtual camera based on the input value from the second input device; and
determine another position of the second virtual camera based on the input value from the first input device.
11 . The image processing apparatus according to claim 1 ,
wherein the first input device comprises a tablet terminal, and
wherein the second input device comprises a head-mount display.
12 . A method for an image processing apparatus, the method comprising:
obtaining a plurality of images captured by a plurality of image capturing apparatuses;
generating a camera parameter of a first virtual camera based on an input value from a first input device;
determining a playback position of a virtual viewpoint image to be played back based on the input value from the first input device;
generating a first virtual viewpoint video based on the plurality of images, the camera parameter of the first virtual camera, and the playback position;
generating a camera parameter of a second virtual camera based on the input value from the first input device and an input value from a second input device; and
generating a second virtual viewpoint video based on the plurality of images, the camera parameter of the second virtual camera, and the playback position.
13 . A non-transitory storage medium storing a program for causing an image processing apparatus to execute a control method, the method comprising:
obtaining a plurality of images captured by a plurality of image capturing apparatuses;
generating a camera parameter of a first virtual camera based on an input value from a first input device;
determining a playback position of a virtual viewpoint image to be played back based on the input value from the first input device;
generating a first virtual viewpoint video based on the plurality of images, the camera parameter of the first virtual camera, and the playback position;
generating a camera parameter of a second virtual camera based on the input value from the first input device and an input value from a second input device; and
generating a second virtual viewpoint video based on the plurality of images, the camera parameter of the second virtual camera, and the playback position.