Information processing apparatus for selecting a camera to be used to generate a virtual viewpoint video from images shot by a plurality of cameras

1. An image processing apparatus for generating virtual viewpoint video including a plurality of frames at a virtual viewpoint by using a plurality of real images at a plurality of real viewpoints obtained by a plurality of cameras comprising:

a memory;

a processor coupled to the memory which executes the following:

acquiring virtual viewpoint information given as a position and an orientation of the virtual viewpoint;

selecting a camera for shooting an image to be used in generating an image at a virtual viewpoint in a frame from a plurality of cameras with respect to each of the plurality of the frames based on a positional relationship between the virtual viewpoint information of each frame and viewpoint information of the plurality of the cameras; and

reselecting the camera selected for each of the plurality of the frames,

wherein, if a camera selected for a target frame differs from a camera selected for a preceding frame of the target frame, and if a rate of covering an image at the virtual viewpoint of the target frame with a shot image of the camera selected for the preceding frame is greater than a predetermined value, the reselecting changes the camera selected for the target frame to the camera selected for the preceding frame.

2. The image processing apparatus according to claim 1 , wherein selecting a camera whose position and orientation at a viewpoint thereof is closest to a position and an orientation of the virtual viewpoint of the frame from among the cameras whose rates of covering the image at the virtual viewpoint of the target frame with the shot image of the camera are greater than the predetermined value in the plurality of the cameras selected for the preceding frame.

3. The image processing apparatus according to claim 1 , wherein reselecting changes the camera selected for the target frame such that the number of times the same camera is consecutively selected is less than a predetermined value.

4. A method for image processing for generating virtual viewpoint video including a plurality of frames at a virtual viewpoint by using a plurality of real images at a plurality of real viewpoints obtained by a plurality of cameras, the method comprising:

acquiring virtual viewpoint information given as a position and an orientation of the virtual viewpoint;

selecting a camera for shooting an image to be used in generating an image at a virtual viewpoint in a frame from a plurality of cameras with respect to each of the plurality of the frames based on a positional relationship between the virtual viewpoint information of each frame and viewpoint information of the plurality of the cameras; and

reselecting the camera selected for each of the plurality of the frames,

wherein, if a camera selected for a target frame differs from a camera selected for a preceding frame of the target frame, and if a rate of covering an image at the virtual viewpoint of the target frame with a shot image of the camera selected for the preceding frame is greater than a predetermined value, the camera selected for the target frame is changed to the camera selected for the preceding frame in the reselection.

5. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method for image processing for generating virtual viewpoint video including a plurality of frames at a virtual viewpoint by using a plurality of real images at a plurality of real viewpoints obtained by a plurality of cameras, the method comprising:

acquiring virtual viewpoint information given as a position and an orientation of the virtual viewpoint;

selecting a camera for shooting an image to be used in generating an image at a virtual viewpoint in a frame from a plurality of cameras with respect to each of the plurality of the frames based on a positional relationship between the virtual viewpoint information of each frame and viewpoint information of the plurality of the cameras; and

reselecting the camera selected for each of the plurality of the frames,

wherein, if a camera selected for a target frame differs from a camera selected for a preceding frame of the target frame, and if a rate of covering an image at the virtual viewpoint of the target frame with a shot image of the camera selected for the preceding frame is greater than a predetermined value, the camera selected for the target frame is changed to the camera selected for the preceding frame in the reselection.