Image search system, image search method and storage medium
The image search system according to the present invention accumulates virtual viewpoint video image data generated based on image data obtained by capturing an object from a plurality of directions by a plurality of cameras and a virtual viewpoint parameter used for generation of the virtual viewpoint video image data in association with each other. Then, the image search system extracts, in a case where a search condition is input via an input unit, virtual viewpoint video image data associated with a virtual viewpoint parameter corresponding to the search condition from the accumulated virtual viewpoint video image data. Further, the image search system presents information of the extracted virtual viewpoint video image data as results of the search. Due to this, convenience relating to a search for a virtual viewpoint video image improves.
1. An image search system comprising:
one or more hardware processors; and
one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the one or more programs including instructions for:
receiving a search instruction, which includes information representing a virtual viewpoint parameter including at least one of a position of a virtual viewpoint, an orientation of view from the virtual viewpoint, and a time of a virtual viewpoint image, by a user;
specifying, among a plurality of virtual viewpoint images, a virtual viewpoint image corresponding to the virtual viewpoint parameter which is specified based on the received search instruction, wherein a difference between a virtual viewpoint parameter associated with the specified virtual viewpoint image and a virtual viewpoint parameter represented by the information is smaller than a threshold value; and
causing a display device to display the specified virtual viewpoint image.
2. The image search system according to claim 1 , wherein
the virtual viewpoint image includes a plurality of frames, and
each frame included in the virtual viewpoint image is associated with the virtual viewpoint parameter.
3. The image search system according to claim 2 , wherein
a part or all of the plurality of frames are associated with time information corresponding to a same time.
4. The image search system according to claim 2 , wherein
the search instruction includes information representing the virtual viewpoint parameter, and a difference between the virtual viewpoint parameter specified based on the information and a typical value based on the virtual viewpoint parameter associated with each frame included in the specified virtual viewpoint image is smaller than a threshold value.
5. The image search system according to claim 4 , wherein
the typical value is a mean or a median of a value of a virtual viewpoint parameter associated with each frame included in the virtual viewpoint image.
6. The image search system according to claim 2 , wherein
a virtual viewpoint image which includes, as a first frame among a plurality of frames, a frame associated with the virtual viewpoint parameter specified based on the received search instruction is specified.
7. The image search system according to claim 2 , wherein
a virtual viewpoint image which includes, as a last frame among a plurality of frames, a frame associated with the virtual viewpoint parameter specified based on the received search instruction is specified.
8. The image search system according to claim 2 , wherein
the search instruction includes information representing a plurality of virtual viewpoint parameters, and
a virtual viewpoint image including a frame associated with at least one of a plurality of virtual viewpoint parameters specified based on the received search instruction is specified.
9. The image search system according to claim 2 , wherein
in a case where the virtual viewpoint image is displayed, information indicating a virtual viewpoint parameter associated with each frame included in the displayed virtual viewpoint image is further displayed.
10. The image search system according to claim 1 , wherein
the search instruction includes information to request a frequently viewed virtual viewpoint image, and
a virtual viewpoint image associated with a virtual viewpoint parameter whose frequency of viewing is higher than a threshold value is specified based on the information.
11. An image search method performed by an image search system including one or more hardware processors and one or more memories storing one or more programs configured to be executed by the one or more hardware processors, the method comprising:
receiving a search instruction, which includes information representing a virtual viewpoint parameter including at least one of a position of a virtual viewpoint, an orientation of view from the virtual viewpoint, and a time of a virtual viewpoint image, by a user;
specifying, among a plurality of virtual viewpoint images, a virtual viewpoint image corresponding to the virtual viewpoint parameter which is specified based on the received search instruction, wherein a difference between a virtual viewpoint parameter associated with the specified virtual viewpoint image and a virtual viewpoint parameter represented by the information is smaller than a threshold value; and
causing a display device to display the specified virtual viewpoint image.
12. A non-transitory computer readable storage medium storing one or more programs for causing one or more hardware processors to perform an image search method, the method comprising:
receiving a search instruction, which includes information representing a virtual viewpoint parameter including at least one of a position of a virtual viewpoint, an orientation of view from the virtual viewpoint, and a time of a virtual viewpoint image, by a user;
specifying, among a plurality of virtual viewpoint images, a virtual viewpoint image corresponding to the virtual viewpoint parameter which is specified based on the received search instruction, wherein a difference between a virtual viewpoint parameter associated with the specified virtual viewpoint image and a virtual viewpoint parameter represented by the information is smaller than a threshold value; and
causing a display device to display the specified virtual viewpoint image.
13. The image search system according to claim 1 , wherein
a time included in the virtual viewpoint parameter is determined based on a time at which a plurality of image capturing devices capture images used for generating a virtual viewpoint image.
14. The image search system according to claim 2 , wherein
the search instruction is received for one or more frames included in the virtual viewpoint image, and
a virtual viewpoint image corresponding to the position of the virtual viewpoint and the orientation of view from the virtual viewpoint which are specified based on the received search instruction is specified, and a time associated with the one or more frames for which the search instruction is received is specified.
15. The image search system according to claim 1 , wherein
a difference between the virtual viewpoint parameter associated with the specified virtual viewpoint image and the virtual viewpoint parameter represented by the information is smaller than a difference between a virtual viewpoint parameter associated with another virtual viewpoint image among the plurality of virtual viewpoint images and the virtual viewpoint parameter represented by the information.