Retrieval device, retrieval method, and computer program product

1. A retrieval device comprising:

a memory; and

one or more processors configured to:

acquire trajectory information indicating a movement trajectory of a target in time-series images;

acquire situation information indicating a sound environment corresponding to the time-series images, the sound environment being recognized by analyzing sound contained in the time-series images or sound acquired independently of the time-series images;

acquire a retrieval query from an input device comprising a microphone, the retrieval query containing a movement trajectory and a peripheral situation, the retrieval query concerning the peripheral situation being input via a sound from the microphone; and

retrieve an image matching with the movement trajectory contained in the retrieval query and matching with the peripheral situation contained in the retrieval query among images contained in the time-series images, wherein

the peripheral situation is matched by comparing the sound input of the retrieval query and the sound contained in the time-series images with each other.

2. The retrieval device according to claim 1 , wherein the one or more processors retrieves first images among the images contained in the time-series images, and retrieves, from the first images, an image of which the trajectory information matches with the movement trajectory contained in the retrieval query, pieces of the situation information of the first images matching with the peripheral situation contained in the retrieval query.

3. The retrieval device according to claim 2 , wherein

the situation information includes information indicating that the peripheral situation of the target is unknown, and

the one or more processors retrieves, from the first images and at least one of images before or after each of the first images, an image of which the trajectory information matches with the movement trajectory contained in the retrieval query, the situation information of the at least one of images indicating that the peripheral situation of the target is unknown.

4. The retrieval device according to claim 1 , wherein

the one or more processors further configured to

detect a position of the target from the time-series images; and

estimate the movement trajectory based on the detected position, and

the one or more processors acquires the trajectory information indicating the estimated movement trajectory.

5. The retrieval device according to claim 1 , wherein

the one or more processors further configured to estimate the peripheral situation from the time-series images, and

the one or more processors acquires the situation information indicating the estimated peripheral situation.

6. The retrieval device according to claim 5 , wherein when a peripheral situation of a second image contained in the time-series images is unknown, the one or more processors estimates that a peripheral situation of the second image is the peripheral situation of at least one of images before or after the second image.

7. The retrieval device according to claim 1 , wherein the target includes at least one of a person, a gathering of a plurality of persons, or a region in which a plurality of persons gather.

8. A retrieval method comprising:

acquiring trajectory information indicating a movement trajectory of a target in tin series images;

acquiring situation information indicating a peripheral situation of the target in the time-series images, the peripheral situation indicating a sound environment corresponding to the time-series images; the sound environment being recognized by analyzing sound contained in the time-series images or sound acquired independently of the time-series images;

acquiring a retrieval query from an input device comprising a microphone, the retrieval query containing a movement trajectory and a peripheral situation, the retrieval query concerning the peripheral situation being input via a sound from the microphone; and

retrieving an image matching with the movement trajectory contained in the retrieval query and matching with the peripheral situation contained in the retrieval query among images contained in the time-series images, wherein

the peripheral situation is matched by comparing the sound input of the retrieval query and the sound contained in the time-series images with each other.

9. The retrieval method according to claim 8 , wherein the retrieving includes retrieving first images among the images contained in the time-series images, and retrieving, from the first images, an image of which the trajectory information matches with the movement trajectory contained in the retrieval query, pieces of the situation information of the first images matching with the peripheral situation contained in the retrieval query.

10. The retrieval method according to claim 9 , wherein

the situation information includes information indicating that the peripheral situation of the target is unknown, and

the retrieving includes retrieving, from the first images and at least one of images before or after each of the first images, an image of which the trajectory information matches with the movement trajectory contained in the retrieval query, the situation information of the at least one of images indicating that the peripheral situation of the target is unknown.

11. The retrieval method according to claim 8 , further comprising:

detecting a position of the target from the time-series images; and

estimating the movement trajectory based on the detected position, wherein

the trajectory information indicating the estimated movement trajectory is acquired.

12. The retrieval method according to claim 8 , further cat yrising:

estimating the peripheral situation from the time-series images, wherein

the situation information indicating the estimated peripheral situation is acquired.

13. The retrieval method according to claim 12 , wherein the estimating includes, when a peripheral situation of a second image contained in the time-series images is unknown, estimating that a peripheral situation of the second image is the peripheral situation at least one of images before or after of the second image.

14. The retrieval method according to claim 8 , wherein the target includes at least one of a person, a gathering of a plurality of persons, or a region in which a plurality of persons gather.

15. A computer program product having a non-transitory computer readable medium comprising instructions that cause a computer to execute:

acquiring trajectory information indicating a movement trajectory of a target in time-series images;

acquiring situation information indicating a peripheral situation of the target in the time-series images, the peripheral situation indicating a sound environment corresponding to the time-series images, the sound environment being recognized by analyzing sound contained in the time-series images or sound acquired independently of the time-series images;

acquiring a retrieval query from an input device comprising a microphone, the retrieval query containing a movement trajectory and a peripheral situation, the retrieval query concerning the peripheral situation being input via a sound from the microphone; and

retrieving an image matching with the movement trajectory contained in the retrieval query and matching with the peripheral situation contained in the retrieval query among images contained in the time-series images, wherein

the peripheral situation is matched by comparing the sound input of the retrieval query and the sound contained in the time-series images with each other.

16. The computer program product according to claim 15 , wherein the retrieving includes retrieving first images among the images contained in the time-series images, and retrieves from the first images, an image of which the trajectory information matches with the movement trajectory contained in the retrieval query, pieces of the situation information of the first images matching with the peripheral situation contained in the retrieval query.