Action analysis device, action analysis method, and action analysis program

1. An action analysis devices comprising: a memory storing instructions, a microphone, a video camera, and one or more processors configured to execute the instructions to perform operations for action analysis, the operations including:

analyzing input acoustic information obtained from the microphone, and generating acoustic analysis information indicating a feature of the acoustic information;

determining a time difference between when an acoustic event identified by the acoustic analysis information occurs and when an event corresponding to the acoustic event occurs in input video obtained by capturing an image of a crowd with the video camera, wherein the video camera is located separate from the microphone;

analyzing an action of the crowd corresponding to the acoustic event based on the input video, the acoustic analysis information, and the time difference, and

outputting result of said crowd action analysis;

wherein determining the time difference includes determining the time difference based on time difference modeling information, the time difference modeling information being information for modeling a time difference between when an acoustic event occurs and when a video event occurs based on a distance between a position where the microphone is installed and a position monitored by the camera.

2. The action analysis device according to claim 1 ,

wherein determining the time difference further includes:

generating time difference information that is information indicating a distribution of the determined time difference, and

wherein analyzing the action of the crowd corresponding to the acoustic event includes analyzing based on the input video, the acoustic analysis information, and the time difference information.

3. The action analysis device according to claim 1 ,

wherein analyzing the action of the crowd includes analyzing based on the input video obtained after a time equivalent to the time difference has elapsed from the occurrence of the acoustic event.

4. The action analysis device according to claim 1 ,

wherein analyzing the action of the crowd includes:

calculating a probability of occurrence of an event based on the acoustic analysis information;

calculating a probability of occurrence of the event based on the input video obtained after a time equivalent to the time difference elapsed from the occurrence of the acoustic event;

determining whether or not the action of the crowd is abnormal based on the calculated probabilities.

5. The action analysis device according to claim 1 ,

wherein determining the time difference between when the acoustic event occurs and when the event corresponding to the acoustic event occurs in the input video includes determining based on a distance between a position where the acoustic information is acquired and an imaging area captured in the input video.

6. The action analysis device according to claim 1 ,

wherein determining the time difference includes determining based on a distance between a position where the acoustic information is acquired and an imaging area captured in the input video and an acoustic feature indicated by the acoustic analysis information.

7. The action analysis device according to claim 1 ,

wherein analyzing the action of the crowd includes:

classifying an event type of the action of the crowd based on the acoustic analysis information;

generating event classification information indicating a result of the classification; and

performing, based on the event classification information, at least one of a process of adjusting a parameter used for analyzing the action of the crowd, or a process of switching an algorithm used for analyzing the action of the crowd.

8. The action analysis device according to claim 7 ,

wherein analyzing the action of the crowd includes calculating a likelihood of a specific event as the event classification information.

9. The action analysis device according to claim 1 , wherein:

analyzing the action of the crowd further includes obtaining a result of analyzing the action of the crowd,

outputting the result of the crowd action analysis includes outputting the result of the crowd action analysis to a predetermined device to execute a predetermined operation.

10. An action analysis method, comprising:

analyzing input acoustic information obtained from a microphone, and generating acoustic analysis information indicating a feature of the acoustic information;

determining a time difference between when an acoustic event identified by the acoustic analysis information occurs and when an event corresponding to the acoustic event occurs in input video obtained by capturing an image of a crowd with a video camera, wherein the video camera is located separate from the microphone;

analyzing an action of the crowd corresponding to the acoustic event, using the input video, the acoustic analysis information, and the time difference, and

outputting result of said crowd action analysis;

wherein determining the time difference includes determining the time difference based on time difference modeling information, the time difference modeling information being information for modeling a time difference between when an acoustic event occurs and when a video event occurs based on a distance between a position where the microphone is installed and a position monitored by the camera.

11. A non-transitory computer-readable medium storing an action analysis program which, when executed by one or more processors, cause the one or more processors to perform operations for action analysis, the operations comprising:

analyzing input acoustic information obtained from a microphone, and generating acoustic analysis information indicating a feature of the acoustic information;

determining a time difference between when an acoustic event identified by the acoustic analysis information occurs and when an event corresponding to the acoustic event occurs in input video obtained by capturing an image of a crowd with a video camera, wherein the video camera is located separate from the microphone;

analyzing an action of the crowd corresponding to the acoustic event based on the input video, the acoustic analysis information, and the time difference, and

outputting result of said crowd action analysis;

wherein determining the time difference includes determining the time difference based on time difference modeling information, the time difference modeling information being information for modeling a time difference between when an acoustic event occurs and when a video event occurs based on a distance between a position where the microphone is installed and a position monitored by the camera.