Event detection system using multi-modal generative artificial intelligence model
Provided is a system for detecting an event using a multimodal generative artificial intelligence (AI) model, the system including an edge device that analyzes videos, which are recorded by one or more cameras installed in a space to be monitored, in real time through one or more AI model parts and detects an event, and a server device that verifies an event detection result of the edge device on the basis of the event detection result transmitted by the edge device and preset specification information of the AI model parts, analyzes a response acquired through the multimodal generative AI model using a prompt that requests additional information for event detection in accordance with a verification result, and controls operations of the AI model parts of the edge device.
1 . A system for detecting an event using a multimodal generative artificial intelligence (AI) model, the system comprising:
an edge device configured to receive videos, which are recorded by one or more cameras installed in a space to be monitored, in real time to analyze the videos, include one or more AI model parts each of which detects a defined event on the basis of a set rule to detect the event, and transmit an event detection result including metainformation related to the detected event;
wherein the edge device comprises:
a video collector configured to receive and store the videos recorded in real time by the one or more cameras; and
a video analyzer including one or more first AI model parts for detecting objects in the recorded videos and detecting the defined event on the basis of the set rule, to detect the event and transit the event detection result including the metainformation related to the detected event; and
a server device configured to verify the event detection result of the edge device on the basis of the event detection result transmitted by the edge device and preset specification information of the AI model parts included in the edge device, generate a prompt that requests additional information for event detection in accordance with a verification result, transmit the prompt, analyze a response acquired from the multimodal generative AI model, and control operations of the AI model parts of the edge device,
wherein the server device comprises:
a prompt generator configured to request that the event detection result of the edge device be verified on the basis of the event detection result transmitted by the edge device and the preset specification information of the AI model parts included in the edge device and generate the prompt that requests additional information from the edge device for event detection in accordance with the verification result;
a generative AI model interoperation part configured to interoperate with the multimodal generative AI model, transmit the generated prompt, and acquire the response;
a response analyzer configured to check accuracy of the event detection result of the edge device by analyzing the response acquired from the multimodal generative AI model and select an AI model part which performs an additional information request included in the response on the basis of the preset specification information of the AI model parts included in the edge device; and
a model controller configured to transmit control information for controlling an operation of the selected AI model part such that the AI model part acquires the requested additional information.
2 . The system of claim 1 , wherein the edge device further comprises a model setting part configured to set and control whether to operate the AI model parts in accordance with performance of the edge device and the control information of the server device.
3 . The system of claim 2 , wherein the model setting part performs control such that some of the AI model parts included in the video analyzer are in a standby state.
4 . The system of claim 3 , wherein the model setting part performs control in accordance with the control information received from the server device such that the AI model parts in the standby state operate to acquire the requested additional information.
5 . The system of claim 4 , wherein the model setting part operates the AI model parts in the standby state in accordance with the control information received from the server device, and, when it is determined that performance of the edge device is insufficient, performs control such that other AI model parts in an operational state are switched to the standby state.
6 . The system of claim 1 , wherein the video analyzer further includes one or more second AI model parts configured to extract attributes related to the objects detected in the recorded videos.
7 . The system of claim 1 , wherein the video analyzer further includes one or more third AI model parts configured to track a designated one of the objects detected in the recorded videos.
8 . The system of claim 1 , wherein the edge device further comprises a statistics calculator configured to calculate statistical information related to the objects detected by the AI model parts, and
the metainformation which is related to the event and included in the event detection result includes the statistical information.
9 . The system of claim 8 , wherein the video analyzer further includes one or more fourth AI model parts configured to detect the event on the basis of the statistical information.
10 . The system of claim 1 , wherein the edge device and the server device are configured as one device.