Method and device for analyzing battlefield threat based on dynamic graph neural network

1 . A computer-implemented method performed by a battlefield threat analysis device comprising an input/output unit, a memory storing a battlefield threat analysis program, and a processor configured to execute the program for analyzing a battlefield threat, the method comprising:

receiving, via the input/output unit, battlefield information including object information and edge information for each of a plurality of objects in a battlefield from an external device, and by a data generation unit executed by the processor, generating a plurality of time point-specific graph data for the plurality of objects from past to a current time point;

calculating, for each object, a plurality of time point-specific feature values and a plurality of time point-specific latent vectors, corresponding to the plurality of time point-specific feature values, by a latent vector calculation unit comprising a plurality of graph neural network (GNN) modules and a plurality of corresponding recurrent neural network (RNN) modules, the calculating including:

i) generating, by the GNN modules, the time point-specific feature values from the plurality of time point-specific graph data of each object, and

ii) generating, by the RNN modules, the time point-specific latent vectors for each object based on the corresponding feature values and one or more past time point latent vectors using a previously learned neural network model,

wherein calculating the latent vectors includes calculating a final latent vector for each object by applying a preset weight to each of the plurality of time point-specific latent vectors of each object, and

wherein calculating the final latent vector includes:

calculating a sum of the plurality of time point-specific latent vectors to which the preset weight is applied, and

calculating the final latent vector for each object by adding the current time point latent vector among the plurality of time point-specific latent vectors and the sum of time point-specific latent vectors;

classifying by a classification unit of the neural network model, a predicted behavior of each of the plurality of objects at a specific time point, including a future time point, based on the corresponding final latent vectors, and predicting, by the classification unit, a threat occurrence probability between the plurality of objects at the future time point based on the classified behaviors; and

outputting, via the input/output unit, a battlefield threat analysis result comprising the behavior classification and the predicted threat occurrence probability to a user terminal for providing battlefield prediction information to a battlefield commander and facilitating convenience in establishing operational plans.

2 . The method of claim 1 , wherein calculating the latent vector includes:

calculating a current time point feature value for each object from current time point graph data among the plurality of time point-specific graph data of each object; and

calculating a current time point latent vector for each object from the current time point feature value based on one or more past time point latent vectors among the plurality of time point-specific latent vectors of each object.

3 . The method of claim 1 , wherein the final latent vector includes an object vector for each object and an edge vector according to connection between each object and adjacent objects.

4 . The method of claim 1 , wherein the neural network model includes a latent vector calculation unit and a classification unit,

wherein the latent vector calculation unit is learned to calculate and output the plurality of time point-specific latent vectors for each of the plurality of objects in response to receiving the plurality of time point-specific graph data for each of the plurality of objects, and

wherein the classification unit is learned to classify and output a behavior for each of the plurality of objects in response to receiving a classification correct answer as label data along with the plurality of time point-specific latent vectors of each of the plurality of objects output from the latent vector calculation unit.

5 . The method of claim 4 , wherein the classification unit generates a loss value by comparing the classification correct answer input as the label data with an actual output classification result, and

wherein the latent vector calculation unit is learned to further receive the loss value and calculate and output the plurality of time point-specific latent vectors for each of the plurality of objects from the plurality of time point-specific graph data.

6 . A non-transitory computer-readable recording medium storing instructions, when executed by one or more processors, that cause the one or more processors to perform the method of claim 1 .

7 . The method of claim 1 , wherein the preset weight applied to the plurality of time point-specific latent vectors is determined by an attention module trained as part of the previously learned neural network model.

8 . The method of claim 1 , wherein the classification unit determines a classification loss value by comparing the predicted behavior with a classification correct answer for the specific time point, and provides the classification loss value to the latent vector calculation unit and the classification unit for further learning to minimize the classification loss value.

9 . A device for analyzing a battlefield threat, the device comprising:

an input/output unit;

a memory configured to store a battlefield threat analysis program; and

one or more processors configured to execute the battlefield threat analysis program to:

receive, via the input/output unit, battlefield information including object information and edge information for each of a plurality of objects in a battlefield from an external device, and by a data generation unit executed by the one or more processors, generating a plurality of time point-specific graph data for the plurality of objects from past to a current time point;

calculate, for each object, a plurality of time point-specific feature values and a plurality of time point-specific latent vectors, corresponding to the plurality of time point-specific feature values, by a latent vector calculation unit comprising a plurality of graph neural network (GNN) modules and a plurality of corresponding recurrent neural network (RNN) modules, by:

i) generating, by the GNN modules, the time point-specific feature values from the plurality of time point-specific graph data of each object, and

ii) generating, by the RNN modules, the time point-specific latent vectors for each object based on the corresponding feature values and one or more past time point latent vectors, using a previously learned neural network model,

wherein in calculating the latent vectors, the one or more processors are configured to calculate a final latent vector for each object by applying a preset weight to each of the plurality of time point-specific latent vectors of each object, and

wherein in calculating the final latent vector, the one or more processors are configured to:

calculate a sum of the plurality of time point-specific latent vectors to which the preset weight is applied, and

calculate the final latent vector for each object by adding the current time point latent vector among the plurality of time point-specific latent vectors and the sum of time point-specific latent vectors;

classify by a classification unit of the neural network model, a predicted behavior of each of the plurality of objects at a specific time point, including a future time point, based on the corresponding final latent vectors, and predicting, by the classification unit, a threat occurrence probability between the plurality of objects at the future time point based on the classified behaviors; and

output, via the input/output unit, a battlefield threat analysis result comprising the behavior classification and the predicted threat occurrence probability to a user terminal for providing battlefield prediction information to a battlefield commander and facilitating convenience in establishing operational plans.

10 . The device of claim 9 , wherein at least one of the one or more processors is configured to:

calculate a current time point feature value for each object from current time point graph data among the plurality of time point-specific graph data of each object, and

calculate a current time point latent vector for each object from the current time point feature value based on one or more past time point latent vectors among the plurality of time point-specific latent vectors of each object.

11 . The device of claim 9 , wherein the final latent vector includes an object vector for each object and an edge vector according to connection between each object and adjacent objects.

12 . The device of claim 9 , wherein the neural network model includes a latent vector calculation unit and a classification unit,

wherein the latent vector calculation unit is configured to be learned to calculate and output the plurality of time point-specific latent vectors for each of the plurality of objects in response to receiving the plurality of time point-specific graph data for each of the plurality of objects, and

wherein the classification unit is configured to be learned to classify and output a behavior for each of the plurality of objects in response to receiving a classification correct answer as label data along with the plurality of time point-specific latent vectors of each of the plurality of objects output from the latent vector calculation unit.

13 . The device of claim 12 , wherein the classification unit is configured to generate a loss value by comparing the classification correct answer input as the label data with an actual output classification result, and

wherein the latent vector calculation unit is configured to be learned to further receive the loss value and calculate and output the plurality of time point-specific latent vectors for each of the plurality of objects from the plurality of time point-specific graph data.