Model training apparatus, model training method, and program for retraining anomaly detection model
An apparatus for training a model includes a storage unit configured to store a parameter of the model trained by using a training data set, and the training data set, a detector configured to use the model to determine whether an anomaly is present in a test data set and store a determined result and the test data set in the storage unit, and a retraining unit configured to retrain the model by using the determined result, the test data set, and the training data set.
1 . An apparatus for training a model comprising:
a processor; and
a memory storing program instructions that cause the processor to:
store a parameter of the model trained by using a training data set, and the training data set;
receive data from a network;
obtain an output from the model to determine whether an anomaly in the network is present in a test data set based on the obtained output and store a determined result and the test data set;
retrain the model by using the determined result, the test data set, and the training data set; and
subsequently receive data from the network, and obtain an output from the retrained model to determine whether an anomaly in the network is present in the subsequently received data in real-time,
wherein the processor retrains the model by using an element of the test data set in response to determining that the element of the test data set is incorrectly determined as the anomaly and determining that a distance of the element of the test data set with respect to the training data set is less than a threshold to prevent overlearning.
2 . The apparatus for training the model as claimed in claim 1 , wherein the processor uses the model to determine a class of each element of the test data set, and the processor determines whether the anomaly is present in the test data set based on the determined class.
3 . The apparatus for training the model as claimed in claim 1 , wherein the processor retrains the model by using a correct answer related to whether the anomaly is present in the test data set so that an element of the test data set that has been incorrectly determined is correctly determined.
4 . The apparatus for training the model as claimed in claim 1 , wherein the processor is configured to determine whether to adopt a new parameter of the model to be used by the processor based on accuracy of anomaly detection performed with respect to validation data by using the new parameter after the processor has obtained the new parameter.
5 . The apparatus for training the model as claimed in claim 1 , wherein the data received from the network includes network traffic data.
6 . The apparatus for training the model as claimed in claim 1 , wherein the output from the model is obtained to determine whether the anomaly in a temporal trend of an increase or decrease in a particular type of data in the network is present in the test data set.
7 . The apparatus for training the model as claimed in claim 1 , wherein the distance is a Euclidean distance on a low-dimensional space to which the test data set and the training data set are mapped.
8 . A model training method comprising:
receiving data from a network;
obtaining an output from a model to determine whether an anomaly in the network is present in a test data set based on the obtained output and storing a determined result and the test data set, the model being trained by using a training data set;
retraining the model by using the determined result, the test data set, and the training data set;
f subsequently receiving data from the network, and obtain an output from the retrained model to determine whether an anomaly in the network is present in the subsequently received data in real-time,
wherein the retraining of the model includes retraining the model by using an element of the test data set in response to determining that the element of the test data set is incorrectly determined as the anomaly and determining that a distance of the element of the test data set with respect to the training data set is less than a threshold to prevent overlearning.
9 . A non-transitory computer-readable recording medium having a program for causing a computer to perform a process comprising:
receiving data from a network;
obtaining an output from a model to determine whether an anomaly in the network is present in a test data set based on the obtained output and storing a determined result and the test data set, the model being trained by using a training data set;
retraining the model by using the determined result, the test data set, and the training data set;
f subsequently receiving data from the network, and obtain an output from the retrained model to determine whether an anomaly in the network is present in the subsequently received data in real-time,
wherein the retraining of the model includes retraining the model by using an element of the test data set in response to determining that the element of the test data set is incorrectly determined as the anomaly and determining that a distance of the element of the test data set with respect to the training data set is less than a threshold to prevent overlearning.