Determination of a machine learning model to be used for a given purpose related to a communication system
It is enabled that accurate determination of a machine learning model suitable for a communication system from among a plurality of machine learning models to be used for a given prediction purpose related to the communication system. An AI ( 70 ) inputs, to each of a plurality of trained machine learning models, input data corresponding to the machine learning model, and acquires a predicted value as an output of the machine learning model. The AI ( 70 ) evaluates an accuracy of a prediction related to the prediction purpose by the machine learning model based on the acquired predicted value and, of test data, a part indicating an actual result value corresponding to the predicted value. The AI ( 70 ) determines at least one machine learning model among the plurality of trained machine learning models based on a result of the evaluation of the accuracy.
1 . A model determination system, comprising one or more processors, the model determination system causing at least one of the one or more processors to execute:
an actual data acquisition process of acquiring actual data configured to indicate a time series of actual result values of a plurality of types of performance index values related to a communication system;
a predicted value acquisition process of inputting, to each of a plurality of trained machine learning models, input data configured to indicate at least one actual result value of a performance index value corresponding to a machine learning model at least one time point before a first time point, and acquiring a predicted value for a second time point that is after the first time point, at least one type of performance index value corresponding to each of the plurality of trained machine learning models being different from each other, the input data being a part of the actual data acquired by the actual data acquisition process, and the predicted value being acquired as an output of the machine learning model;
a prediction accuracy evaluation process of evaluating, for each of the plurality of trained machine learning models, an accuracy by the machine learning model based on the predicted value at the second time point and an actual result value at the second time point of the actual data acquired by the actual data acquisition process; and
a model determination process of determining at least one machine learning model usable for a prediction at a third time point that is after the second time point among the plurality of trained machine learning models based on a result of evaluating the accuracy.
2 . The model determination system according to claim 1 ,
wherein the machine learning model is configured to output the predicted value of at least one of the plurality of types of performance index values, and
wherein a type of the actual result value at least one time point before the first time point input to the machine learning model and a type of the predicted value output by the machine learning model are different.
3 . The model determination system according to claim 1 ,
wherein the machine learning model is configured to output the predicted value of at least one of the plurality of types of performance index values, and
wherein a type of the actual result value at least one time point before the first time point input to the machine learning model and a type of the predicted value output by the machine learning model are the same.
4 . The model determination system according to claim 1 , wherein the model determination system further causes the at least one of the one or more processors to execute a learning process of generating the plurality of trained machine learning models by executing learning configured to use data that is different from the actual data and that is configured to indicate the actual result values of the plurality of types of performance index values related to the communication system.
5 . The model determination system according to claim 1 ,
wherein the model determination system further causes the at least one of the one or more processors to execute:
a monitoring process of monitoring at least one type of performance index value related to the communication system; and
an additional performance index value type identification process of identifying, for each of the plurality of trained machine learning models, an additional performance index value type which is a type of performance index value which is required to be added to targets of the monitoring in order to use the machine learning model, and
wherein the model determination system causes the at least one of the one or more processors to execute the model determination process such that the at least one machine learning model usable for the prediction at the third time point is determined based on the result of evaluating the accuracy and the additional performance index value type.
6 . The model determination system according to claim 5 , wherein the model determination system further causes the at least one of the one or more processors to execute a monitoring target addition process of adding, to monitoring targets in the monitoring process, the performance index value of the additional performance index value type which is required to be added in order to use the at least one machine learning model usable for the prediction at the third time point that is determined.
7 . The model determination system according to claim 1 , wherein the model determination system is further configured to cause the at least one of the one or more processors to execute:
a monitoring process of monitoring at least one type of performance index value related to the communication system; and
a monitoring target addition process of adding, to monitoring targets in the monitoring process, a type of performance index value which is required to be added in order to use the at least one machine learning model usable for the prediction at the third time point that is determined.
8 . The model determination system according to claim 1 , wherein the model determination system further causes the at least one of the one or more processors to execute a prediction process of predicting the performance index value of the communication system by using the at least one machine learning model usable for the prediction at the third time point that is determined.
9 . The model determination system according to claim 1 , wherein the model determination system causes the at least one of the one or more processors to execute the model determination process such that, for each of a plurality of time slots, at least one machine learning model useable in prediction in a time slot of the plurality of time slots is determined.
10 . A computer-implemented model determination method, comprising:
acquiring an actual data configured to indicate a time series of actual result values of a plurality of types of performance index values related to a communication system;
inputting, to each of a plurality of trained machine learning models, input data configured to indicate at least one actual result value of a performance index value corresponding to a machine learning model at least one time point before a first time point and acquiring a predicted value for a second time point that is after the first time point, at least one type of performance index value corresponding to each of the plurality of trained machine learning models being different from each other, the input data being a part of the actual data, and the predicted value being acquired as an output of the machine learning model;
evaluating, for each of the plurality of trained machine learning models, an accuracy of a prediction by the machine learning model based on the predicted value at the second time point and an actual result value at the second time point of the actual data; and
determining at least one machine learning model usable for a prediction at a third time point that is later than the second time point among the plurality of trained machine learning models based on a result of evaluating the accuracy.