DETERMINATION SYSTEM, DETERMINATION METHOD AND PROGRAM

1 : A determination system, comprising:

at least one processor; and

at least one memory device that stores a plurality of instructions, which when executed by the at least one processor, cause the at least one processor to:

estimation processing relating to the input data, based on an output produced when a plurality of elements included in input data, on which the estimation processing is to be performed, are input to a first machine learning model;

learning of a second machine learning model based on a part of the plurality of elements included in the input data and known result data of the input data; and

a determination validity of the first machine learning model based on an output of the second machine learning model after execution of the learning.

2 : The determination system according to claim 1 , wherein the at least one memory device that stores the plurality of instructions further causes the at least one processor to:

execute learning of a third machine learning model based on the plurality of elements included in the input data and the known result data of the input data, and

execute the estimation processing relating to new input data, when it is determined that the first machine learning model is not valid, based on an output produced when the plurality of elements included in the new input data are input to the third machine learning model after execution of the learning.

3 : The determination system according to claim 1 , wherein the at least one memory device that stores the plurality of instructions further causes the at least one processor to:

execute learning of a third machine learning model based on the plurality of elements included in the input data and the known result data of the input data, and

execute the estimation processing relating to new input data, when it is determined that the first machine learning model is not valid, based on an output produced when the plurality of elements included in the new input data are input to any one of the first machine learning model and the third machine learning model to be determined in accordance with the new input data.

4 : The determination system according to claim 1 , wherein the at least one memory device that stores the plurality of instructions further causes the at least one processor to:

identify a strength of a trend in the output of the second machine learning model,

execute learning of a third machine learning model based on the plurality of elements included in the input data and the known result data of the input data, and

execute the estimation processing relating to new input data, when it is determined that the first machine learning model is not valid, based on a value obtained by adding a weighting corresponding to the strength of the trend to a value indicating an output produced when the plurality of elements included in the new input data are input to the third machine learning model after execution of the learning and a value indicating an output produced when the plurality of elements are input to the first machine learning model.

5 : The determination system according to claim 2 , wherein the at least one memory device that stores the plurality of instructions further causes the at least one processor to:

determine the determination validity of the first machine learning model based on a difference between an output of the third machine learning model after execution of the learning and an output of the second machine learning model produced when a part of the inputs to the third machine learning model is input to the second machine learning model.

6 : The determination system according to claim 1 , wherein the at least one memory device that stores the plurality of instructions further causes the at least one processor to:

execute the estimation processing relating to new input data, when it is determined that the first machine learning model is not valid, based on an output produced when a part of the plurality of elements included in the new input data is input to a fourth machine learning model.

7 : The determination system according to claim 1 , wherein the at least one memory device that stores the plurality of instructions further cause the at least one processor to:

determine the determination validity of the first machine learning model based on a difference between an output of the second machine learning model before execution of the learning and an output of the second machine learning model after execution of the learning.

8 : A determination method, comprising the steps of:

executing estimation processing relating to the input data, based on an output produced when a plurality of elements included in input data on which the estimation processing is to be performed are input to a first machine learning model;

executing learning of a second machine learning model based on a part of the plurality of elements included in the input data and known result data of the input data, and

determining a determination validity of the first machine learning model based on an output of the second machine learning model after execution of the learning.

9 : A non-transitory computer readable information storage medium storing a program which is to be executed by a computer to execute the procedures of:

executing estimation processing relating to the input data, based on an output produced when a plurality of elements included in input data on which the estimation processing is to be performed are input to a first machine learning model;

executing learning of a second machine learning model based on a part of the plurality of elements included in the input data and known result data of the input data; and

determining a determination validity of the first machine learning model based on an output of the second machine learning model after execution of the learning.