Data-creation assistance apparatus and data-creation assistance method
To efficiently verify and improve a robustness of a learning model for supervised machine learning. A data-creation assistance apparatus 100 includes: a storage device 101 configured to store a neural network model 110 and test data 120 ; and a computing device 104 configured to specify an uncertainty of an inference result acquired by the neural network model 110 ; acquire gradient information of the test data 120 by a back propagation process using the uncertainty as a loss; apply various minute changes to the test data 120 to generate a plurality of minutely changed test data, and calculate deviations between each of the plurality of pieces minutely changed test data and the test data 120 ; and specify, based on the uncertainty information, the gradient information, and the deviations, a minute change that increases or decreases the uncertainty.
1. A data-creation assistance apparatus comprising:
a storage device configured to store a neural network model used for supervised machine learning and test data attached with a label of ground truth; and
a computing device configured to execute
a process of specifying an uncertainty of an inference result from the neural network model by inputting the test data to the neural network model,
a process of acquiring gradient information of the test data by a back propagation process using the uncertainty as a loss,
a process of generating a plurality of minutely changed test data obtained by applying a various minute change to the test data, and calculating deviations between each of the plurality of minutely changed test data and the test data; and
a process of specifying, based on the uncertainty information, the gradient information, and the deviations, the minute change that increases or decreases the uncertainty or the minutely changed test data to which the minute change is applied;
wherein when executing the process of specifying the uncertainty, the computing device is further configured to:
execute a process of dropout inference, in which the test data is used as an input, an output of neurons included in the neural network model is randomly set to 0, and an inference result is output, a plurality of times,
acquire a plurality of dropout inference results, and
specify a variance related to the plurality of dropout inference results as the uncertainty.
2. The data-creation assistance apparatus according to claim 1 , wherein
the computing device is further configured to in response to
acquire the gradient information, calculate the gradient of the test data by the back propagation process in which the variance of the plurality of acquired dropout inference results is used as a loss function, and
in response to specify the minutely changed test data, compare the minutely changed test data with the gradient information, and specify minutely changed test data and a minute change vector that are most similar to the gradient indicated by the gradient information.
3. The data-creation assistance apparatus according to claim 1 , wherein
the computing device is further configured to
in the process of dropout inference, use the inference result up to an intermediate layer of the neural network.
4. The data-creation assistance apparatus according to claim 3 , wherein
the computing device is further configured to
acquire an inference result from each of the plurality of neural network models by inputting the test data to each of the plurality of neural network models, and specify a variance related to the inference result as the uncertainty.
5. The data-creation assistance apparatus according to claim 1 , wherein
the computing device is further configured to
execute a process of displaying a set of the minutely changed test data and uncertainty information of the minutely changed test data.
6. The data-creation assistance apparatus according to claim 1 , wherein
the computing device is further configured to
execute a process of displaying a set of the minutely changed test data and uncertainty information of the minutely changed test data, and displaying that the uncertainty information is above or below a predetermined threshold.
7. The data-creation assistance apparatus according to claim 1 , wherein
the storage device is configured to
store a plurality of training data used for supervised machine learning together with the label of ground truth, and
the computing device is further configured to
execute a relearning process of the neural network model by using the plurality of training data as an input, using minute change vector included in the minutely changed test data to generate minutely changed training data obtained by applying minute change to the training data, and giving the minutely changed training data to the neural network model.
8. The data-creation assistance apparatus according to claim 7 , wherein
the computing device is further configured to
execute a process of specifying the uncertainty by using the neural network model that has undergone the relearning and using the test data as the input, and displaying information indicating that the uncertainty is above or below a predetermined threshold, and
execute, when the uncertainty is above or below the predetermined threshold, the relearning process of the neural network model by giving the minutely changed training data to the neural network model.
9. A data-creation assistance method realized by an information processing apparatus that is configured to execute:
a process of storing a neural network model used for supervised machine learning and test data attached with a label of ground truth, inputting the test data into the neural network model, and specifying an uncertainty of an inference result from the neural network model;
a process of acquiring gradient information of the test data by a back propagation process using the uncertainty as a loss;
a process of giving various minute changes to the test data to generate a plurality of minutely changed test data, and calculating deviations between each of the plurality of minutely changed test data and the test data; and
a process of specifying, based on information of the uncertainty, the gradient information, and the deviations, a minute change that increases or decreases the uncertainty or minutely changed test data to which the minute change is applied;
wherein when specifying the uncertainty, the information processing apparatus is further configured to:
execute a process of dropout inference, in which the test data is used as an input, an output of neurons included in the neural network model is randomly set to 0, and an inference result is output, a plurality of times,
acquire a plurality of dropout inference results, and
specify a variance related to the plurality of dropout inference results as the uncertainty.