Bias adjustment device, information processing device, information processing method, and information processing program
A bias adjustment device uses an identification model developed by machine learning using training data, and includes a calculation unit that calculates first identification accuracy of the identification model trained on first training data and second identification accuracy of the identification model trained on second training data acquired by an adjustment of the number of pieces of data of the first training data, a prediction unit that predicts a change in identification accuracy with respect to the number of pieces of training data on the basis of the first identification accuracy and the second identification accuracy, and a control unit that adjusts the number of pieces of data used for the training or changes the identification model, on the basis of the predicted change in the identification accuracy, in such a manner that the predicted change in the identification accuracy becomes a predetermined target value.
1 . A bias adjustment device for an identification model by machine learning using training data, the bias adjustment device comprising:
processing circuitry configured to:
calculate first identification accuracy of the identification model trained on first training data for each of a plurality of defined bias categories and second identification accuracy of the identification model trained on second training data acquired by an adjustment of a number of pieces of data of the first training data for each of the plurality of defined bias categories;
predict a change in identification accuracy with respect to the number of pieces of the first training data on a basis of the first identification accuracy for each of the plurality of defined bias categories and the second identification accuracy for each of the plurality of defined bias categories;
determine a required number of pieces of data to achieve a predetermined target value based on the predicted change;
when the required number of pieces of data does not exceed a threshold, adjust a number of pieces of data used for training the identification model on a basis of the predicted change in the identification accuracy for each of the plurality of defined bias categories so that the predicted change in the identification accuracy becomes the predetermined target value;
when the required number of pieces of data does exceed the threshold, change a network structure of the identification model on the basis of the predicted change in the identification accuracy for each of the plurality of defined bias categories so that the predicted change in the identification accuracy becomes the predetermined target value; and
train the identification model using the adjusted number of pieces of data or the changed identification model to generate an updated identification model.
2 . The bias adjustment device according to claim 1 , wherein the processing circuitry is configured to cause the predicted change in the identification accuracy to be displayed in a graph or text.
3 . The bias adjustment device according to claim 1 , wherein the first training data is at least data related to race, gender, an address, income, or educational background.
4 . An information processing device comprising:
processing circuitry configured to:
acquire training data used for training of a model by machine learning;
generate accuracy information for each of a plurality of defined bias categories indicating accuracy of the model on a basis of first accuracy of a first model, which is the model trained on the training data, and second accuracy of a second model that is the model trained on adjusted data in which a number of pieces of data of the training data is adjusted according to each of the plurality of defined bias categories:
predict a change in accuracy with respect to the number of pieces of the training data on a basis of the accuracy information for each of the plurality of defined bias categories;
determine a required number of pieces of data to achieve a predetermined target value based on the predicted change;
when the required number of pieces of data does not exceed a threshold, adjust a number of pieces of data used for training the model on a basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value;
when the required number of pieces of data does exceed the threshold, change a network structure of the model on the basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value; and
train the model using the adjusted number of pieces of data or the changed model to generate an updated model.
5 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate the accuracy information for each of the plurality of defined bias categories on a basis of the second model trained on the adjusted data in which the number of pieces of data is reduced from the training data.
6 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate, on a basis of the first accuracy and the second accuracy, the accuracy information for each of the plurality of defined bias categories indicating an accuracy change in the model due to the adjustment of the number of pieces of data according to each of the plurality of defined bias categories.
7 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate the accuracy information for each of the plurality of defined bias categories on a basis of the first accuracy of the first model, which accuracy is measured by utilization of evaluation data, and the second accuracy of the second model which accuracy is measured by utilization of the evaluation data.
8 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate, on a basis of the first accuracy and the second accuracy, the accuracy information for each of the plurality of defined bias categories indicating a prediction of an accuracy change in the model of a case where the number of pieces of data is adjusted according to each of the plurality of defined bias categories.
9 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate the accuracy information for each of the plurality of defined bias categories indicating a prediction of an accuracy change in the model of a case where the number of pieces of data of the training data is increased.
10 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate the accuracy information for each of the plurality of defined bias categories including a prediction line of the accuracy of the model of a case where the number of pieces of data of the training data is increased.
11 . The information processing device according to claim 4 , wherein the processing circuitry is configured to generate the accuracy information related to one of the plurality of defined bias categories on a basis of the second accuracy of the second model trained on the adjusted data in which the number of pieces of data of the training data is adjusted for the one of the plurality of defined bias categories.
12 . The information processing device according to claim 11 , wherein the processing circuitry is configured to generate, on a basis of the second accuracy of the second model trained on the adjusted data in which the number of pieces of data of the training data is adjusted according to the one of the plurality of defined bias categories, accuracy information of the model.
13 . The information processing device according to claim 12 , wherein the processing circuitry is configured to generate the accuracy information of the model on a basis of the second accuracy of the second model trained on the adjusted data in which only the number of pieces of data corresponding to the one of the plurality of defined bias categories in the training data is adjusted.
14 . The information processing device according to claim 12 , wherein the processing circuitry is configured to:
acquire the training data including data corresponding to each of the plurality of defined bias categories, and
generate the accuracy information of the model on a basis of the second accuracy of the second model trained on the adjusted data in which the number of pieces of data corresponding to the one of the plurality of defined bias categories in the training data is reduced.
15 . The information processing device according to claim 12 , wherein the processing circuitry is configured to generate the accuracy information of the model on a basis of the first accuracy of the first model with respect to the one of the plurality of defined bias categories and the second accuracy of the second model with respect to the one of the plurality of defined bias categories.
16 . The information processing device according to claim 15 , wherein the processing circuitry is configured to generate the accuracy information of the model on a basis of the first accuracy of the first model which accuracy is measured by utilization of evaluation data corresponding to the one of the plurality of defined bias categories and the second accuracy of the second model which accuracy is measured by utilization of the evaluation data corresponding to the one of the plurality of defined bias categories.
17 . The information processing device according to claim 12 , wherein the processing circuitry is configured to generate the accuracy information indicating the accuracy of the model with respect to each of the plurality of defined bias categories on a basis of a plurality of pieces of the second accuracy of a plurality of second models respectively trained on a plurality of pieces of the adjusted data in which the number of pieces of data of the training data is adjusted for each of the plurality of defined bias categories.
18 . The information processing device according to claim 4 , wherein the processing circuitry is configured to cause the accuracy information to be displayed.
19 . An information processing method comprising:
acquiring training data used for training of a model by machine learning,
generating accuracy information for each of a plurality of defined bias categories indicating accuracy of the model on a basis of first accuracy of a first model, which is the model trained on the training data, and second accuracy of a second model that is the model trained on adjusted data in which a number of pieces of data of the training data is adjusted according to each of the plurality of defined bias categories,
predicting a change in accuracy with respect to the number of pieces of the training data on a basis of the accuracy information for each of the plurality of defined bias categories,
determining a required number of pieces of data to achieve a predetermined target value based on the predicted change,
when the required number of pieces of data does not exceed a threshold, adjusting a number of pieces of data used for training the model on a basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value,
when the required number of pieces of data does exceed the threshold, changing a network structure of the model on the basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value, and
training the model using the adjusted number of pieces of data or the changed model to generate an updated model.
20 . A non-transitory computer readable medium storing an information processing program including instructions that when executed by a computer causes the computer to perform a method, the method comprising:
acquiring training data used for training of a model by machine learning,
generating accuracy information for each of a plurality of defined bias categories indicating accuracy of the model on a basis of first accuracy of a first model, which is the model trained on the training data, and second accuracy of a second model that is the model trained on adjusted data in which a number of pieces of data of the training data is adjusted according to each of the plurality of defined bias categories,
predicting a change in accuracy with respect to the number of pieces of the training data on a basis of the accuracy information for each of the plurality of defined bias categories,
determining a required number of pieces of data to achieve a predetermined target value based on the predicted change,
when the required number of pieces of data does not exceed a threshold, adjusting a number of pieces of data used for training the model on a basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value,
when the required number of pieces of data does exceed the threshold, changing a network structure of the model on the basis of the predicted change in the accuracy for each of the plurality of defined bias categories so that the predicted change in the accuracy becomes the predetermined target value, and
training the model using the adjusted number of pieces of data or the changed model to generate an updated model.