Confidence score based machine learning model training
Systems and methods for training a machine learning model are disclosed. A system may be configured to obtain a plurality of training samples. The system includes a machine learning model to generate predictions and generate a confidence score for each generated prediction. In this manner, the system is configured to, for each training sample of the plurality of training samples, generate a prediction by a machine learning model based on the training sample and generating a confidence score associated with the prediction by the machine learning model. The system is also configured to train the machine learning model based on the plurality of predictions and associated confidence scores. For example, one or more training samples may be excluded from use in training the machine learning model based on the associated one or more confidence scores (such as the confidence score being less than a threshold).
1. A computer-implemented method for training a machine learning model for making predictions, comprising:
obtaining a plurality of training samples;
generating, by the machine learning model, a plurality of predictions and a plurality of confidence scores by, for each training sample of the plurality of training samples:
generating a prediction by the machine learning model based on the training sample; and
generating, by the machine learning model, a confidence score associated with the prediction by the machine learning model;
excluding one or more training samples of the plurality of training samples for subsequent training of the machine learning model based on one or more confidence scores associated with the one or more training samples; and
training the machine learning model based on the plurality of predictions and associated confidence scores, wherein the machine learning model is further trained using one or more non-excluded training samples of the plurality of training samples, and based on a loss function L(X, y, p), wherein:
the plurality of training samples is {X, y}, X comprising a plurality of input data including input data X i for integer i from 1 to N, wherein an integer N is a number of training samples in the plurality of training samples;
y comprises a plurality of desired predictions based on X, wherein y includes y i for integer i from 1 to N; and
p comprises the plurality of confidence scores associated with {X, y}, wherein p i is a confidence score associated with y i for integer i from 1 to N.
2. The method of claim 1 , wherein:
the plurality of training samples includes training sample i for integer i from 1 to N, wherein:
training sample i is {X i , y i };
X i is an input data of training sample i; and
y i is a desired prediction of training sample i based on X i .
3. The method of claim 2 , wherein excluding one or more training samples includes excluding training sample q if p q is less than a threshold confidence score, wherein q is an integer and 1≤q≤N.
4. The method of claim 3 , wherein if each p i for integer i from 1 to N equals a maximum confidence score, the output of the loss function L(X, y, p) is a standard training loss.
5. The method of claim 4 , wherein the loss function L(X, y, p) includes a regularization term R to prevent the loss function L(X, y, p) from equaling zero when each p i for integer i from 1 to N equals a minimum confidence score.
6. The method of claim 5 , wherein training the machine learning model is based on the loss function L(X, y, p)=Σ i=1 N p i l(ŷ i , y i )+λR(p) associated with the machine learning model, wherein:
ŷ i is an actual prediction by the machine learning model based on X i ;
l(ŷ i , y i ) is an individual loss determined between ŷ i and y i ; and
λ is a tunable parameter to adjust the regularization term R based on the plurality of confidence scores p.
7. The method of claim 6 , wherein training the machine learning model also includes adjusting the machine learning model to increase one or more confidences scores towards the maximum confidence score when the one or more training samples associated with the one or more increased confidence scores are to be used to train the machine learning model.
8. The method of claim 7 , wherein training the machine learning model also includes adjusting the machine learning model to decrease a confidence score towards a minimum confidence score when the training sample associated with the decreased confidence score is not to be used to train the machine learning model.
9. A system for training a machine learning model for making predictions, comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, causes the system to perform operations comprising:
obtaining a plurality of training samples;
generating, by the machine learning model, a plurality of predictions and a plurality of confidence scores by, for each training sample of the plurality of training samples:
generating a prediction by the machine learning model based on the training sample; and
generating, by the machine learning model, a confidence score associated with the prediction by the machine learning model;
excluding one or more training samples of the plurality of training samples for subsequent training of the machine learning model based on one or more confidence scores associated with the one or more training samples; and
training the machine learning model based on the plurality of predictions and associated confidence scores, wherein the machine learning model is further trained using one or more non-excluded training samples of the plurality of training samples, and based on a loss function L(X, y, p), wherein:
the plurality of training samples is {X, y}, X comprising a plurality of input data including input data X i for integer i from 1 to N, wherein an integer N is a number of training samples in the plurality of training samples;
y comprises a plurality of desired predictions based on X, wherein y includes y i for integer i from 1 to N; and
p comprises the plurality of confidence scores associated with {X, y}, wherein p i is a confidence score associated with y i for integer i from 1 to N.
10. The system of claim 9 , wherein:
the plurality of training samples includes training sample i for integer i from 1 to N, wherein:
training sample i is {X i , y i };
X i is an input data of training sample i; and
y i is a desired prediction of training sample i based on X i .
11. The system of claim 10 , wherein the execution of instructions to exclude one or more training samples causes the system to perform operations further including excluding training sample q if p q is less than a threshold confidence score, wherein q is an integer and 1≤q≤N.
12. The system of claim 11 , wherein if each p i for integer i from 1 to N equals a maximum confidence score, the output of the loss function L(X, y, p) is a standard training loss.
13. The system of claim 12 , wherein the loss function L(X, y, p) includes a regularization term R to prevent the loss function L(X, y, p) from equaling zero when each p i for integer i from 1 to N equals a minimum confidence score.
14. The system of claim 13 , wherein training the machine learning model is based on the loss function L(X, y, p)=Σ i=1 N p i l(ŷ i , y i )+λR(p) associated with the machine learning model, wherein:
ŷ i is an actual prediction by the machine learning model based on X i ;
l(ŷ i , y i ) is an individual loss determined between ŷ i and y i ; and
λ is a tunable parameter to adjust the regularization term R based on the plurality of confidence scores p.
15. The system of claim 14 , wherein the execution of instructions to train the machine learning model causes the system to perform operations further including adjusting the machine learning model to increase one or more confidences scores towards the maximum confidence score when the one or more training samples associated with the one or more increased confidence scores are to be used to train the machine learning model.
16. The system of claim 15 , wherein the execution of the instructions to train the machine learning model causes the system to perform operations further including adjusting the machine learning model to decrease a confidence score towards a minimum confidence score when the training sample associated with the decreased confidence score is not to be used to train the machine learning model.