Training network to maximize true positive rate at low false positive rate
Some embodiments provide a method for training a machine-trained (MT) network that processes input data using network parameters. The method maps a set of input instances to a set of output values by propagating the set of input instances through the MT network. The set of input instances includes input instances for each of multiple categories. For a particular input instance selected as an anchor instance, the method calculates a true positive rate (TPR) for the MT network as a function of a distance between the output value for the anchor instance and the output value for each input instance not in a same category as the anchor instance. The method calculates a loss function for the anchor instance that maximizes the TPR for the MT network at low false positive rate. The method trains the network parameters using the calculated loss function.
1. A method for training a machine-trained (MT) network that processes input data using a plurality of network parameters, the method comprising:
for each input image of a set of input images propagating the input image through the MT network to generate a corresponding output value indicating a category into which the MT network classifies the input image, wherein the set of input images comprises, for each respective category of a plurality of categories, a respective plurality of input images;
for a particular input image selected as an anchor image:
for each respective image in a different category than the anchor image, computing a distance between the output value for the anchor image and the output value for the respective image in the different category;
calculating a true positive rate (TPR) for the anchor image in the MT network using a function that compares (i) the computed distances between the output value for the anchor image and the output values for the input images in different categories than the anchor image with (ii) an average distance between the output value for the anchor image and the output values for other images in the same category as the anchor image; and
calculating a loss function for the anchor image that maximizes the TPR for the anchor image in the MT network at low false positive rate; and
training the network parameters using the calculated loss function.
2. The method of claim 1 , wherein the average distance between the output value for the anchor image and output values for other images in the same category is a mean distance, the method further comprising calculating the mean distance between the output value for the anchor image and the output value for each input image in the same category as the anchor image.
3. The method of claim 2 further comprising calculating a standard deviation for the anchor image as a function of the mean distance, wherein the function used to calculate the TPR further uses the standard deviation calculated for the anchor image.
4. The method of claim 3 , wherein the standard deviation is further a function of a lower bound for the mean distance.
5. The method of claim 1 , wherein the function used to calculate the TPR that compares the computed distances between the output values for the input images in different categories than the anchor with the average distance between the output value for the anchor image and the output values for other images in the same category than the anchor image is a cumulative distribution (CDR) function.
6. The method of claim 1 , wherein each output value is a point in multiple dimensions, wherein the distance between the output value for the anchor image and the output value for a respective image in a different category is computed by summing a square of distances between a point representing the output value for the anchor image and a point representing the output value for the respective image in each of the multiple dimensions.
7. The method of claim 1 , wherein the distances between the output value for the anchor image and the output values for other images in the same category as the anchor image are normally distributed.
8. The method of claim 7 , wherein the distances between the output value for the anchor image and the output values for other images in the same category as the anchor image are calculated using a squared distance function.
9. The method of claim 1 , wherein the MT network is a neural network comprising input nodes, output nodes, and interior nodes between the input nodes and output nodes, wherein each node produces a node output value and each interior node and output node receives as input values a set of node output values of other nodes.
10. A non-transitory machine-readable medium storing a program which when executed by at least one processing unit trains a machine-trained (MT) network that processes input data using a plurality of network parameters, the program comprising sets of instructions for:
for each input image of a set of input images propagating the input image through the MT network to generate a corresponding output value indicating a category into which the MT network classifies the input image, wherein the set of input images comprises, for each respective category of a plurality of categories, a respective plurality of input images;
for a particular input image selected as an anchor image:
for each respective image in a different category than the anchor image, computing a distance between the output value for the anchor image and the output value for the respective image in the different category;
calculating a true positive rate (TPR) for the anchor image in the MT network using a function that compares (i) the computed distances between the output value for the anchor image and the output values for the input images in different categories than the anchor image with (ii) an average distance between the output value for the anchor image and the output values for other images in the same category as the anchor image; and
calculating a loss function for the anchor image that maximizes the TPR for the anchor image in the MT network at low false positive rate; and
training the network parameters using the calculated loss function.
11. The non-transitory machine-readable medium of claim 10 , wherein the average distance between the output value for the anchor image and output values for other images in the same category is a mean distance, wherein the program further comprises a set of instructions for calculating the mean distance between the output value for the anchor image and the output value for each input image in the same category as the anchor image.
12. The non-transitory machine-readable medium of claim 11 , wherein the program further comprises a set of instructions for calculating a standard deviation for the anchor image as a function of the mean distance, wherein the function used to calculate the TPR further uses the standard deviation calculated for the anchor image.
13. The non-transitory machine-readable medium of claim 12 , wherein the standard deviation is further a function of a lower bound for the mean distance.
14. The non-transitory machine-readable medium of claim 10 , wherein the function used to calculate the TPR that compares the computed distances between the output values for the input images in different categories than the anchor with the average distance between the output value for the anchor image and the output values for other images in the same category than the anchor image is a cumulative distribution (CDR) function.
15. The non-transitory machine-readable medium of claim 10 , wherein each output value is a point in multiple dimensions, wherein the distance between the output value for the anchor image and the output value for a respective image in a different category is computed by summing a square of distances between the point representing the output value for the anchor image and the point representing the output value for the respective image in each of the multiple dimensions.
16. The non-transitory machine-readable medium of claim 10 , wherein the distances between the output value for the anchor image and the output values for other images in the same category as the anchor image are normally distributed.
17. The non-transitory machine-readable medium of claim 16 , wherein the distances between the output value for the anchor image and the output values for other images in the same category as the anchor image are calculated using a squared distance function.
18. The non-transitory machine-readable medium of claim 10 , wherein the MT network is a neural network comprising input nodes, output nodes, and interior nodes between the input nodes and output nodes, wherein each node produces a node output value and each interior node and output node receives as input values a set of node output values of other nodes.