Method and system for training a neural network
A method for training a neural network for detecting a plurality of classes of object within a sample comprises providing a training data set comprising a plurality of samples, each annotated according to whether the samples include labelled objects of interest. In a first type of samples, all objects of interest are labelled according to their class and comprise a foreground of the samples, the remainder of the samples comprising background. In a second type of samples, some objects of interest are labelled in a foreground and their background may comprise unlabelled objects. A third type of samples comprise only background comprising no objects of interest. Negative mining is only performed on the results of processing the first and third types of samples.
1. A method for training a neural network for detecting a plurality of classes of object within a sample, the method comprising:
providing a training data set comprising a plurality of samples, each annotated according to whether the samples include labelled objects of interest, where;
in a first type of samples, all objects of interest are labelled according to their class and comprise a foreground of said samples, the remainder of said samples comprising background;
in a second type of samples, some objects of interest are labelled in a foreground and their background may comprise unlabelled objects; and
a third type of samples comprise only background comprising no objects of interest;
for any given training epoch:
selecting a number of samples of each of said first, second and third type;
applying a given generation of said neural network to said selected samples to produce a respective set of one or more feature maps for each sample;
for portions of said feature maps corresponding to locations of objects labelled as being objects of a given class from said first type of samples, performing a positive loss function to determine a difference between at least one feature map value and a positive value associated with foreground;
for portions of said feature maps not corresponding to locations of objects labelled as being objects of a given class from said first type of samples, performing a negative loss function to determine a difference between at least feature map value and a negative value associated with background;
for portions of said feature maps corresponding to locations of labelled objects from said second type of samples, only performing a loss function to determine a difference between at least one feature map value and a value associated with foreground;
for limited portions of said feature maps of said third type of samples, performing a negative loss function to determine a difference between at least feature map value and a negative value associated with background; and
combining said loss function results before updating the weights for a subsequent generation of said neural network.
2. The method according to claim 1 wherein said samples comprise images and wherein said objects of interest comprise portions of said images.
3. The method according to claim 2 wherein said neural network comprises a multi-box detector in which said neural network tests regions of a sample image to produce respective feature map values for objects of each class at different scales and aspect ratios.
4. The method according to claim 3 wherein said loss function takes into account a location of a test region relative to a labelled object.
5. The method according to claim 3 wherein said loss function takes into account a centeredness of a test region relative to a labelled object.
6. The method according to claim 1 comprising generating said third type of samples for a training epoch from background portions of said first type of samples.
7. The method according to claim 1 wherein said first type of samples from which said third type of samples are generated are from another training epoch.
8. The method according to claim 1 wherein for a given epoch, samples of said first type comprise approximately 50% of said selected samples with the remaining samples being divided between said second and third type of samples.
9. The method according to claim 8 where said remaining samples are divided evenly between said second and third type of samples.
10. The method according to claim 1 wherein said step of performing a negative loss function to determine a difference between at least feature map value and a negative value associated with background in said third type of samples is performed for no more than for a limited multiple of a number of instances of performing said positive loss function to determine a difference between at least one feature map value and a positive value associated with foreground in said second type of samples.
11. The method according to claim 10 wherein said limited multiple is 3.
12. The method according to claim 1 wherein said method is responsive to said second type of samples being positively labelled as objects of a given class in said foreground to perform a positive loss function to determine a difference between at least one feature map value and a positive value associated with foreground.
13. The method according to claim 1 wherein said method is responsive to said second type of samples being negatively labelled as not being objects of a given class in said foreground to perform a negative loss function to determine a difference between at least one feature map value and a negative value associated with foreground.
14. A system for training a neural network comprising a processor configured to perform the steps of the method of claim 1 .
15. A computer program product comprising a non-transitory computer readable medium, comprising instructions which when executed by a processor of a computing device are configured to perform the steps of the method of claim 1 .