Joint training of neural networks using multi-scale hard example mining
An example apparatus for mining multi-scale hard examples includes a convolutional neural network to receive a mini-batch of sample candidates and generate basic feature maps. The apparatus also includes a feature extractor and combiner to generate concatenated feature maps based on the basic feature maps and extract the concatenated feature maps for each of a plurality of received candidate boxes. The apparatus further includes a sample scorer and miner to score the candidate samples with multi-task loss scores and select candidate samples with multi-task loss scores exceeding a threshold score.
1. An apparatus for mining multi-scale hard examples comprising:
memory;
machine readable instructions; and
processor circuitry to execute the machine readable instructions to at least:
generate candidate feature maps based on a mini-batch of sample candidates;
enable multi-scale extraction functionality by generating concatenated feature maps based on the candidate feature maps;
improve feature discrimination by extracting the concatenated feature maps for each of a plurality of received candidate boxes at two or more scales;
train a convolutional neural network with the concatenated feature maps;
improve accuracy of the trained convolutional neural network by scoring the candidate samples with multi-task loss scores, the multi-task loss scores corresponding to loss information; and
in response to detecting multi-task loss scores satisfying a threshold value, cause respective ones of the candidate samples to be selected for training.
2. The apparatus of claim 1 , wherein the loss information includes at least one of classification loss information or localization loss information.
3. The apparatus of claim 1 , wherein the processor circuitry is to generate candidate boxes by instantiating a region proposal network in response to receiving a plurality of mini-batches of images.
4. The apparatus of claim 1 , wherein respective ones of the concatenated feature maps include a plurality of channels having the candidate feature maps resized to a reference layer size.
5. The apparatus of claim 1 , wherein the multi-task loss score for each candidate sample is to be calculated based on a localization score and a classification score corresponding to classification and localization losses calculated for each candidate sample in a respective Stochastic Gradient Descent (SGD).
6. The apparatus of claim 1 , wherein the processor circuitry is to invoke the convolutional neural network to resize sample candidates from the mini-batch into a standard scale.
7. The apparatus of claim 1 , wherein the processor circuitry is to generate the concatenated feature maps by:
selecting a reference layer in the convolutional neural network; and
up-sampling or down-sampling feature maps from other layers in the convolutional neural network, respective ones of the concatenated feature maps including a feature map of the reference layer and the up-sampled or down-sampled feature maps of other layers in the convolutional neural network.
8. A computer readable storage device or storage disk comprising instructions that, when executed, cause processor circuitry to at least:
generate candidate feature maps based on a mini-batch of sample candidates;
enable multi-scale extraction functionality by generating concatenated feature maps based on the candidate feature maps;
improve feature discrimination by extracting the concatenated feature maps for each of a plurality of received candidate boxes at two or more scales;
train a convolutional neural network with the concatenated feature maps;
improve accuracy of the trained convolutional neural network by scoring the candidate samples with multi-task loss scores, the multi-task loss scores corresponding to loss information; and
in response to detecting multi-task loss scores satisfying a threshold score, cause respective ones of the candidate samples to be selected for training.
9. The storage device or storage disk of claim 8 , wherein the loss information includes at least one of classification loss information or localization loss information.
10. The storage device or storage disk of claim 8 , wherein the instructions, when executed, cause the processor circuitry to generate candidate boxes by instantiating a region proposal network in response to receiving a plurality of mini-batches of images.
11. The storage device or storage disk of claim 8 , wherein respective ones of the concatenated feature maps include a plurality of channels having the candidate feature maps resized to a reference layer size.
12. The storage device or storage disk of claim 8 , wherein the instructions, when executed, cause the processor circuitry to calculate the multi-task loss score for each candidate sample based on a localization score and a classification score corresponding to the classification and localization losses calculated for each candidate sample in a respective Stochastic Gradient Descent (SGD).
13. The storage device or storage disk of claim 8 , wherein the instructions, when executed, cause the processor circuitry to invoke the convolutional neural network to resize sample candidates from the mini-batch into a standard scale.
14. The storage device or storage disk of claim 8 , wherein the instructions, when executed, cause the processor circuitry to generate the concatenated feature maps by:
selecting a reference layer in the convolutional neural network; and
up-sampling or down-sampling feature maps from other layers in the convolutional neural network, respective ones of the concatenated feature maps including a feature map of the reference layer and the up-sampled or down-sampled feature maps of other layers in the convolutional neural network.
15. A system for mining multi-scale hard examples, comprising:
means for generating candidate feature maps using a mini-batch of sample candidates;
means for enabling multi-scale extraction functionality by generating concatenated feature maps based on the candidate feature maps;
means for improving feature discrimination by extracting the concatenated feature maps for each of a plurality of received candidate boxes;
means for training a convolutional neural network with the concatenated feature maps;
means for improving accuracy of the trained convolutional neural network by scoring the candidate samples with multi-task loss scores, the multi-task loss scores corresponding to loss information; and
means for causing respective ones of the candidate samples to be selected for training in response to detecting multi-task scores satisfying a threshold score.
16. The system of claim 15 , wherein the candidate boxes are to be generated by a region proposal network in response to receiving a plurality of mini-batches of example images.
17. The system of claim 15 , wherein each of the concatenated feature maps include a plurality of channels comprising the feature maps resized to a reference layer size.
18. The system of claim 15 , wherein the multi-task loss score for each candidate sample is to be calculated based on a localization score and a classification score corresponding to the classification and localization losses calculated for each candidate sample in a respective Stochastic Gradient Descent (SGD).
19. The system of claim 15 , wherein the means for generating candidate feature maps is to resize sample candidates from the received mini-batch into a standard scale.
20. The system of claim 15 , wherein the concatenated feature maps are calculated by:
selecting a reference layer in the convolutional neural network; and
up-sampling or down-sampling feature maps from other layers in the convolutional neural network, respective ones of the concatenated feature maps including a feature map of the reference layer and the up-sampled or down-sampled feature maps of other layers in the convolutional neural network.