Multi-scale distillation for low-resolution detection

1. A method of training a neural network, comprising:

receiving a high-resolution version of a student training image and a low-resolution version of the student training image;

generating a first feature map based on the high-resolution version of the student training image using a high-resolution encoder of a teacher network;

generating a second feature map based on the low-resolution version of the student training image using a low-resolution encoder of the teacher network;

generating a fused feature map based on the first feature map and the second feature map using a crossing feature-level fusion module of the teacher network;

generating a third feature map based on the low-resolution version of the student training image using an encoder of a student network;

computing a knowledge distillation (KD) loss based on a comparison of the third feature map from the student network and the fused feature map from the teacher network; and

updating parameters of the student network based on the KD loss.

2. The method of claim 1 , comprising:

generating prediction information for an object in the student training image; and

computing a detection loss based on the prediction information, wherein the parameters of the student network are updated based on the detection loss.

3. The method of claim 1 , comprising:

generating a plurality of fused feature maps using the teacher network, wherein each of the plurality of fused feature maps comprises a different resolution level;

generating a plurality of student training feature maps using the student network; and

comparing each of the plurality of fused feature maps to a corresponding feature map of the plurality of student training feature maps to obtain the KD loss.

4. The method of claim 1 , comprising:

receiving a high-resolution version of a teacher training image and a low-resolution version of the teacher training image;

generating a first teacher training feature map based on the high-resolution version of the teacher training image using the high-resolution encoder of the teacher network;

generating a second teacher training feature map based on the low-resolution version of the teacher training image using the low-resolution encoder of the teacher network;

generating a fused teacher training feature map based on the first teacher training feature map and the second teacher training feature map using the crossing feature-level fusion module of the teacher network;

generating teacher prediction information for an object in the teacher training image;

computing a teacher detection loss based on the teacher prediction information; and

updating parameters of the teacher network based on the teacher detection loss.

5. The method of claim 1 , further comprising:

computing a shift offset parameter based on a number of levels in the high-resolution encoder of the teacher network and a number of levels in the low-resolution encoder of the teacher network.

6. The method of claim 4 , further comprising:

generating a first set of teacher training feature maps based on the high-resolution version of the teacher training image using the high-resolution encoder of the teacher network; and

generating a second set of teacher training feature maps based on the low-resolution version of the teacher training image using the low-resolution encoder of the teacher network.

7. The method of claim 4 , further comprising:

generating high-resolution prediction information using the high-resolution encoder of the teacher network;

generating low-resolution prediction information using the low-resolution encoder of the teacher network; and

computing an arrangement loss based on the high-resolution prediction information and the low-resolution prediction information, wherein the parameters of the teacher network are updated based on the arrangement loss.

8. The method of claim 4 , further comprising:

concatenating the first teacher training feature map and the second teacher training feature map along a channel dimension to obtain a concatenated teacher training feature map, wherein the fused teacher training feature map is generated based on the concatenated teacher training feature map.

9. An apparatus for training a neural network, comprising:

a high-resolution encoder of a teacher network configured to generate a first feature map based on a high-resolution version of a student training image;

a low-resolution encoder of the teacher network configured to generate a second feature map based on a low-resolution version of the student training image;

a crossing feature-level fusion module of the teacher network configured to generate a fused feature map based on the first feature map and the second feature map; and

an encoder of a student network configured to generate a third feature map based on the low-resolution version of the student training image, wherein the encoder of the student network is trained using a knowledge distillation (KD) loss based on a comparison of the third feature map from the student network and the fused feature map from the teacher network.

10. The apparatus of claim 9 , further comprising:

a decoder of the teacher network configured to generate prediction information based on an output of the crossing feature-level fusion module.

11. The apparatus of claim 9 , further comprising:

a decoder of the student network configured to decode the third feature map to obtain prediction information for the low-resolution version of the student training image.

12. The apparatus of claim 9 , wherein:

the high-resolution encoder of the teacher network comprises a feature pyramid network (FPN) and the low-resolution encoder of the teacher network comprises an FPN with fewer levels than the high-resolution encoder of the teacher network.

13. The apparatus of claim 11 , wherein:

the encoder of the student network comprises a same number of levels as the low-resolution encoder of the teacher network.

14. The apparatus of claim 9 , wherein:

the first feature map is output from a third level of the high-resolution encoder of the teacher network and the second feature map is output from a second level of the low-resolution encoder of the teacher network, the second level being lower than the third level.

15. A non-transitory computer readable medium storing code for training a machine learning model, the code comprising instructions executable by at least one processor to:

receive a high-resolution version of a student training image and a low-resolution version of the student training image;

generate a first feature map based on the high-resolution version of the student training image using a high-resolution encoder of a teacher network;

generate a second feature map based on the low-resolution version of the student training image using a low-resolution encoder of the teacher network;

generate a fused feature map based on the first feature map and the second feature map using a crossing feature-level fusion module of the teacher network;

generate a third feature map based on the low-resolution version of the student training image using an encoder of a student network;

compute a knowledge distillation (KD) loss based on a comparison of the third feature map from the student network and the fused feature map from the teacher network; and

update parameters of the student network based on the KD loss.

16. The non-transitory computer readable medium of claim 15 , the code further comprising instructions executable by the at least one processor to:

generate prediction information for an object in the student training image; and

compute a detection loss based on the prediction information, wherein the parameters of the student network are updated based on the detection loss.

17. The non-transitory computer readable medium of claim 15 , the code further comprising instructions executable by the at least one processor to:

generate a plurality of fused feature maps using the teacher network, wherein each of the plurality of fused feature maps comprises a different resolution level;

generate a plurality of student training feature maps using the student network; and

compare each of the plurality of fused feature maps to a corresponding feature map of the plurality of student training feature maps to obtain the KD loss.

18. The non-transitory computer readable medium of claim 15 , the code further comprising instructions executable by the at least one processor to:

receive a high-resolution version of a teacher training image and a low-resolution version of the teacher training image;

generate a first teacher training feature map based on the high-resolution version of the teacher training image using the high-resolution encoder of the teacher network;

generate a second teacher training feature map based on the low-resolution version of the teacher training image using the low-resolution encoder of the teacher network;

generate a fused teacher training feature map based on the first teacher training feature map and the second teacher training feature map using the crossing feature-level fusion module of the teacher network;

generate teacher prediction information for an object in the teacher training image;

compute a teacher detection loss based on the teacher prediction information; and

update parameters of the teacher network based on the teacher detection loss.

19. The non-transitory computer readable medium of claim 15 , the code further comprising instructions executable by the at least one processor to:

compute a shift offset parameter based on a number of levels in the high-resolution encoder of the teacher network and a number of levels in the low-resolution encoder of the teacher network.