Source-free cross domain detection method with strong data augmentation and self-trained mean teacher modeling
A method for implementing source-free domain adaptive detection is presented. The method includes, in a pretraining phase, applying strong data augmentation to labeled source images to produce perturbed labeled source images and training an object detection model by using the perturbed labeled source images to generate a source-only model. The method further includes, in an adaptation phase, training a self-trained mean teacher model by generating a weakly augmented image and multiple strongly augmented images from unlabeled target images, generating a plurality of region proposals from the weakly augmented image, selecting a region proposal from the plurality of region proposals as a pseudo ground truth, detecting, by the self-trained mean teacher model, object boxes and selecting pseudo ground truth boxes by employing a confidence constraint and a consistency constraint, and training a student model by using one of the multiple strongly augmented images jointly with an object detection loss.
1 . A method for implementing source-free domain adaptive detection, the method comprising:
in a pretraining phase:
applying a first level of data augmentation to labeled source images to produce perturbed labeled source images; and
training an object detection model by using the perturbed labeled source images to generate a source-only model; and
in an adaptation phase, train a self-trained mean teacher model by:
generating an augmented image with a second level of data augmentation that is less perturbed than the first level of data augmentation and multiple first level of data augmentation augmented images from unlabeled target images;
generating a plurality of region proposals from the second level augmented image;
selecting a region proposal from the plurality of region proposals as a pseudo ground truth;
detecting, by the self-trained mean teacher model, object boxes and selecting pseudo ground truth boxes by employing a confidence constraint and a consistency constraint; and
training a student model by using one of the multiple first level of data augmentation augmented images jointly with an object detection loss.
2 . The method of claim 1 , wherein the confidence constraint requires a highest classification score of a detected object of interest in a scene to be greater than a predetermined threshold.
3 . The method of claim 1 , wherein the consistency constraint requires a detected object of interest in a scene to be classified as a same class when mapped to all of the multiple first level of data augmentation augmented images.
4 . The method of claim 1 , wherein the self-trained mean teacher model is updated as a moving average of the student model and the student model is updated with gradient back-propagation.
5 . The method of claim 1 , wherein the first level of data augmentation is a stochastic function and the multiple first level of data augmentation augmented images are all different with respect to each other.
6 . The method of claim 1 , wherein the student model is trained by further using a KL divergence loss of classification scores of the selected pseudo ground truth and an L2 norm of bounding box regression offsets of the selected pseudo ground truth.
7 . The method of claim 1 , wherein the plurality of region proposals are mapped to the multiple first level of data augmentation augmented images to help select the pseudo ground truth boxes.
8 . A non-transitory computer-readable storage medium comprising a computer-readable program for implementing source-free domain adaptive detection, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
in a pretraining phase:
applying a first level of data augmentation to labeled source images to produce perturbed labeled source images; and
training an object detection model by using the perturbed labeled source images to generate a source-only model; and
in an adaptation phase, train a self-trained mean teacher model by:
generating an augmented image with a second level of data augmentation that is less perturbed than the first level of data augmentation and multiple first level of data augmentation augmented images from unlabeled target images;
generating a plurality of region proposals from the second level augmented image;
selecting a region proposal from the plurality of region proposals as a pseudo ground truth;
detecting, by the self-trained mean teacher model, object boxes and selecting pseudo ground truth boxes by employing a confidence constraint and a consistency constraint; and
training a student model by using one of the multiple first level of data augmentation augmented images jointly with an object detection loss.
9 . The non-transitory computer-readable storage medium of claim 8 , wherein the confidence constraint requires a highest classification score of a detected object of interest in a scene to be greater than a predetermined threshold.
10 . The non-transitory computer-readable storage medium of claim 8 , wherein the consistency constraint requires a detected object of interest in a scene to be classified as a same class when mapped to all of the multiple first level of data augmentation augmented images.
11 . The non-transitory computer-readable storage medium of claim 8 , wherein the self-trained mean teacher model is updated as a moving average of the student model and the student model is updated with gradient back-propagation.
12 . The non-transitory computer-readable storage medium of claim 8 , wherein the first level of data augmentation is a stochastic function and the multiple first level of data augmentation augmented images are all different with respect to each other.
13 . The non-transitory computer-readable storage medium of claim 8 , wherein the student model is trained by further using a KL divergence loss of classification scores of the selected pseudo ground truth and an L2 norm of bounding box regression offsets of the selected pseudo ground truth.
14 . The non-transitory computer-readable storage medium of claim 8 , wherein the plurality of region proposals are mapped to the multiple first level of data augmentation augmented images to help select the pseudo ground truth boxes.
15 . A system for implementing source-free domain adaptive detection, the system comprising:
a memory; and
one or more processors in communication with the memory configured to:
in a pretraining phase:
apply a first level of data augmentation to labeled source images to produce perturbed labeled source images; and
train an object detection model by using the perturbed labeled source images to generate a source-only model; and
in an adaptation phase, train a self-trained mean teacher model by:
generating an augmented image with a second level of data augmentation that is less perturbed than the first level of data augmentation and multiple first level of data augmentation augmented images from unlabeled target images;
generating a plurality of region proposals from the second level augmented image;
selecting a region proposal from the plurality of region proposals as a pseudo ground truth;
detecting, by the self-trained mean teacher model, object boxes and selecting pseudo ground truth boxes by employing a confidence constraint and a consistency constraint; and
training a student model by using one of the multiple first level of data augmentation augmented images jointly with an object detection loss.
16 . The system of claim 15 , wherein the confidence constraint requires a highest classification score of a detected object of interest in a scene to be greater than a predetermined threshold.
17 . The system of claim 15 , wherein the consistency constraint requires a detected object of interest in a scene to be classified as a same class when mapped to all of the multiple first level of data augmentation augmented images.
18 . The system of claim 15 , wherein the self-trained mean teacher model is updated as a moving average of the student model and the student model is updated with gradient back-propagation.
19 . The system of claim 15 , wherein the first level of data augmentation is a stochastic function and the multiple first level of data augmentation augmented images are all different with respect to each other.
20 . The system of claim 15 , wherein the student model is trained by further using a KL divergence loss of classification scores of the selected pseudo ground truth and an L2 norm of bounding box regression offsets of the selected pseudo ground truth.