Object detection method for detecting one or more objects using a plurality of deep convolution neural network layers and object detection apparatus using the same method and non-transitory storage medium thereof
According to an exemplary embodiment, the disclosure provides an object detection method includes not limited to obtaining a set of a plurality of object annotated images in a source domain and have a first image style; obtaining a minority set of a plurality of object annotated images in a target domain and having a second image style; obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style; performing an image style transfer to generate a converted set of object annotated images having the second image style; generating object annotation for the majority set of the plurality of unannotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and performing an active domain adaptation to generate an object detection model.
1 . An object detection method for detecting one or more objects using a plurality of deep convolution neural layers, the method comprising:
obtaining a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtaining a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
performing an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
training a pseudo-labeler with supervised training based on the converted set of object annotated images having the second image style and the minority set of the object annotated images in the second image style;
generating, by the pseudo-labeler, object annotations for the majority set of the plurality of unannotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
performing an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.
2 . The object detection method of claim 1 further comprising:
recognizing a classification of a detected object of a captured image by using the object detection model to indicate the detected object with an object annotation which indicates that the detected object has been classified.
3 . The object detection method of claim 2 , wherein the object annotation is a two-dimensional (2D) label which is a 2D bounding box or a perspective-induced 2D bounding box or a 2D point to indicate that the object of the target image has been classified.
4 . The object detection method of claim 2 , wherein performing the classification of the detected object of the captured image based on the object detection model comprising:
performing the classification of the detected object of the captured image with the least annotation density based on density estimators which are prioritized for gradient descent according to a scheduling policy.
5 . The object detection method of claim 2 further comprising:
generating a model for a main detector which comprises a backbone architecture and an end layer and is generated in response to performing supervised training based on the active domain adaptation.
6 . The object detection method of claim 5 , further comprising:
performing a data augmentation on the minority set of the plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images to increase data variations so as to introduce a degree of generalization to the main detector during the supervised training.
7 . The object detection method of claim 5 , wherein the backbone architecture is configured to extract features at multi-scale resolutions, and end layer branch is configured to concatenate the multi-scale features to generate prediction regression for the object annotation.
8 . The object detection method of claim 1 , wherein performing the image style transfer comprising:
constructing a plurality of deep convolution neural layers which converting from the first image style of a 2D data array of the source domain into the second image style of another 2D data array of the target domain to generate the converted set of object annotated images having the second image style.
9 . The object detection method of claim 8 , wherein constructing the plurality of deep convolution neural layers further comprising:
encoding a 2D data array of a raw input space into a plurality of data arrays of a feature space and a first plurality of data arrays of a style space.
10 . The object detection method of claim 9 , wherein constructing the plurality of deep convolution neural layers further comprising:
replacing the first plurality of data array of the style space with a second plurality of data arrays of the style space from the target domain and combining the second plurality of data arrays of the style space with the plurality of data arrays of the feature space.
11 . The object detection method of claim 9 , wherein the set of the plurality of object annotated images which are in the source domain has a different quantity from the minority set of the plurality of object annotated images which are in the target domain plus the majority set of the plurality of unannotated images which are in the target domain.
12 . The object detection method of claim 9 , wherein in response to the set of the plurality of object annotated images which are in the source domain has a different quantity from the minority set of the plurality of object annotated images which are in the target domain plus the majority set of the plurality of unannotated images which are in the target domain, the processor is further configured to duplicate either an image in the source domain or an image in the target domain so that the quality is no longer different.
13 . The object detection method of claim 8 , wherein the performing the image style transfer from the source domain to the target domain comprising:
performing unsupervised learning by pairing a first image from the source domain without any annotation with a second image from the target domain without any annotation.
14 . The object detection method of claim 13 further comprising:
performing a source domain encoding the first image by extracting a first style feature and a first content feature from the first image; and
performing a target domain encoding the second image by extracting a second style feature and a second content feature from the second image.
15 . The object detection method of claim 14 further comprising:
performing a source domain decoding by receiving the first style feature, the first content feature, and the second style feature to generate a source target output image and a source to source output image; and
performing a target domain decoding by receiving the first content feature, the second content feature, and the second style feature to generate a target to target output image and a source to target output image.
16 . The object detection method of claim 14 further comprising:
calculating loss functions for the first style feature and the second style feature; and
performing a benchmark based on the calculated loss functions by iteratively adjusting weights of the plurality of deep convolution neural layers.
17 . The object detection method of claim 1 , performing a data augmentation on the minority set of the plurality of object annotated images and the converted set of object annotated images so as to introduce a degree of generalization to the pseudo-labeler during the supervised training.
18 . An object detection apparatus comprising:
a storage device; and
a processor coupled to the storage device and configured to:
obtain a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtain a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtain a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
perform an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
train a pseudo-labeler with supervised training based on the converted set of object annotated images having the second image style and the minority set of the object annotated images in the second image style;
generate, by the pseudo-labeler, object annotation for the majority set of the plurality of unannotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
perform an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.
19 . A non-transitory storage medium which stores computer readable program loaded into a processor of a computer to control the computer to perform functions comprising:
obtaining a set of a plurality of object annotated images which are in a source domain and have a first image style;
obtaining a minority set of a plurality of object annotated images which are in a target domain and having a second image style;
obtaining a majority set of a plurality of unannotated images which are in the target domain and having the second image style;
performing an image style transfer from the source domain to the target domain by converting the plurality of object annotated images in the source domain from having the first image style into having the second image style to generate a converted set of object annotated images having the second image style;
training a pseudo-labeler with supervised training based on the converted set of object annotated images having the second image style and the minority set of the object annotated images in the second image style;
generating, by the pseudo-labeler, object annotation for the majority set of the plurality of unannotated images in the second image style to change from the majority set of a plurality of unannotated images into a majority set of a plurality of annotated images; and
performing an active domain adaptation by adapting the minority set of a plurality of object annotated images, the converted set of object annotated images, and the majority set of the plurality of annotated images so as to generate an object detection model.