Apparatus and method for retraining object detection using undetected image
An apparatus for retraining an object detector according to an exemplary embodiment includes an inputter configured to receive an undetected image, a style transferer configured to generate one or more first augmented images that have the same content attribute as an object area of the undetected image, but a different style attribute, a content transferer configured to generate one or more second augmented images that have the same style attribute as the object area, but a different content attribute, and an influence analyzer configured to analyze a cause of non-detection of the undetected image by comparing object detection reliabilities of the undetected image, the first augmented image, and the second augmented image.
1. An apparatus for retraining an object detector comprising:
an inputter configured to receive an undetected image;
a style transferer configured to generate one or more first augmented images that have a same content attribute as an object area of the undetected image, but a different style attribute;
a content transferer configured to generate one or more second augmented images that have a same style attribute as the object area, but a different content attribute; and
an influence analyzer configured to analyze a cause of non-detection of the undetected image by comparing object detection reliabilities of the undetected image, the first augmented image, and the second augmented image.
2. The apparatus of claim 1 , wherein the style transferer is configured to extract the content attribute from the object area and generate the first augmented images by combining the extracted content attribute and a preset style-set.
3. The apparatus of claim 1 , wherein the content transferer is configured to extract the style attribute from the object area and generate the second augmented images by combining the extracted content attribute and a preset content-set.
4. The apparatus of claim 1 , wherein the influence analyzer is configured to compute a degree of influence of style and a degree of influence of content on the non-detection by using the object detection reliability of each of the undetected image, the first augmented images, and the second augmented images, and preset advance data.
5. The apparatus of claim 4 , wherein the degree of influence of style is computed by the following equation:
Style (1,2) =Max(Conf A2 −Conf A1 , 0)+Max(Conf B2 −Conf B1 , 0),
where A1 represents an undetected object, B2 represents advance data, A2 represents a first augmented image in which a style attribute of A1 is transferred to correspond to B2, B1 represents a second augmented image in which a content attribute of A1 is transferred to correspond to B2, Conf A1 , Conf A2 , Conf B1 , and Conf B2 represent an object detection reliability of A1, A2, B1, and B2, respectively, and Style (1,2) represents a degree of influence of style of A1.
6. The apparatus of claim 4 , wherein the degree of influence of content is computed by the following equation:
Content (A,B) =Max(Conf B1 −Conf A1 , 0)+Max(Conf B2 −Conf A2 , 0),
where A1 represents an undetected object, B2 represents advance data, A2 represents a first augmented image in which a style attribute of A1 is transferred to correspond to B2, B1 represents a second augmented image in which a content attribute of A1 is transferred to correspond to B2, Conf A1 , Conf A2 , Conf B1 , and Conf B2 represent an object detection reliability of A1, A2, B1, and B2, respectively, and Content (A,B) represents a degree of influence of content of A1.
7. The apparatus of claim 4 , further comprising a training data generator configured to generate one or more retraining images from the undetected image based on the degree of influence of style and the degree of influence of content.
8. The apparatus of claim 7 , wherein the training data generator is configured to change rates of change of style attribute and content attribute according to the degree of influence of style and the degree of influence of content when augmenting the retraining image.
9. The apparatus of claim 4 , further comprising a loss function updater configured to update a loss function of the object detector based on the degree of influence of style and the degree of influence of content.
10. The apparatus of claim 9 , wherein the loss function updater is configured to change a weight of a classification loss of the loss function according to the degree of influence of style and the degree of influence of content.
11. A method of retraining an object detector, which is performed by a computing device that includes one or more processors and memory storing one or more programs to be executed by the one or more processors, the method comprising:
receiving an undetected image;
generating one or more first augmented images that have a same content attribute as an object area of the undetected image, but a different style attribute;
generating one or more second augmented images that have a same style attribute as the object area, but a different content attribute; and
analyzing a cause of non-detection of the undetected image by comparing object detection reliabilities of the undetected image, the first augmented image, and the second augmented image.
12. The method of claim 11 , wherein the generating of the first augmented images comprises extracting the content attribute from the object area and generating the first augmented images by combining the extracted content attribute and a preset style-set.
13. The method of claim 11 , wherein the generating of the second augmented images comprises extracting the style attribute from the object area and generating the second augmented images by combining the extracted content attribute and a preset content-set.
14. The method of claim 11 , wherein the analyzing comprises calculating a degree of influence of style and a degree of influence of content on the non-detection by using the object detection reliability of each of the undetected image, the first augmented images, and the second augmented images, and preset advance data.
15. The method of claim 14 , wherein the degree of influence of style is computed by the following equation:
Style (1,2) =Max(Conf A2 −Conf A1 , 0)+Max(Conf B2 −Conf B1 , 0),
where A1 represents an undetected object, B2 represents advance data, A2 represents a first augmented image in which a style attribute of A1 is transferred to correspond to B2, B1 represents a second augmented image in which a content attribute of A1 is transferred to correspond to B2, Conf A1 , Conf A2 , Conf B1 , and Conf B2 represent an object detection reliability of A1, A2, B1, and B2, respectively, and Style (1,2) represents a degree of influence of style of A1.
16. The method of claim 14 , wherein the degree of influence of content is computed by the following equation:
Content (A,B) =Max(Conf B1 −Conf A1 , 0)+Max(Conf B2 −Conf A2 , 0),
where A1 represents an undetected object, B2 represents advance data, A2 represents a first augmented image in which a style attribute of A1 is transferred to correspond to B2, B1 represents a second augmented image in which a content attribute of A1 is transferred to correspond to B2, Conf A1 , Conf A2 , Conf B1 , and Conf B2 represent an object detection reliability of A1, A2, B1, and B2, respectively, and Content (A,B) represents a degree of influence of content of A1.
17. The method of claim 14 , further comprising generating one or more retraining images from the undetected image based on the degree of influence of style and the degree of influence of content.
18. The method of claim 17 , wherein the generating of the retraining image comprises changing rates of change of style attribute and content attribute according to the degree of influence of style and the degree of influence of content when augmenting the retraining image.
19. The method of claim 14 , further comprising updating a loss function of the object detector based on the degree of influence of style and the degree of influence of content.
20. The method of claim 19 , wherein the updating of the loss function comprises changing a weight of a classification loss of the loss function according to the degree of influence of style and the degree of influence of content.