Apparatus and method for detecting an object and a computer readable recording medium therefor
An apparatus and method for detecting an object and a computer readable recording medium therefor are disclosed. The apparatus includes: a first camera that obtains a first image of a first view angle range; a second camera that obtains a second image of a second view angle range having an overlapping region with the first view angle range; an object detection network that detects an object in the first and second images; a stereo matching device that corrects a deviation of a depth value between the first and second images in the overlapping region based on a feature value output by a backbone network of the object detection network; and a feature vector determination device that performs adversarial learning to reduce a deviation of a feature vector between objects in the overlapping region and a non-overlapping region while the object detection network detects the object.
1 . An apparatus for detecting an object, the apparatus comprising:
a first camera configured to obtain a first image of a first view angle range;
a second camera configured to obtain a second image of a second view angle range having an overlapping region with the first view angle range;
an object detection network configured to detect an object in the first image and the second image;
a stereo matching device configured to correct a deviation of a depth value between the first image and the second image in the overlapping region based on a feature value output by a backbone network of the object detection network; and
a feature vector determination device configured to perform adversarial learning to reduce a deviation of a feature vector between an object in the overlapping region and an object in a non-overlapping region while the object detection network detects the object.
2 . The apparatus of claim 1 , wherein the stereo matching device is configured to correct the deviation of the depth value by reducing a deviation between a predicted disparity map and a ground truth disparity map in the overlapping region.
3 . The apparatus of claim 2 , wherein the stereo matching device is configured to:
generate a matched feature value by matching a first image feature value of the first image and a second image feature value of the second image output by the backbone network of the object detection network;
obtain multi-scale feature values for the matched feature value by using a multi-scale layer; and
generate a cost volume based on the multi-scale feature values.
4 . The apparatus of claim 3 , wherein the stereo matching device is configured to obtain a stereo focus loss corresponding to a deviation between the cost volume and the ground truth disparity map, and wherein the object detection network is configured to learn an object detection model to reduce the stereo focus loss.
5 . The apparatus of claim 4 , wherein the feature vector determination device is configured to output a regional classification loss proportional to a deviation of the feature vector based on the feature vector provided from the object detection network.
6 . The apparatus of claim 5 , wherein the object detection network includes:
a transformer configured to output prediction location information and score information of the object based on the feature value output from the backbone network; and
a class classifier configured to output a bounding box loss and a class loss based on the prediction location information, the score information, and the regional classification loss.
7 . The apparatus of claim 6 , wherein the object detection network is configured to perform learning to reduce the regional classification loss output from the feature vector determination device.
8 . The apparatus of claim 7 , wherein the object detection network is configured to
perform the learning to reduce a total loss obtained by subtracting the regional classification loss from a sum of the bounding box loss, the class loss, and the stereo focus loss.
9 . A method of detecting an object, the method comprising:
extracting, by overlapping region detection device, an overlapping region between a first image captured by a first camera and a second image captured by a second camera;
correcting, by a stereo matching device, a deviation of a depth value between the first image and the second image in the overlapping region while an object is detected in the first image and the second image by using an object detection network; and
performing, by a feature vector determination device, adversarial learning to reduce a feature vector deviation between objects in the overlapping region and an non-overlapping region.
10 . The method of claim 9 , wherein correcting the deviation of the depth value includes:
obtaining a predicted disparity map in the overlapping region; and
reducing a deviation between the predicted disparity map and a ground truth disparity map.
11 . The method of claim 10 , wherein obtaining the predicted disparity map includes:
generating a matched feature value by matching a first image feature value of the first image and a second image feature value of the second image output from a backbone network of the object detection network;
obtaining multi-scale feature values for the matched feature value by using a multi-scale layer; and
generating a cost volume based on the multi-scale feature values.
12 . The method of claim 11 , wherein correcting the depth value further includes:
obtaining a stereo focus loss corresponding to a deviation between the cost volume and the ground truth disparity map; and
learning an object detection model to reduce the stereo focus loss.
13 . The method of claim 12 , wherein performing the adversarial learning further includes outputting a regional classification loss proportional to a deviation of the feature vector based on the feature vector provided from the object detection network.
14 . The method of claim 13 , wherein performing the adversarial learning further includes providing, by a network, the regional classification loss to a class classifier that outputs a bounding box loss and a class loss.
15 . The method of claim 14 , wherein performing the adversarial learning further includes learning the object detection network to reduce the regional classification loss.
16 . The method of claim 9 , wherein extracting the overlapping region includes obtaining 4D information including the depth value based on 2D information extracted from the first image and the second image.
17 . A computer readable recording medium that stores computer readable instructions for performing operations, wherein the operations include:
extracting an overlapping region between a first image captured by a first camera and a second image captured by a second camera;
correcting a deviation of a depth value between the first image and the second image in the overlapping region; and
performing adversarial learning to reduce a deviation of a feature vector between an object in the overlapping region and an object in a non-overlapping region while an object is detected in the first image and the second image by using an object detection network.
18 . The computer readable recording medium of claim 17 , wherein correcting the deviation of the depth value includes:
obtaining, by a backbone network of the object detection network, a predicted disparity map based on a feature value; and
obtaining a stereo focus loss based on a deviation between the predicted disparity map and a ground truth disparity map.
19 . The computer readable recording medium of claim 18 , wherein correcting the depth value further includes learning an object detection model to reduce the stereo focus loss.
20 . The computer readable recording medium of claim 17 , wherein the performing of the adversarial learning includes:
outputting, by a feature vector determination device, a regional classification loss proportional to a deviation of the feature vector based on the feature vector provided from the object detection network; and
learning the object detection network to reduce the regional classification loss.