Method and apparatus for detecting anchor-free object based on deep learning
A method for detecting an anchor-free object based on deep learning may include receiving a target image for object detection, inputting the image to a backbond network and performing learning to generate a heatmap, post-processing the heatmap, and outputting the post-processed result on a screen.
1. A method for detecting an anchor-free object based on deep learning, the method comprising:
receiving a target image for object detection, by an input device;
inputting the image to a backbond network and performing learning to generate a heatmap, by a learning device;
post-processing the heatmap, by a post-processing device; and
outputting the post-processed result on a screen, by an output device
wherein the generated heatmap is configured with channels as many as the number of objects to be classified, the objects being included in the image,
wherein the heatmap on which the object is displayed is generated using a two-dimensional (2D) Gaussian kernel for each channel through the learning.
2. The method of claim 1 , wherein the backbone network is a deformable convolutional neural network having a variable filter size and shape depending on the image.
3. The method of claim 1 , wherein the inputting of the image to the backbond network and the performing of the learning to generate the heatmap includes:
generating a ground truth heatmap including the 2D Gaussian kernel having the same size as a region of each object based on box coordinates included in a ground truth label; and
performing learning to reduce a Euclidean distance between the ground truth heatmap and an output heatmap of the backbone network.
4. The method of claim 1 , wherein a loss function L used for the learning is a mean squared error, and a size weight mask (SWM) technique of multiplying a weight corresponding to an object size by a loss value to output the loss value irrespective of the object size is applied to the loss function L.
5. The method of claim 4 , wherein the loss function L is that
L
=
∑
p
M
(
p
)
×
(
Y
(
p
)
-
Y
^
(
p
)
)
2
,
where, p denotes each pixel of the heatmap, Y denotes the ground truth heatmap, Ŷ denotes the backbone network output heatmap, and M denotes the SWM, and
wherein the SWM pixel value M(p) in the loss function L is determined as a value obtained by dividing a size of the entire heatmap by the size of the object for a region where there is the object and is determined as “1” for a region where there is no object.
6. The method of claim 1 , wherein the post-processing of the heatmap includes:
determining center coordinates of the object based on a local maximum value of the 2D Gaussian kernel;
calculating an end point for each direction of the object based on the determined center coordinates; and
generating a boundary box based on the calculated end point for each direction.
7. The method of claim 6 , wherein the local maximum value is determined through max pooling computation.
8. The method of claim 6 , wherein the calculating of the end point for each direction of the object based on the determined center coordinates includes:
setting the determined center coordinates to a search start point;
comparing a local value on coordinates moving a certain level in an upward direction, a downward direction, a leftward direction, or a rightward direction from the start point with a threshold; and
determining coordinates, corresponding to the local value less than the threshold, for each direction as an end point of the direction.
9. An apparatus for detecting an anchor-free object based on deep learning, the apparatus comprising:
an input device configured to receive a target image for object detection;
a learning device configured to input the image to a backbond network and perform learning to generate a heatmap;
a post-processing device configured to post-process the heatmap; and
an output device configured to output the post-processed result on a screen,
wherein the generated heatmap is configured with channels as many as the number of objects to be classified, the objects being included in the image,
wherein the learning device generates the heatmap on which the object is displayed using a 2D Gaussian kernel for each channel through the learning.
10. The apparatus of claim 9 , wherein the backbone network is a deformable convolutional neural network having a variable filter size and shape depending on the image.
11. The apparatus of claim 9 , wherein the learning device generates a ground truth heatmap including the 2D Gaussian kernel having the same size as a region of each object based on box coordinates included in a ground truth label and performs learning to reduce a Euclidean distance between the ground truth heatmap and an output heatmap of the backbone network.
12. The apparatus of claim 9 , wherein a loss function L used for the learning is a mean squared error, and a size weight mask (SWM) technique of multiplying a weight corresponding to an object size by a loss value to output the loss value irrespective of the object size is applied to the loss function L.
13. The apparatus of claim 12 , wherein The loss function L is that
L
=
∑
p
M
(
p
)
×
(
Y
(
p
)
-
Y
^
(
p
)
)
2
,
where, p denotes each pixel of the heatmap, Y denotes the ground truth heatmap, Ŷ denotes the backbone network output heatmap, and M denotes the SWM, and
wherein the SWM pixel value M(p) in the loss function L is determined as a value obtained by dividing a size of the entire heatmap by the size of the object for a region where there is the object and is determined as “1” for a region where there is no object.
14. The apparatus of claim 9 , wherein the preprocessing device determines center coordinates of the object based on a local maximum value of the 2D Gaussian kernel, calculates an end point for each direction of the object based on the determined center coordinates, and generates a boundary box based on the calculated end point for each direction.
15. The apparatus of claim 14 , wherein the local maximum value is determined through max pooling computation.
16. The apparatus of claim 14 , wherein the post-processing device sets the determined center coordinates to a search start point, compares a local value on coordinates moving a certain level in an upward direction, a downward direction, a leftward direction, or a rightward direction from the start point with a threshold, and identifies coordinates, corresponding to the local value less than the threshold, for each direction to determine the coordinates as an end point of the direction.
17. A method for detecting an anchor-free object based on deep learning, the method comprising:
receiving a target image for object detection, by an input device;
inputting the image to a backbond network and performing learning to generate a heatmap, by a learning device;
post-processing the heatmap, by a post-processing device; and
outputting the post-processed result on a screen, by an output device,
wherein the backbone network is a deformable convolutional neural network having a variable filter size and shape depending on the image.