Anchor-free RPN-based object detection method
Disclosed is a computing device for generating region information of at least one object included in an image, the computing device including: a memory including computer-executable components; and a processor for executing the computer-executable components stored in the memory, in which the computer-executable components include: a key point heat map generation component for generating, for the image, a key point heat map including key point information of the at least one object; a non-rotating bounding box generation component for generating a non-rotating bounding box based on the generated key point heat map; a rotating bounding box generation component for generating a rotating bounding box by using the non-rotating bounding box; and a final bounding box generation component for representing a region occupied by the at least one object in the image by using at least one of the non-rotating bounding box or the rotating bounding box.
1. A computing device for generating region information of at least one object included in an image, the computing device comprising:
a memory including computer-executable components; and
a processor for executing the computer-executable components stored in the memory,
wherein the computer-executable components include:
a key point heat map generation component for generating, for the image, a key point heat map including key point information of the at least one object;
a non-rotating bounding box generation component for generating a non-rotating bounding box based on the generated key point heat map;
a rotating bounding box generation component for generating a rotating bounding box by using the non-rotating bounding box; and
a final bounding box generation component for representing a region occupied by the at least one object in the image by using at least one of the non-rotating bounding box or the rotating bounding box.
2. The computing device of claim 1 , wherein the key point heat map includes information about a probability that each of the at least one pixel included in the image is a center point of the at least one object.
3. The computing device of claim 1 , wherein the key point heat map generation component corresponds a pixel corresponding to a center point of the object among pixels included in the image to a key point, and corresponds a pixel other than the pixel corresponding to the center point of the object among the pixels included in the image to a background.
4. The computing device of claim 3 , wherein the key point heat map generation component is trained using a loss function based on the number of at least one object and ground truth about the center point of each of the at least one object.
5. The computing device of claim 1 , wherein the non-rotating bounding box generation component determines at least one of size information and shape information of the at least one object, and generates a non-rotating bounding box of each of the at least one object based on at least one of the determined size information and shape information.
6. The computing device of claim 5 , wherein the non-rotating bounding box generation component includes a neural network trained based on ground truth information of a size of the at least one object in order to determine size information of the at least one object.
7. The computing device of claim 1 , wherein the rotating bounding box generation component generates one or more candidate rotating bounding boxes by adding one or more candidate rotation angle information to the generated non-rotating bounding boxes.
8. The computing device of claim 7 , wherein the rotating bounding box generation component determines a candidate rotating bounding box in which Intersection Over Union (IoU) with a corresponding object region satisfies a predetermined reference from among the one or more candidate rotating bounding boxes as a rotating bounding box.
9. The computing device of claim 1 , wherein the rotating bounding box generation component performs Rotated Region Of Interest (RROI) pooling, and
the final bounding box generation component generates a final bounding box based on a result of the RROI pooling.
10. The computing device of claim 1 , wherein the non-rotating bounding box generation component performs ROI pooling and the rotating bounding box generation component performs RROI pooling, and
the final bounding box generation component generates a final bounding box by using a result of the ROI pooling and a result of the RROI pooling.
11. A method of generating, by a computing device, region information of at least one object included in an image, the method comprising:
generating a key point heat map including key point information of the at least one object with respect to the image;
generating a non-rotating bounding box based on the generated key point heat map;
generating a rotating bounding box by using the non-rotating bounding box; and
generating a final bounding box representing a region occupied by the at least one object in the image by using at least one of the non-rotating bounding box or the rotating bounding box.
12. A non-transitory computer-readable storage medium storing computer program instructions for causing one or more processors to generate region information of at least one object included in an image, the instructions comprising:
generating a key point heat map including key point information of the at least one object with respect to the image;
generating a non-rotating bounding box based on the generated key point heat map;
generating a rotating bounding box by using the non-rotating bounding box; and
generating a final bounding box representing a region occupied by the at least one object in the image by using at least one of the non-rotating bounding box or the rotating bounding box.