Method and apparatus with object tracking

1. A processor-implemented method with object tracking, comprising:

determining an initial template image based on an input bounding box and an input image;

generating an initial feature map by extracting features from the initial template image;

generating a transformed feature map by performing feature transformation adapted to objectness on the initial feature map;

generating an objectness probability map and a bounding box map indicating bounding box information corresponding to each coordinate of the objectness probability map by performing objectness-based bounding box regression analysis on the transformed feature map; and

determining a refined bounding box based on the objectness probability map and the bounding box map.

2. The method of claim 1 , wherein

the generating of the objectness probability map comprises generating the objectness probability map by performing classification on the transformed feature map; and

the generating of the bounding box map comprises generating the bounding box map by performing localization on the transformed feature map.

3. The method of claim 1 , wherein the determining of the refined bounding box comprises:

determining a first temporary bounding box based on the objectness probability map and the bounding box map; and

comparing an area of the input bounding box and an area of the first temporary bounding box.

4. The method of claim 3 , wherein the determining of the first temporary bounding box comprises determining the first temporary bounding box based on bounding box information of a bounding box map corresponding to a highest probability of the objectness probability map.

5. The method of claim 3 , wherein the comparing of the area of the input bounding box and the area of the first temporary bounding box comprises comparing a ratio of the input bounding box to the first temporary bounding box with a threshold.

6. The method of claim 3 , wherein the determining of the refined bounding box comprises:

determining a second temporary bounding box based on the objectness probability map and the bounding box map based on a result of the comparing; and

determining the refined bounding box by comparing the first temporary bounding box and the second temporary bounding box.

7. The method of claim 6 , wherein the determining of the second temporary bounding box comprises, in response to the ratio of the input bounding box to the first temporary bounding box being greater than the threshold, determining the second temporary bounding box based on the objectness probability map and the bounding box map.

8. The method of claim 6 , wherein the determining of the refined bounding box by comparing the first temporary bounding box and the second temporary bounding box comprises:

determining a first intersection of union (IOU) of the first temporary bounding box and the input bounding box;

determining a second IOU of the second temporary bounding box and the input bounding box; and

determining the refined bounding box by comparing the first IOU and the second IOU.

9. The method of claim 8 , wherein the determining of the refined bounding box by comparing the first IOU and the second IOU comprises:

in response to the first IOU being greater than the second IOU, determining the first temporary bounding box to be the refined bounding box; and

in response to the second IOU being greater than or equal to the first IOU, determining the second temporary bounding box to be the refined bounding box.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .

11. An apparatus with object tracking, comprising:

one or more processors configured to:

determine an initial template image based on an input bounding box and an input image;

generate an initial feature map by extracting features of the initial template image;

generate a transformed feature map by performing feature transformation adapted to objectness on the initial feature map;

generate an objectness probability map and a bounding box map indicating bounding box information corresponding to each coordinate of the objectness probability map by performing objectness-based bounding box regression analysis on the transformed feature map; and

determine a refined bounding box based on the objectness probability map and the bounding box map.

12. The apparatus of claim 11 , wherein, for the determining of the refined bounding box, the one or more processors are configured to determine the refined bounding box based on bounding box information of a bounding box map corresponding to a highest probability of the objectness probability map.

13. The apparatus of claim 11 , wherein the one or more processors are configured to:

for the generating of the objectness probability map, generate the objectness probability map by performing classification on the transformed feature map, and

for the generating of the bounding box map, generate the bounding box map by performing localization on the transformed feature map.

14. The apparatus of claim 11 , wherein, for the determining of the refined bounding box, the one or more processors are configured to:

determine a first temporary bounding box based on the objectness probability map and the bounding box map, and

compare an area of the input bounding box and an area of the first temporary bounding box.

15. The apparatus of claim 14 , wherein, for the determining of the first temporary bounding box, the one or more processors are configured to determine the first temporary bounding box based on bounding box information of a bounding box map corresponding to a highest probability of the objectness probability map.

16. The apparatus of claim 14 , wherein, for the comparing of the area of the input bounding box and the area of the first temporary bounding box, the one or more processors are configured to compare a ratio of the input bounding box to the first temporary bounding box with a threshold.

17. The apparatus of claim 14 , wherein, for the determining of the refined bounding box, the one or more processors are configured to:

determine a second temporary bounding box based on the objectness probability map and the bounding box map based on a result of the comparing, and

determine the refined bounding box by comparing the first temporary bounding box and the second temporary bounding box.

18. The apparatus of claim 17 , wherein, for the determining of the second temporary bounding box, the one or more processors are configured to, in response to the ratio of the input bounding box to the first temporary bounding box being greater than the threshold, determine the second temporary bounding box based on the objectness probability map and the bounding box map.

19. The apparatus of claim 17 , wherein, for the determining of the refined bounding box by comparing the first temporary bounding box and the second temporary bounding box, the one or more processors are configured to:

calculate a first intersection of union (IOU) of the first temporary bounding box and the input bounding box,

calculate a second IOU of the second temporary bounding box and the input bounding box, and

determine the refined bounding box by comparing the first IOU and the second IOU.

20. The apparatus of claim 19 , wherein, for the determining of the refined bounding box by comparing the first IOU and the second IOU, the one or more processors are configured to:

in response to the first IOU being greater than the second IOU, determine the first temporary bounding box to be the refined bounding box, and

in response to the second IOU being greater than or equal to the first IOU, determine the second temporary bounding box to be the refined bounding box.

21. A processor-implemented method with object tracking, comprising:

generating an objectness probability map and a bounding box map by performing objectness-based bounding box regression analysis based on an input bounding box and an input image;

determining a first temporary bounding box and a second temporary bounding box based on the maps;

determining a refined bounding box to be either one of the first temporary bounding box and the second temporary bounding box by comparing the temporary bounding boxes; and

performing object tracking based on the refined bounding box.

22. The method of claim 21 , wherein a proportion of the refined bounding box that is a background region is less than a proportion of the input bounding box that is the background region.

23. The method of claim 21 , wherein the determining of first temporary bounding boxes comprises determining the second temporary bounding box in response to an area ratio of the input bounding box to the first temporary bounding box being greater than the threshold.

24. The method of claim 21 , wherein the determining of the refined bounding box comprises determining refined bounding box to be the second temporary bounding in response to an intersection of union (IOU) of the second temporary bounding box and the input bounding box being greater than or equal to an IOU of the first temporary bounding box and the input bounding box.

25. The method of claim 21 , wherein the performing of the object tracking comprises determining a bounding box in a subsequent image based on the refined bounding box.