Method and apparatus with target object tracking
A processor-implemented method with target object tracking includes: setting a search area for a target object included in an input image based on a position of a first target box in a template image; selecting a network path from a plurality of network paths of a neural network model according to a resizing ratio of a size of another image that is input to the neural network model to a size of the search area; and tracking the target object by estimating a position of a second target box corresponding to the target object in the input image according to the selected network path.
1. A processor-implemented method with target object tracking, the method comprising:
setting a search area for a target object included in an input image based on a position of a first target box in a template image;
selecting a network path from a plurality of network paths of a neural network model according to a result of a comparison between a resizing ratio and a predetermined threshold value, the resizing ratio corresponding to a ratio of a size of another image that is input to the neural network model to a size of the search area; and
tracking the target object by estimating a position of a second target box corresponding to the target object in the input image according to the selected network path.
2. The method of claim 1 , wherein the selecting of the network path comprises:
determining the resizing ratio based on a value obtained by dividing the size of the other image by the size of the search area; and
selecting a network path determined according to the resizing ratio from the plurality of network paths, according to a result of a comparison between the resizing ratio and a set threshold value.
3. The method of claim 2 , wherein the plurality of network paths comprise blocks configured as convolutional layers respectively corresponding to different kernel sizes.
4. The method of claim 2 , wherein the selecting of the network path determined according to the resizing ratio comprises:
selecting a first network path corresponding to a first kernel size, in response to a determination that the resizing ratio is greater than the threshold value;
selecting a second network path corresponding to a second kernel size, in response to a determination that the resizing ratio is equal to the threshold value; and
selecting a third network path corresponding to a third kernel size, in response to the resizing ratio being less than the threshold value.
5. The method of claim 4 , wherein
the second kernel size is larger than the first kernel size, and
the third kernel size is larger than the second kernel size.
6. The method of claim 1 , wherein the neural network model comprises a Super Net having separate convolutional layers configured for a plurality of kernel sizes.
7. The method of claim 1 , wherein the tracking of the target object comprises:
generating a second feature map from the search area resized according to the selected network path; and
estimating the position of the second target box corresponding to the target object in the input image using a first feature map of the template image and the second feature map.
8. The method of claim 7 , wherein the estimating of the position of the second target box comprises:
determining a similarity between the first feature map and the second feature map; and
estimating the position of the second target box corresponding to the target object in the input image based on the similarity.
9. The method of claim 1 , further comprising storing the position of the second target box to set the search area in an input image subsequent to the input image.
10. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .
11. An apparatus with target object tracking, the apparatus comprising:
one or more processors configured to:
set a search area for a target object included in an input image based on the position of the first target box;
select a network path from a plurality of network paths of a neural network model according to a result of a comparison between a resizing ratio and a predetermined threshold value, the resizing ratio corresponding to a ratio of a size of another image that is input to the neural network model to a size of the search area; and
track the target object by estimating a position of a second target box corresponding to the target object in the input image based on the selected network path.
12. The apparatus of claim 11 , wherein, for the selecting of the network path, the one or more processors are further configured to:
determine the resizing ratio based on a value obtained by dividing the size of the other image by the size of the search area; and
select a network path determined according to the resizing ratio from the plurality of network paths, according to a result of a comparison between the resizing ratio and a set threshold value.
13. The apparatus of claim 12 , wherein the plurality of network paths comprise blocks configured as convolutional layers respectively corresponding to different kernel sizes.
14. The apparatus of claim 12 , wherein, for the selecting of the network path determined according to the resizing ratio, the one or more processors are further configured to:
select a first network path corresponding to a first kernel size, in response to the resizing ratio being greater than the threshold value;
select a second network path corresponding to a second kernel size, in response to the resizing ratio being equal to the threshold value; and
select a third network path corresponding to a third kernel size, in response to the resizing ratio being less than the threshold value.
15. The apparatus of claim 14 , wherein
the second kernel size is larger than the first kernel size, and
the third kernel size is larger than the second kernel size.
16. The apparatus of claim 11 , wherein the neural network model comprises a Super Net having separate convolutional layers configured for a plurality of kernel sizes.
17. The apparatus of claim 11 , wherein, for the tracking of the target object, the one or more processors are further configured to:
generate a second feature map from the search area resized according to the selected network path; and
estimate the position of the second target box corresponding to the target object in the input image using the first feature map and the second feature map.
18. An electronic device, comprising:
a camera configured to capture either one or both of a template image and an input image; and
one or more processors configured to:
set a search area for a target object included in the input image based on a position of a first target box in the template image;
select a network path from a plurality of network paths of a neural network model according to a result of a comparison between a resizing ratio and a predetermined threshold value, the resizing ratio corresponding to a ratio of a size of another image that is input to the neural network model to a size of the search area; and
perform either one or both of auto focusing and auto zooming on the target object by estimating a position of a second target box corresponding to the target object in the input image by resizing the search area according to the selected network path.
19. The electronic device of claim 18 , wherein, for the selecting of the network path, the one or more processors are further configured to:
determine the resizing ratio between the size of the other image and the size of the search area; and
select a network path determined according to the resizing ratio from the plurality of network paths, according to a result of a comparison between the resizing ratio and a set threshold value.
20. The electronic device of claim 19 , wherein, for the selecting of the network path determined according to the resizing ratio, the one or more processors are further configured to:
select a first network path corresponding to a first kernel size, in response to the resizing ratio being greater than the threshold value;
select a second network path corresponding to a second kernel size larger than the first kernel size, in response to the resizing ratio being equal to the threshold value; and
select a third network path corresponding to a third kernel size larger than the second kernel size, in response to the resizing ratio being less than the threshold value.