Image processing apparatus, image processing method, and non-transitory computer-readable storage medium
An apparatus sets coefficients in a first array based on first information indicating an image capturing orientation of a first image, generates a first map by applying the coefficients to the first image, acquires a template feature corresponding to an object based on the first map, registers the template feature in an array based on the first information, sets coefficients in a second array based on second information indicating an image capturing orientation of a second image, generates a second map by applying the coefficients set in the second array to the second image, sets the template feature in a feature array based on the second information, performs a correlation calculation between the template feature set in the feature array and the second map, and detects the object from the second image based on a result of the correlation calculation.
1 . An image processing apparatus comprising:
a controller comprising one or more processors and/or one or more circuits, the controller being configured to function as a plurality of units comprising:
(1) a first generation unit configured to (a) set filter coefficients in a first coefficient array based on first orientation information indicating an image capturing orientation of a first captured image and (b) generate a first feature map by applying the filter coefficients set in the first coefficient array to the first captured image;
(2) a registration unit configured to (a) acquire, based on the first feature map, a template feature corresponding to a target object and (b) register the template feature in a first feature array based on the first orientation information;
(3) a second generation unit configured to (a) set filter coefficients in a second coefficient array based on second orientation information indicating an image capturing orientation of a second captured image and (b) generate a second feature map by applying the filter coefficients set in the second coefficient array to the second captured image;
(4) a calculation unit configured to (a) set the registered template feature in a second feature array based on the second orientation information and (b) perform a correlation calculation between the template feature set in the second feature array and the second feature map; and
(5) a detection unit configured to detect, based on a result of the correlation calculation, the target object from the second captured image,
wherein the first generation unit sets the filter coefficients in the first coefficient array by rotating the filter coefficients in accordance with the first orientation information, and
wherein the registration unit registers the acquired template feature in the first feature array by rotating the acquired template feature inversely to the rotation of the filter coefficients by the first generation unit.
2 . The image processing apparatus according to claim 1 , wherein the second generation unit sets the filter coefficients in the second coefficient array by rotating the filter coefficients in accordance with the second orientation information, and
wherein the calculation unit sets the registered template feature in the second feature array by rotating the registered template feature in accordance with the rotation of the filter coefficients by the second generation unit.
3 . The image processing apparatus according to claim 1 , wherein the registration unit acquires a feature within a region of the target object in the first feature map as a template feature.
4 . The image processing apparatus according to claim 1 , wherein the first generation unit generates the first feature map based on a convolution calculation between the filter coefficients that are set in the first coefficient array and the first captured image, and
wherein the second generation unit generates the second feature map based on a convolution calculation between the filter coefficients that are set in the second coefficient array and the second captured image.
5 . The image processing apparatus according to claim 4 , wherein the convolution calculation is executed by using a hierarchical neural network.
6 . The image processing apparatus according to claim 5 , wherein the convolution calculation is performed in prescribed units for each layer of the hierarchical neural network.
7 . The image processing apparatus according to claim 1 , wherein the calculation unit performs the correlation calculation by a convolution calculation between the second feature map and the template feature set in the second feature array.
8 . The image processing apparatus according to claim 1 , wherein the detection unit generates, based on the result of the correlation calculation, a detection map indicating a likelihood of a position of the target object in the second captured image.
9 . The image processing apparatus according to claim 8 , wherein the plurality of units further comprises a control unit configured to perform, in accordance with the detection map, control according to image capturing.
10 . The image processing apparatus according to claim 9 , wherein the control unit performs control in order to track and capture the target object.
11 . The image processing apparatus according to claim 1 , wherein the plurality of units further comprises a unit configured to acquire images captured as the first captured image and the second captured image.
12 . The image processing apparatus according to claim 1 , wherein the plurality of units further comprises a unit configured to acquire the first orientation information and the second orientation information.
13 . An image processing method performed by an image processing apparatus, the method comprising:
setting filter coefficients in a first coefficient array based on first orientation information indicating an image capturing orientation of a first captured image;
generating a first feature map by applying the filter coefficients set in the first coefficient array to the first captured image;
acquiring, based on the first feature map, a template feature corresponding to a target object;
registering the template feature in a first feature array based on the first orientation information;
setting filter coefficients in a second coefficient array based on second orientation information indicating an image capturing orientation of a second captured image;
generating a second feature map by applying the filter coefficients set in the second coefficient array to the second captured image;
setting the registered template feature in a second feature array based on the second orientation information;
performing a correlation calculation between the template feature set in the second feature array and the second feature map; and
detecting, based on a result of the correlation calculation, the target object from the second captured image,
wherein the setting filter coefficients in the first coefficient array sets the filter coefficients in the first coefficient array by rotating the filter coefficients in accordance with the first orientation information, and
wherein the registering the template feature registers the acquired template feature in the first feature array by rotating the acquired template feature inversely to the rotation of the filter coefficients.
14 . A non-transitory computer-readable storage medium that stores a computer program to cause a computer to function as a plurality of units,
wherein the computer comprises one or more processors and/or one or more circuits, and the plurality of units comprises: (1) a first generation unit configured to (a) set filter coefficients in a first coefficient array based on first orientation information indicating an image capturing orientation of a first captured image and (b) generate a first feature map by applying the filter coefficients set in the first coefficient array to the first captured image; (2) a registration unit configured to (a) acquire, based on the first feature map, a template feature corresponding to a target object and (b) register the template feature in a first feature array based on the first orientation information; (3) a second generation unit configured to (a) set filter coefficients in a second coefficient array based on second orientation information indicating an image capturing orientation of a second captured image and (b) generate a second feature map by applying the filter coefficients set in the second coefficient array to the second captured image; (4) a calculation unit configured to (a) set the registered template feature in a second feature array based on the second orientation information and (b) perform a correlation calculation between the template feature set in the second feature array and the second feature map; and (5) a detection unit configured to detect, based on a result of the correlation calculation, the target object from the second captured image,
wherein the first generation unit sets the filter coefficients in the first coefficient array by rotating the filter coefficients in accordance with the first orientation information, and
wherein the registration unit registers the acquired template feature in the first feature array by rotating the acquired template feature inversely to the rotation of the filter coefficients by the first generation unit.