Granted Patent B2

US 12,633,091 · App. 18/734,165 · Granted May 19, 2026

Inventors: Niclas Danielsson (Lund, SE); Amanda Nilsson (Lund, SE); Christian Colliander (Lund, SE); Sarah Laross (Lund, SE)

Assignee: AXIS AB

G06V10/7715G06T3/4038G06V10/82

Abstract

Claims (46)

1 . A method for feature extraction for object re-identification or object classification of detected objects, comprising the steps of:

from one or more image frames, determining a plurality of images, each image depicting an object detected by an object detecting application, each image corresponding to a cut-out from an image frame among the one or more image frames, the cut-out corresponding to the detected object;

concatenating the plurality of images into a composite image according to a 2D grid pattern wherein each cell of the grid pattern accommodates one image of the plurality of images;

feeding the composite image through a convolutional neural network, (CNN), trained for feature extraction for object re-identification or object classification, wherein each convolutional layer of the CNN is configured to, while convolving input data to the convolutional layer using a convolutional filter:

determine a currently convolved image of the plurality of images by determining a centre coordinate of a subset of the input data, the subset of input data currently covered by the convolutional filter, wherein the centre coordinate corresponds to an anchor point of the convolutional filter, and mapping the centre coordinate to the grid pattern; and

selectively nullifying all weights of the convolutional filter that cover input data derived from any of the plurality of images not being the currently convolved image.

2 . The method of claim 1 , further comprising the step of:

associating the convolutional layer with a filter map which for each possible centre coordinate of the convolutional filter when convolving the input data is indicative of which weights of the convolutional filter that cover input data derived from any of the plurality of images not being the currently convolved image,

wherein the step of selectively nullifying all weights comprises:

selectively nullifying weights of the convolutional filter that for the determined centre coordinate are indicated by the filter map to cover input data derived from any of the plurality of images not being the currently convolved image.

3 . The method of claim 2 , wherein the step of associating the convolutional layer with a filter map comprises:

associating the convolutional layer with a plurality of binary masks, each binary mask being associated with one of the plurality of images and having coordinates corresponding to the coordinates of the input data, wherein a binary mask being associated with a specific image from the plurality of images has a first binary value for all coordinates of the input data that are derived from the specific image, and a second binary value for all remaining coordinates of the input data;

wherein the step of selectively nullifying all weights comprises:

selecting a binary mask from the plurality of binary masks, wherein the selected binary mask is associated with the currently convolved image; and

selectively nullifying weights of the convolutional filter by masking the convolutional filter using a sub-mask of the selected binary mask corresponding to the subset of the input data currently covered by the convolutional filter.

4 . The method of claim 3 , wherein the filter map has coordinates corresponding to possible centre coordinates of the convolutional filter when convolving the input data, and wherein the filter map comprises, at each coordinate of the filter map, an indication of which coordinates of the convolutional filter that should have their weights nullified when the centre coordinate corresponds to that coordinate of the filter map.

5 . The method of claim 2 , wherein the indication comprises a first value, i, indicating which columns of the convolutional filter that should be nullified, and a second value, j, indicating which rows of the convolutional filter that should be nullified.

6 . The method of claim 5 ,

wherein a positive value of i indicates that a rightmost i column(s) of the convolutional filter should be nullified, and wherein a negative value of i indicates that a leftmost i column(s) of the convolutional filter should be nullified, or vice versa; and

wherein a positive value of j indicates that a bottom j row(s) of the convolutional filter should be nullified, and wherein a negative value of j indicates that a top j row(s) of the convolutional filter should be nullified, or vice versa.

7 . The method of claim 2 , wherein the CNN comprises at least a first and a second convolutional layer, wherein output from the first convolutional layer is input to the second convolutional layer, wherein the step of associating the second convolutional layer with a filter map comprises sampling the filter map associated with the first convolutional layer at least in part according to a stride of the first convolutional layer and associating the sampled filter map with the second convolutional layer.

8 . The method of claim 1 , wherein the CNN comprises at least a first and a second convolutional layer, wherein the composite image is input to the first convolution layer, wherein output from the first convolutional layer is input to the second convolutional layer, wherein mapping the centre coordinate to the grid pattern for the second convolution layer comprises sampling the grid pattern at least in part according to a stride of the first convolutional layer and mapping the centre coordinate to the sampled grid pattern.

9 . The method of claim 1 , wherein the CNN comprises at least a first and a second convolutional layer, wherein the composite image is input to the first convolution layer, wherein output from the first convolutional layer is input to the second convolutional layer, wherein mapping the centre coordinate to the grid pattern for the second convolutional layer comprises determining a centre coordinate in the input to the first convolutional layer that corresponds to the centre coordinate at least in part according to a stride of the first convolutional layer and mapping the determined centre coordinate to the grid pattern.

10 . The method of claim 1 , wherein the CNN comprises one or more pooling layers, wherein a stride of the pooling layer is adapted based on a size of the input data and the grid pattern such that the pooling layer outputs one pooled feature map for each of the plurality of images.

11 . The method of claim 1 , wherein a pooling layer outputs one pooled feature map for each of the plurality of images, wherein the method further comprises:

flatten each of the pooled feature maps into a corresponding flattened vector; and

separately apply a fully connected layer on each of the flattened vectors.

12 . The method of claim 1 , wherein the plurality of images originates from a plurality of image frames from one or more video sequences.

13 . The method of claim 1 , further comprising the step of:

outputting, by the CNN, a feature vector for each of the plurality of images.

14 . A non-transitory computer-readable storage medium having stored thereon instructions for implementing a method, when executed on a device having processing capabilities, the method for feature extraction for object re-identification or object classification of detected objects, comprising the steps of:

from one or more image frames, determining a plurality of images, each image depicting an object detected by an object detecting application, each image corresponding to a cut-out from an image frame among the one or more image frames, the cut-out corresponding to the detected object;

concatenating the plurality of images into a composite image according to a 2D grid pattern wherein each cell of the grid pattern accommodates one image of the plurality of images;

feeding the composite image through a convolutional neural network, (CNN), trained for feature extraction for object re-identification or object classification, wherein each convolutional layer of the CNN is configured to, while convolving input data to the convolutional layer using a convolutional filter:

determine a currently convolved image of the plurality of images by determining a centre coordinate of a subset of the input data, the subset of input data currently covered by the convolutional filter, where in the centre coordinate corresponds to an anchor point of the convolutional filter, and mapping the centre coordinate to the grid pattern; and

selectively nullifying all weights of the convolutional filter that cover input data derived from any of the plurality of images not being the currently convolved image.

15 . A system for feature extraction for object re-identification or object classification of detected objects, comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computer

executable instructions that, when executed by the one or more processors, cause the

system to perform actions comprising:

from one or more image frames, determining a plurality of images, each image depicting an object detected by an object detecting application, each image corresponding to a cut-out from an image frame among the one or more image frames, the cut-out corresponding to the detected object;

concatenating the plurality of images into a composite image according to a 2D grid pattern, wherein each cell of the 2D grid pattern accommodates one image of the plurality of images;

feeding the composite image through a convolutional neural network, (CNN), trained for feature extraction for object re-identification or object classification, wherein each convolutional layer of the CNN is configured to, while convolving input data to the convolutional layer using a convolutional filter:

determine a currently convolved image of the plurality of images by determining a centre coordinate of a subset of the input data, the subset of data currently covered by the convolutional filter, wherein the centre coordinate corresponds to an anchor point of the convolutional filter, and mapping the centre coordinate to the grid pattern; and

selectively nullifying all weights of the convolutional filter that cover input data derived from any of the plurality of images not being the currently convolved image.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (16)