Method for re-recognizing object image based on multi-feature information capture and correlation analysis

1. A method for re-recognizing an object image based on a multi-feature information capture and correlation analysis comprising:

a) collecting a plurality of object images to form an object image re-recognition database, labeling identifier (ID) information of an object image in the object image re-recognition database, and dividing the object image re-recognition database into a training set and a test set;

b) establishing an object image re-recognition model by using the multi-feature information capture and correlation analysis;

c) optimizing an objective function of the object image re-recognition model by using a cross-entropy loss function and a triplet loss function to obtain an optimized object image re-recognition model;

d) marking the object images with the ID information to obtain marked object images, inputting the marked object images into the optimized object image re-recognition model in step c) for training to obtain a trained object image re-recognition model and storing the trained object image re-recognition model;

e) inputting a to-be-retrieved object image into the trained object image re-recognition model in step d) to obtain a feature of a to-be-retrieved object; and

f) comparing the feature of the to-be-retrieved object with features of the object images in the test set and sorting comparison results by a similarity measurement

wherein step b) comprises the following steps:

b-1) setting an image input network to two branch networks comprising a first feature branch network and a second feature branch network;

b-2) inputting an object image h in the training set into the first feature branch network, wherein h∈ e×w×3 , represents a real number space, e represents a number of horizontal pixels of the object image h, w represents a number of vertical pixels of the object image h, and 3 represents a number of channels of each red, green, and blue (RGB) image; processing the object image h by using a convolutional layer to obtain a feature map f; processing the feature map f by using a channel attention mechanism; performing a global average pooling and a global maximum pooling on the feature map f to obtain two one-dimensional vectors; normalizing the two one-dimensional vectors through a convolution, a Rectified Linear Unit (ReLU) activation function, a 1*1 convolution, and sigmoid function operations in turn to weight the feature map f to obtain a weighted feature map f; performing a maximum pooling and an average pooling on all channels at each position in the weighted feature map f by using a spatial attention mechanism to obtain a maximum pooled feature map and an average pooled feature map; stitching the maximum pooled feature map and the average pooled feature map to obtain a stitched feature map; performing a 7*7 convolution on the stitched feature map, and then normalizing the stitched feature map by using a batch normalization layer and a sigmoid function to obtain a normalized stitched feature map; and multiplying the normalized stitched feature map by the feature map f to obtain a new feature;

b-3) inputting the object image h in the training set into the second feature branch network, wherein h∈ e×w×3 ; dividing the image h into n two-dimensional blocks; representing embeddings of the two-dimensional blocks as a one-dimensional vector h l ∈ n×(p 2 ·3) by using a linear transformation layer, wherein P represents a resolution of an image block, and n=ew/p 2 ; calculating an average embedding h a of all the two-dimensional blocks according to a formula

h

a

=

∑

i

n

⁢

h

i

n

,

 wherein h i represents an embedding of an i th block obtained through a Gaussian distribution initialization, and i∈{1, . . . , n}; calculating an attention coefficient a i of the i th block according to a formula a i =q T σ(W 1 h 0 +W 2 h i +W 3 h a ), wherein q T represents a weight, σ represents the sigmoid function, h 0 represents a class marker, and W 1 , W 2 , and W 3 are weights; calculating a new embedding h l of each of the two-dimensional blocks according to a formula

h

l

=

∑

i

=

1

n

⁢

a

i

⁢

h

i

n

;

 and calculating a new class marker h′ 0 according to a formula h′ 0 =W 4 [h 0 ∥h 1 ], wherein W 4 represents a weight;

b-4) taking the new class marker h′ 0 and a sequence with an input size of h l ∈ n×d c as an overall representation of a new image, wherein d c =d*m, d represents a dimension size of a head of each self-attention mechanism in a multi-head attention mechanism, and m represents a number of heads of the multi-head attention mechanism; adding position information in the new image, and then taking the new image as an input of a transformer encoder to complete the establishment of the object image re-recognition model.

2. The method for re-recognizing the object image based on the multi-feature information capture and correlation analysis according to claim 1 , wherein the transformer encoder in step b-4) comprises the multi-head attention mechanism and a feedforward layer;

the multi-head attention mechanism comprises a plurality of self-attention mechanisms; a weight Attention (h i , i) of an i th value in the sequence h l ∈ n×d is calculated according to a formula

Attention

⁢

(

h

l

,

i

)

=

Softmax

⁢

(

Q

i

T

⁢

K

i

d

)

⁢

V

i

,

 wherein Q represents an i th queried vector, T represents an inversion, K i represents a vector of a correlation between i th queried information and queried information from other blocks of the two-dimensional blocks, and V i represents a vector of the i th queried information; a new output embedding SA(h l ) of the multi-head attention mechanism is calculated according to a formula SA(h l )=Proj (Concat i-1 m (Attention (h l ,i); an input h′ of the feedforward layer is calculated according to a formula h′=ωLN(h l +SA(h l )); an output y of the transformer encoder is calculated according to a formula y=ωLN(h′+FFN(h′)), wherein Proj(·) represents a linear mapping, Concat(·) represents a stitching operation, FFN(h′)=∂W 2 (h l W 1 +c 1 )+c 2 , ∂ represents a Gaussian Error Linear Unit (GELU) activation function, c 1 and c 2 are learnable offsets, ω represents a ratio, LN represents a normalization operation; and the new feature output from the first feature branch network and a feature y output from the second feature branch network are stitched into a feature vector of the object image.

3. The method for re-recognizing the object image based on the multi-feature information capture and correlation analysis according to claim 1 , wherein in step c), a cross-entropy loss V ID is calculated according to a formula

V

ID

=

-

∑

i

=

1

n

⁢

g

i

⁢

log

⁡

(

p

i

)

⁢

{

g

i

=

0

,

y

≠

i

g

i

=

1

,

y

=

i

,

wherein g i represents an indicator variable, n represents a number of classes in the training set, and P i represents a predicted probability of a class-i image; and the triplet loss function V t is calculated according to a formula V t =[∥v a −v p ∥ 2 −v a −v n ∥ 2 +α] + , wherein α represents a spacing, v a represents a sample of a class marker learned by a transformer, v p represents a positive sample of the class marker learned by the transformer, v n represents a negative sample of the class marker learned by the transformer, [d] + is max[d,0], and d=∥v a −v p ∥ 2 −∥v a −v n ∥ 2 +α.