Method for correlation filter based visual tracking

1. A method for learning deep convolutional features specifically designed for correlation filter based visual tracking, the method comprising steps of:

selecting a first image from a first image patch;

selecting a second image from a second image patch;

forward propagating the selected first image by a convolutional neural network model formula, wherein the convolutional neural network model formula has random weights with a zero mean for a plurality of parameters;

forward propagating the selected second image by the convolutional neural network model formula;

computing a correlation filter using the forward propagated second image and a centered correlation response;

circularly correlating the forward propagated first image and a computed correlation filter to generate a predicted response map;

calculating a loss by comparing the predicted response map with a desired correlation corresponding the selected first image and the second image and updating the plurality of parameters of the convolutional neural network model formula according to the calculated loss, wherein, the step of calculating the loss is performed by the following formula;

L

i

⁡

(

θ

)

=



∑

l

=

1

d

⁢

h

i

l

⁡

(

θ

)

⁢

⁢

⁢

⁢

x

i

l

⁡

(

θ

)

-

g

i



2

wherein L i (θ) represents the loss value for the training samples x i , y i and g i , h i l (θ) represents computed correlation filter for an l th feature map ∀l∈{1, . . . , d}, x i l (θ) represents an output of the forward propagated first image (x i )∀l∈{1, . . . , d}, {circle around (*)} represents a circularly correlation operation and g i represents the desired correlation.

2. The method according to claim 1 , wherein, the gradient is calculated with the following formula using a chain rule;

∇

θ

⁢

L

i

=

∑

l

⁢

dL

dx

i

l

⁢

dx

i

l

d

⁢

⁢

θ

+

∑

l

⁢

dL

dy

i

l

⁢

dy

i

l

d

⁢

⁢

θ

.

3. The method according to claim 1 , wherein, the step of updating the parameters of the convolutional neural network model formula according to the calculated loss further comprises following formula;

θ

←

θ

-

μ

⁢

⁢

dL

i

d

⁢

⁢

θ

wherein μ is a learning rate and

dL

i

d

⁢

⁢

θ

is a mean gradient for the model.

4. A method for generating the predicted response map using the convolutional neural network model formula, with the plurality of parameters updated according to the method for learning deep convolutional features specifically designed for correlation filter based visual tracking according to claim 1 , comprising steps of:

selecting the first image from the first image patch;

selecting the second image from the second image patch;

forward propagating the selected first image by the convolutional neural network model formula, with the plurality of updated parameters;

forward propagating the selected second image by the convolutional neural network model formula with the plurality of updated parameters;

computing the correlation filter using the forward propagated second image and the centered correlation response;

circularly correlating the forward propagated first image and the computed correlation filter to generate the predicted response map

wherein any layer of the convolutional neural network model formula with the plurality of updated parameters is used in the steps of the forward propagating first image and the forward propagating second image.

5. The method according to claim 4 , wherein, when different layers of the convolutional neural network model formula with the plurality of updated parameters (θ) generate a plurality of images with different sizes, resizing all the plurality of generated images are to a same size by a resizing operation.

6. A method for learning deep convolutional features specifically designed for correlation filter based visual tracking, the method comprising steps of:

selecting a first image from a first image patch;

selecting a second image from a second image patch;

forward propagating the selected first image by a convolutional neural network model formula, wherein the convolutional neural network model formula has random weights with a zero mean for a plurality of parameters;

forward propagating the selected second image by the convolutional neural network model formula;

computing a correlation filter using the forward propagated second image and a centered correlation response, wherein the computing correlation filter using the forward propagated second image is performed by the following formula

h

i

l

=

F

-

1

⁢

{

Y

i

l

*

⊙

C

i

l

∑

m

=

1

d

⁢

Y

i

m

⊙

Y

i

m

*

+

λ

}

wherein, h i l represents correlation filter for an l th feature channel of the computed correlation filter h i , F −1 represents an inverse discrete fourier transform (DFT) operation, C i represents a signal of the centered correlation response in DFT domain, Y i l represents an l th feature channel of the second image in DFT domain and λ represents a regularization parameter;

circularly correlating the forward propagated first image and a computed correlation filter to generate a predicted response map;

calculating a loss by comparing the predicted response map with a desired correlation corresponding the selected first image and the second image and updating the plurality of parameters of the convolutional neural network model formula according to the calculated loss.

7. The method according to claim 6 , wherein, after updating the parameters of convolutional neural network model formula, repeating the steps of forward propagating the first image, forward propagating the second image, computing the plurality of correlation filters, the circular convolution in frequency domain using DFT, generating the predicted response map, calculating the loss and updating the plurality of model parameters using iterated with a plurality of updated parameters at least once.

8. The method according to claim 6 , wherein, the convolutional neural network model formula is a deep fully convolutional network.

9. The method according to claim 6 , wherein, the step of circularly correlating the forward propagated first image and the computed correlation filter is performed by the following formula;

c

⁡

[

n

]

=

∑

i

⁢

a

⁡

[

i

]

⁢

b

⁡

[

n

+

i

]

=

F

-

1

⁢

{

A

*

⊙

B

}

wherein F −1 {.} represents an inverse DFT operation, b[n+i] represents a circularly shifted version of the signal b[n] by i amount to the left, and ⊙ is an element wise multiplication operation.

10. The method according to claim 6 , wherein, the step of updating the parameters of the convolutional neural network model formula according to the calculated loss comprises calculating a loss gradient.