Video feature extraction method, device and computer-readable storage medium
A method for extracting video features may include: obtaining a target video sequence that comprises a number of video frames; performing video frame feature extraction on the target video sequence to obtain video frame features of each of the video frames; performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features; wherein the feature weight of each of the video frame features is determined by the video frame features of all of the video frames in the target video sequence; and performing feature weighting on each of the video frame features according to the feature weight of each of the video frame features to obtain video features of the target video sequence.
1 . A computer-implemented video feature extraction method, the method comprising:
obtaining a target video sequence that comprises a plurality of video frames;
performing video frame feature extraction on the target video sequence to obtain video frame features of each of the plurality of video frames;
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features; wherein the feature weight of each of the video frame features is determined by the video frame features of all of the video frames in the target video sequence, and wherein the feature weight of any video frame feature is an average of feature similarities between the same video frame feature and the video frame features of all of the video frames in the target video sequence; and
performing feature weighting on each of the video frame features according to the feature weight of each of the video frame features to obtain video features of the target video sequence.
2 . The method of claim 1 , wherein performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features comprises:
calculating a similarity of each of the video frame features to obtain a feature similarity matrix; and
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features according to the feature similarity matrix.
3 . The method of claim 2 , wherein calculating the similarity of each of the video frame features to obtain the feature similarity matrix comprises:
performing a convolution operation on each of the video frame features to obtain a feature vector of each of the video frame features; and
calculating the similarity of each of the video frame features to obtain the feature similarity matrix according to the feature vector of each of the video frame features.
4 . The method of claim 3 , wherein the similarity is calculated according to the following equation:
K
i
,
j
=
f
(
F
i
)
f
(
F
j
)
f
(
F
i
)
f
(
F
j
)
,
where K i,j represents the feature similarity between an i-th video frame feature F i and a j-th video frame feature F j , f(F i ) represents the feature vector of the video frame feature F i , f(F j ) represents the feature vector of the video frame feature F j , ∥*∥ represents a norm of a vector *.
5 . The method of claim 4 , wherein the feature weight is calculated according to the following equation:
s
i
=
1
T
∑
t
=
1
T
K
i
,
t
,
where s i is the feature weight of the i-th video frame feature F i , K i,t is a t-th feature similarity in an i-th row in the feature similarity matrix, and T represents an amount of the video frames in the target video sequence.
6 . The method of claim 5 , wherein the feature weighting is performed according to the following equation:
F
=
1
T
∑
t
=
1
T
s
t
F
t
,
where F represents the video features of the target video sequence, F t represents a t-th video frame feature, and s t is the feature weight of the video frame feature F t .
7 . The method of claim 1 , wherein performing video frame feature extraction on the target video sequence to obtain video frame features of each of the plurality of video frame comprises:
performing video frame feature extraction on each of the video frames in the target video sequence to obtain the video frame features of each video frame using a preset video frame feature extraction network; wherein the video frame feature extraction network is a deep neural network for video frame feature extraction.
8 . A device for extracting video features comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing programs that, when executed by the one or more processors, cause performance of operations comprising:
obtaining a target video sequence that comprises a plurality of video frames;
performing video frame feature extraction on the target video sequence to obtain video frame features of each of the plurality of video frames;
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features; wherein the feature weight of each of the video frame features is determined by the video frame features of all of the video frames in the target video sequence, and wherein the feature weight of any video frame feature is an average of feature similarities between the same video frame feature and the video frame features of all of the video frames in the target video sequence; and
performing feature weighting on each of the video frame features according to the feature weight of each of the video frame features to obtain video features of the target video sequence.
9 . The device of claim 8 , wherein performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features comprises:
calculating a similarity of each of the video frame features to obtain a feature similarity matrix; and
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features according to the feature similarity matrix.
10 . The device of claim 9 , wherein calculating the similarity of each of the video frame features to obtain the feature similarity matrix comprises:
performing a convolution operation on each of the video frame features to obtain a feature vector of each of the video frame features; and
calculating the similarity of each of the video frame features to obtain the feature similarity matrix according to the feature vector of each of the video frame features.
11 . The device of claim 10 , wherein the similarity is calculated according to the following equation:
K
i
,
j
=
f
(
F
i
)
f
(
F
j
)
f
(
F
i
)
f
(
F
j
)
,
where K i,j represents the feature similarity between a i-th video frame feature F i and a j-th video frame feature F j , f(F i ) represents the feature vector of the video frame feature F i , f(F j ) represents the feature vector of the video frame feature F j , ∥*∥ represents a norm of a vector *.
12 . The device of claim 11 , wherein the feature weight is calculated according to the following equation:
s
i
=
1
T
∑
t
=
1
T
K
i
,
t
,
where s i is the feature weight of the i-th video frame feature F i , K i,t is a t-th feature similarity in an i-th row in the feature similarity matrix, and T represents an amount of the video frames in the target video sequence.
13 . The device of claim 12 , wherein the feature weighting is performed according to the following equation:
F
=
1
T
∑
t
=
1
T
s
t
F
t
,
where F represents the video features of the target video sequence, F t represents a t-th video frame feature, and s t is the feature weight of the video frame feature F t .
14 . The device of claim 8 , wherein performing video frame feature extraction on the target video sequence to obtain video frame features of each of the plurality of video frame comprises:
performing video frame feature extraction on each of the video frames in the target video sequence to obtain the video frame features of each video frame using a preset video frame feature extraction network; wherein the video frame feature extraction network is a deep neural network for video frame feature extraction.
15 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a device, cause the at least one processor to perform a method, the method comprising:
obtaining a target video sequence that comprises a plurality of video frames;
performing video frame feature extraction on the target video sequence to obtain video frame features of each of the plurality of video frames;
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features; wherein the feature weight of each of the video frame features is determined by the video frame features of all of the video frames in the target video sequence, and wherein the feature weight of any video frame feature is an average of feature similarities between the same video frame feature and the video frame features of all of the video frames in the target video sequence; and
performing feature weighting on each of the video frame features according to the feature weight of each of the video frame features to obtain video features of the target video sequence.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features comprises:
calculating a similarity of each of the video frame features to obtain a feature similarity matrix; and
performing feature weight calculation on each of the video frame features to obtain the feature weight of each of the video frame features according to the feature similarity matrix.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein calculating the similarity of each of the video frame features to obtain the feature similarity matrix comprises:
performing a convolution operation on each of the video frame features to obtain a feature vector of each of the video frame features; and
calculating the similarity of each of the video frame features to obtain the feature similarity matrix according to the feature vector of each of the video frame features.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein the similarity is calculated according to the following equation:
K
i
,
j
=
f
(
F
i
)
f
(
F
j
)
f
(
F
i
)
f
(
F
j
)
,
where K i,j represents the feature similarity between a i-th video frame feature F j and a j-th video frame feature F j , f(F i ) represents the feature vector of the video frame feature F i , f(F j ) represents the feature vector of the video frame feature F j , ∥*∥ represents a norm of a vector *.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein the feature weight is calculated according to the following equation:
s
i
=
1
T
∑
t
=
1
T
K
i
,
t
,
where s i is the feature weight of the i-th video frame feature F i , K i,t is a t-th feature similarity in an i-th row in the feature similarity matrix, and T represents an amount of the video frames in the target video sequence.
20 . The non-transitory computer-readable storage medium of claim 19 , wherein the feature weighting is performed according to the following equation:
F
=
1
T
∑
t
=
1
T
s
t
F
t
,
where F represents the video features of the target video sequence, F t represents a t-th video frame feature, and s t is the feature weight of the video frame feature F t .