Method and device for motion vector prediction in video transcoding using full resolution residuals
View Patent ↗A transcoder and methods of encoding inter-prediction frames of a downsampled video wherein the downsampled video is a spatially downsampled version of a full-resolution video. Full-resolution motion vectors are downscaled and a weighting factor is calculated for each downscaled motion vector based upon the transform domain residual coefficients associated with that full-resolution motion vector. A motion vector prediction is made based on the weighted average using the downscaled motion vectors and their weighting factors.
1. A method of encoding a downsampled video, wherein the downsampled video is a spatially downsampled version of a full-resolution video, the downsampled video including a frame having a macroblock partitioned into at least one partition, one partition of the at least one partition corresponding to at least two full-resolution partitions in a corresponding frame of the full-resolution video, each of the at least two full-resolution partitions having an associated full-resolution motion vector relative to a reference frame, the method comprising:
downscaling the associated full-resolution motion vectors;
calculating a weighting factor for each of the downscaled full-resolution motion vectors, wherein each weighting factor is based upon transform domain residual coefficients associated with that full-resolution motion vector;
determining a motion vector prediction as the average of the product of each of the downscaled full-resolution motion vectors with its respective weighting factor;
selecting a desired motion vector for said one partition from a search area within the reference frame, the search area being centered at a point indicated by the motion vector prediction; and
encoding the downsampled video to generate an encoded downsampled video, including the desired motion vector for said one partition,
wherein there are k full-resolution partitions, and for each i th downscaled full-resolution motion vector said calculating comprises calculating the corresponding weighting factor w i in accordance with the expression:
w
i
=
D
C
i
+
∑
∀
j
∈
TCOEF
s
_
A
C
A
C
i
,
j
∑
∀
m
∈
k
(
D
C
m
+
∑
∀
j
∈
TCOEF
s
_
A
C
A
C
m
,
j
)
wherein DC i is the DC transform domain residual coefficient of the i th full-resolution partition, and AC i,j is the j th AC transform domain residual coefficient of the i th full-resolution partition, j being an index from 1 to the number of transform domain residual AC coefficients associated with the i th full-resolution partition, and m being an index from 1 to k.
2. The method claimed in claim 1 , wherein, for each downscaled full-resolution motion vector, its weighting factor is based upon the magnitude of transform domain residual coefficients associated with that downscaled full-resolution motion vector relative to the magnitude of transform domain residual coefficients associated with the other downscaled full-resolution motion vectors associated with the at least two full-resolution partitions.
3. The method claimed in claim 2 , wherein a first downscaled full-resolution motion vector has larger magnitude transform domain residual coefficients associated with it than the transform domain residual coefficients associated with a second downscaled full-resolution motion vector, and wherein said calculating includes calculating a first weighting factor for the first downscaled full-resolution motion vector that is larger than a second weighting factor for the second downscaled full-resolution motion vector.
4. The method claimed in claim 2 , wherein a first downscaled full-resolution motion vector has larger magnitude transform domain residual coefficients associated with it than the transform domain residual coefficients associated with a second downscaled full-resolution motion vector, and wherein said calculating includes calculating a first weighting factor for the first downscaled full-resolution motion vector that is smaller than a second weighting factor for the second downscaled full-resolution motion vector.
5. A method of encoding a downsampled video, wherein the downsampled video is a spatially downsampled version of a full-resolution video, the downsampled video including a frame having a macroblock partitioned into at least one partition, one partition of the at least one partition corresponding to at least two full-resolution partitions in a corresponding frame of the full-resolution video, each of the at least two full-resolution partitions having an associated full-resolution motion vector relative to a reference frame, the method comprising:
downscaling the associated full-resolution motion vectors;
calculating a weighting factor for each of the downscaled full-resolution motion vectors, wherein each weighting factor is based upon transform domain residual coefficients associated with that full-resolution motion vector;
determining a motion vector prediction as the average of the product of each of the downscaled full-resolution motion vectors with its respective weighting factor;
selecting a desired motion vector for said one partition from a search area within the reference frame, the search area being centered at a point indicated by the motion vector prediction; and
encoding the downsampled video to generate an encoded downsampled video, including the desired motion vector for said one partition,
wherein there are k full-resolution partitions, and for each i th downscaled full-resolution motion vector said calculating comprises calculating the corresponding weighting factor w i in accordance with the expression:
w
i
=
ⅇ
-
(
D
C
i
+
∑
∀
j
∈
TCOEFe
_
A
C
A
C
i
,
j
)
∑
∀
m
∈
k
(
ⅇ
-
(
D
C
m
+
∑
∀
j
∈
TCOEFa
_
A
C
A
C
m
,
j
)
)
wherein DC i is the DC transform domain residual coefficient of the i th full-resolution partition, and AC i,j is the j th AC transform domain residual coefficient of the i th full-resolution partition, j being an index from 1 to the number of transform domain residual AC coefficients associated with the i th full-resolution partition, and m being an index from 1 to k.
6. The method claimed in claim 1 , wherein the spatially downsampled video has been downsampled by a downsampling ratio, and wherein the downscaling comprises downscaling by the downsampling ratio.
7. The method claimed in claim 1 , wherein said selecting includes searching the search area for candidate motion vectors and selecting as the desired motion vector the candidate motion vector that minimizes a rate-distortion cost expression.
8. The method claimed in claim 1 , further including:
decoding a full-resolution encoded video to produce the full-resolution video, including decoding the at least two full-resolution partitions and their associated full-resolution motion vectors; and
spatially downsampling the full-resolution video in the pixel domain to produce the downsampled video.
9. An encoder for encoding a downsampled video, wherein the downsampled video is a spatially downsampled version of a full-resolution video, the downsampled video including a frame having a macroblock partitioned into at least one partition, one partition of the at least one partition corresponding to at least two full-resolution partitions in a corresponding frame of the full-resolution video, each of the at least two full-resolution partitions having an associated full-resolution motion vector relative to a reference frame, the encoder comprising:
a processor;
a memory;
a communications system for outputting an encoded downsampled video; and
an encoding application stored in memory and containing instructions for configuring the processor to encode the downsampled video by performing the method claimed in claim 1 .
10. The encoder claimed in claim 9 , wherein, for each downscaled full-resolution motion vector, its weighting factor is based upon the magnitude of transform domain residual coefficients associated with that downscaled full-resolution motion vector relative to the magnitude of transform domain residual coefficients associated with the other downscaled full-resolution motion vectors associated with the at least two full-resolution partitions.
11. The encoder claimed in claim 10 , wherein a first downscaled full-resolution motion vector has larger magnitude transform domain residual coefficients associated with it than the transform domain residual coefficients associated with a second downscaled full-resolution motion vector, and wherein the encoding application configures the processor to calculate a first weighting factor for the first downscaled full-resolution motion vector that is larger than a second weighting factor for the second downscaled full-resolution motion vector.
12. The encoder claimed in claim 10 , wherein a first downscaled full-resolution motion vector has larger magnitude transform domain residual coefficients associated with it than the transform domain residual coefficients associated with a second downscaled full-resolution motion vector, and wherein the encoding application configures the processor to calculate a first weighting factor for the first downscaled full-resolution motion vector that is smaller than a second weighting factor for the second downscaled full-resolution motion vector.
13. The encoder claimed in claim 9 , wherein the spatially downsampled video has been downsampled by a downsampling ratio, and wherein the encoding application configures the processor to downsample the associated full-resolution motion vectors by the downsampling ratio.
14. The encoder claimed in claim 9 , wherein the encoding application configures the processor to select the desired motion vector by searching the search area for candidate motion vectors and selecting as the desired motion vector the candidate motion vector that minimizes a rate-distortion cost expression.
15. A transcoder, comprising:
a decoder configured to decode a full-resolution encoded video to produce the full-resolution video, including decoding the at least two full-resolution partitions and their associated full-resolution motion vectors;
a spatial downsampler configured to spatially downsample the full-resolution video in the pixel domain to produce the downsampled video; and
the encoder claimed in claim 9 .
16. A computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, configure the processor to execute the method claimed in claim 1 .
17. An encoder for encoding a downsampled video, wherein the downsampled video is a spatially downsampled version of a full-resolution video, the downsampled video including a frame having a macroblock partitioned into at least one partition, one partition of the at least one partition corresponding to at least two full-resolution partitions in a corresponding frame of the full-resolution video, each of the at least two full-resolution partitions having an associated full-resolution motion vector relative to a reference frame, the encoder comprising:
a processor;
a memory;
a communications system for outputting an encoded downsampled video; and
an encoding application stored in memory and containing instructions for configuring the processor to encode the downsampled video by performing the method claimed in claim 5 .
18. A computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, configure the processor to execute the method claimed in claim 5 .