SELECTION OF MOTION VECTOR PRECISION
Approaches to selection of motion vector (“MV”) precision during video encoding are presented. These approaches can facilitate compression that is effective in terms of rate-distortion performance and/or computational efficiency. For example, a video encoder determines an MV precision for a unit of video from among multiple MV precisions, which include one or more fractional-sample MV precisions and integer-sample MV precision. The video encoder can identify a set of MV values having a fractional-sample MV precision, then select the MV precision for the unit based at least in part on prevalence of MV values (within the set) having a fractional part of zero. Or, the video encoder can perform rate-distortion analysis, where the rate-distortion analysis is biased towards the integer-sample MV precision. Or, the video encoder can collect information about the video and select the MV precision for the unit based at least in part on the collected information.
1 . One or more non-transitory computer-readable media having stored therein computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations comprising:
encoding frames of a video sequence, thereby producing encoded data, wherein the encoding the frames includes:
setting an indicator, the indicator indicating (a) whether or not motion vector (“MV”) precision is adaptively selected for units of the video sequence, and (b) if the MV precision for the units is not adaptively selected, whether the MV precision for the units is fractional-sample precision or integer-sample precision;
encoding the indicator in a first-layer syntax structure that applies for the video sequence, wherein the indicator indicates the MV precision for the units is adaptively selected; and
for a current unit among the units:
selecting whether MV precision for the current unit is the fractional-sample precision or the integer-sample precision, wherein the selecting includes:
performing rate-distortion analysis to evaluate encoding of the current unit with the MV precision being the integer-sample precision and to evaluate encoding of the current unit with the MV precision being the fractional-sample precision; and
based at least in part on results of the rate-distortion analysis, determining whether the MV precision for the current unit is the fractional-sample precision or the integer-sample precision; and
encoding a flag in a second-layer syntax structure for the current unit, the flag indicating the MV precision for the current unit to be the fractional-sample precision or the integer-sample precision; and
outputting the encoded data as part of a bitstream, the encoded data including the encoded indicator in the first-layer syntax structure and, the MV precision for the units being adaptively selected, the flag in the second-layer syntax structure for the current unit that indicates the MV precision for the current unit.
2 . The one or more computer-readable media of claim 1 , wherein the first-layer syntax structure is a sequence-layer syntax structure, the units are the frames of the video sequence, and the second-layer syntax structure is a picture-layer syntax structure.
3 . The one or more computer-readable media of claim 1 , wherein the first-layer syntax structure is a sequence parameter set, the units are slices of the frames of the video sequence, and the second-layer syntax structure is a slice-header-layer syntax structure.
4 . The one or more computer-readable media of claim 1 , wherein, for the indicator:
a first possible value indicates that the MV precision for the units is not adaptively selected and further indicates that the MV precision for the units is the fractional-sample precision;
a second possible value indicates that the MV precision for the units is not adaptively selected and further indicates that the MV precision for the units is the integer-sample precision; and
a third possible value indicates that the MV precision for the units is adaptively selected.
5 . The one or more computer-readable media of claim 1 , wherein the encoding the indicator uses two bits in the first-layer syntax structure or includes entropy coding a two-bit value for the first-layer syntax structure.
6 . The one or more computer-readable media of claim 1 , wherein the fractional-sample precision is quarter-sample precision.
7 . The one or more computer-readable media of claim 1 , wherein the rate-distortion analysis is biased towards the integer-sample precision.
8 . The one or more computer-readable media of claim 7 , wherein the rate-distortion analysis is biased towards the integer-sample precision for the current unit by scaling up a distortion cost for the fractional-sample precision, scaling up a rate cost for the fractional-sample precision, adding a distortion penalty for the fractional-sample precision, adding a rate penalty for the fractional-sample precision, using a larger Lagrangian multiplier factor for the fractional-sample precision, scaling down a distortion cost for the integer-sample precision, scaling down a rate cost for the integer-sample precision, and/or using a smaller Lagrangian multiplier factor for the integer-sample precision.
9 . The one or more computer-readable media of claim 1 , wherein the performing the rate-distortion analysis includes:
determining a first rate-distortion cost, for the encoding of the current unit with the MV precision being the integer-sample precision, using a rate cost for the encoding of the current unit with the MV precision being the integer-sample precision, a distortion cost for the encoding of the current unit with the MV precision being the integer-sample precision, and a first weighting factor; and
determining a second rate-distortion cost, for the encoding of the current unit with the MV precision being the fractional-sample precision, using a rate cost for the encoding of the current unit with the MV precision being the fractional-sample precision, a distortion cost for the encoding of the current unit with the MV precision being the fractional-sample precision, and a second weighting factor.
10 . The one or more computer-readable media of claim 9 , wherein the first weighting factor is different than the second weighting factor.
11 . A computer system comprising:
a video encoder, implemented using one or more processing units of the computer system, configured to encode frames of a video sequence, thereby producing encoded data, by performing operations that include:
setting an indicator, the indicator indicating (a) whether or not motion vector (“MV”) precision is adaptively selected for units of the video sequence, and (b) if the MV precision for the units is not adaptively selected, whether the MV precision for the units is fractional-sample precision or integer-sample precision;
encoding the indicator in a first-layer syntax structure that applies for the video sequence, wherein the indicator indicates the MV precision for the units is adaptively selected; and
for a current unit among the units:
selecting whether MV precision for the current unit is the fractional-sample precision or the integer-sample precision, wherein the selecting includes:
performing rate-distortion analysis to evaluate encoding of the current unit with the MV precision being the integer-sample precision and to evaluate encoding of the current unit with the MV precision being the fractional-sample precision; and
based at least in part on results of the rate-distortion analysis, determining whether the MV precision for the current unit is the fractional-sample precision or the integer-sample precision; and
encoding a flag in a second-layer syntax structure for the current unit, the flag indicating the MV precision for the current unit to be the fractional-sample precision or the integer-sample precision; and
a buffer, implemented using memory of the computer system, configured to store the encoded data for output as part of a bitstream, the encoded data including the encoded indicator in the first-layer syntax structure and, the MV precision for the units being adaptively selected, the flag in the second-layer syntax structure for the current unit that indicates the MV precision for the current unit.
12 . The computer system of claim 11 , wherein:
the first-layer syntax structure is a sequence-layer syntax structure, the units are frames of the video sequence, and the second-layer syntax structure is a picture-layer syntax structure; or
the first-layer syntax structure is a sequence parameter set, the units are slices of the frames of the video sequence, and the second-layer syntax structure is a slice-header-layer syntax structure.
13 . The computer system of claim 11 , wherein the encoding the indicator uses two bits in the first-layer syntax structure or includes entropy coding a two-bit value for the first-layer syntax structure.
14 . The computer system of claim 11 , wherein the rate-distortion analysis is biased towards the integer-sample precision.
15 . The computer system of claim 14 , wherein the rate-distortion analysis is biased towards the integer-sample precision for the current unit by scaling up a distortion cost for the fractional-sample precision, scaling up a rate cost for the fractional-sample precision, adding a distortion penalty for the fractional-sample precision, adding a rate penalty for the fractional-sample precision, using a larger Lagrangian multiplier factor for the fractional-sample precision, scaling down a distortion cost for the integer-sample precision, scaling down a rate cost for the integer-sample precision, and/or using a smaller Lagrangian multiplier factor for the integer-sample precision.
16 . The computer system of claim 11 , wherein the performing the rate-distortion analysis includes:
determining a first rate-distortion cost, for the encoding of the current unit with the MV precision being the integer-sample precision, using a rate cost for the encoding of the current unit with the MV precision being the integer-sample precision, a distortion cost for the encoding of the current unit with the MV precision being the integer-sample precision, and a first weighting factor; and
determining a second rate-distortion cost, for the encoding of the current unit with the MV precision being the fractional-sample precision, using a rate cost for the encoding of the current unit with the MV precision being the fractional-sample precision, a distortion cost for the encoding of the current unit with the MV precision being the fractional-sample precision, and a second weighting factor.
17 . The computer system of claim 16 , wherein the first weighting factor is different than the second weighting factor.
18 . In a computer system that implements a video encoder, a method comprising:
encoding frames of a video sequence, thereby producing encoded data, wherein the encoding the frames includes:
setting an indicator, the indicator indicating (a) whether or not motion vector (“MV”) precision is adaptively selected for units of the video sequence, and (b) if the MV precision for the units is not adaptively selected, whether the MV precision for the units is fractional-sample precision or integer-sample precision;
encoding the indicator in a first-layer syntax structure that applies for the video sequence, wherein the indicator indicates the MV precision for the units is adaptively selected; and
for a current unit among the units:
selecting whether MV precision for the current unit is the fractional-sample precision or the integer-sample precision, wherein the selecting includes:
performing rate-distortion analysis to evaluate encoding of the current unit with the MV precision being the integer-sample precision and to evaluate encoding of the current unit with the MV precision being the fractional-sample precision; and
based at least in part on results of the rate-distortion analysis, determining whether the MV precision for the current unit is the fractional-sample precision or the integer-sample precision; and
encoding a flag in a second-layer syntax structure for the current unit, the flag indicating the MV precision for the current unit to be the fractional-sample precision or the integer-sample precision; and
outputting the encoded data as part of a bitstream, the encoded data including the encoded indicator in the first-layer syntax structure and, the MV precision for the units being adaptively selected, the flag in the second-layer syntax structure for the current unit that indicates the MV precision for the current unit.
19 . The method of claim 18 , wherein:
the first-layer syntax structure is a sequence-layer syntax structure, the units are the frames of the video sequence, and the second-layer syntax structure is a picture-layer syntax structure; or
the first-layer syntax structure is a sequence parameter set, the units are slices of the frames of the video sequence, and the second-layer syntax structure is a slice-header-layer syntax structure.
20 . The method of claim 18 , wherein the rate-distortion analysis is biased towards the integer-sample precision for the current unit by scaling up a distortion cost for the fractional-sample precision, scaling up a rate cost for the fractional-sample precision, adding a distortion penalty for the fractional-sample precision, adding a rate penalty for the fractional-sample precision, using a larger Lagrangian multiplier factor for the fractional-sample precision, scaling down a distortion cost for the integer-sample precision, scaling down a rate cost for the integer-sample precision, and/or using a smaller Lagrangian multiplier factor for the integer-sample precision.