Smooth sub-block motion vector prediction
This disclosure relates generally to video coding and particularly to smooth-sub-block motion compensation, disclosing methods and systems for determining subblock motion vectors of a video block based on motion information of its spatial neighboring block, temporal blocs spatially co-located with the neighboring blocks, or co-located temporal block of the video block.
1 . A method for decoding a video block comprising a plurality of sub-blocks of a current frame in a video bitstream, the method comprising;
receiving a syntax element signaled in the video bitstream indicative of whether the video block is inter-predicted under a sub-block motion compensation mode, wherein the sub-block motion compensation mode is a parallel mode to an affine warp motion mode and an overlapped Block Motion Compensation (OBMC) mode;
determining based on a value of the received syntax element that the video block is to be inter-predicted under the sub-block motion compensation mode;
determining a set of reference motion vectors corresponding to a set of neighboring block or sub-block positions of the video block based on available candidate motion vectors containing motion information of one or more neighboring blocks of the video block in the current frame and temporal blocks of the video block; and
deriving a sub-block motion vector for each of the plurality of sub-blocks by selecting reference motion vectors from the set of reference motion vectors for each of the sub-blocks at a sub-block level and calculating the sub-block motion vector for each of the plurality of sub-blocks based on the reference motion vectors selected at the sub-block level.
2 . The method of claim 1 , wherein deriving the sub-block motion vector for each of the plurality of sub-blocks based on the reference motion vectors comprises:
deriving horizontal component of the sub-block motion vector based on horizontal components of the reference motion vectors; and
deriving vertical component of the sub-block motion vector based on vertical components of the reference motion vectors.
3 . The method of claim 1 , wherein, for each of the at least one of the set of neighboring block or sub-block positions, a reference motion vector is determined by selecting from the available candidate motion vectors comprising and in a predetermined availability-checking order of:
a first motion vector determined from a neighboring block of the video block in the current frame, the neighboring block being spatially co-located with the neighboring block or sub-block position; and
a second motion vector derived based on a temporal block spatially co-located with the neighboring block or sub-block position.
4 . The method of claim 1 , wherein, for each of at least one of the set of neighboring block or sub-block positions, a reference motion vector is determined by selecting from the available candidate motion vectors comprising and in a predetermined availability-checking order of:
a first motion vector determined from a neighboring block of the video block in the current frame, the neighboring block being spatially co-located with the neighboring block or sub-block position; and
a second motion vector determined from a secondary neighboring block of the video block in the current frame adjacent to the neighboring block or sub-block position.
5 . The method of claim 1 , wherein for a neighboring block or sub-block position with no available candidate motion vectors, the corresponding reference motion vector is padded from a reference motion vector associated with its adjacent neighboring block or sub-block position.
6 . The method of claim 1 , wherein a reference motion vector associated with a bottom-right neighboring block or sub-block position is:
fetched from co-located temporal motion vectors if available; and
otherwise, derived as a weighted average of reference motion vectors associated with a top- right neighboring block or sub-block position and a bottom-left neighboring block or sub-block position.
7 . The method of claim 1 , wherein a reference motion vector associated with a bottom-right neighboring block or sub-block position is:
fetched from co-located temporal motion vectors if available; and
otherwise not used for deriving the sub-block motion vector for each of the plurality of sub-blocks.
8 . The method of claim 1 , wherein:
the set of neighboring block or sub-block positions comprise corner positions including a top-left corner position, a top-right corner position, and bottom right corner position;
the reference motion vectors associated with the corner positions are selected from the available candidate motion vectors; and
other reference motion vectors associated with neighboring block or subblock positions other than the corner positions are interpolated from the reference motion vectors associated with the corner positions.
9 . The method of claim 1 , wherein deriving the sub-block motion vector for a sub- block of the video block based on the reference motion vectors comprises:
selecting from the set of reference motion vectors, a subset of reference motion vectors of neighboring block or sub-block positions that are left, top, right, and bottom to the sub-block; and
deriving the sub-block motion vector for the sub-block based on the subset of reference motion vectors.
10 . The method of claim 9 , wherein the sub-block motion vector for the sub-block is derived from the subset of reference motion vectors using a bi-linear equation or using a weighted average of the subset of reference motion vectors.
11 . The method of claim 10 , wherein a set of weighting factors for deriving the sub-block motion vector form the subset of reference motion vectors are selected among a predetermined or signaled multiple sets of weighting factors.
12 . The method of claim 1 , further comprising:
parsing the video bitstream to extract one or more motion vector deltas; and
applying the motion vector deltas to adjust at least one of the set of reference motion vectors before deriving the sub-block motion vector based thereof.
13 . The method of claim 12 , wherein the one or more motion vector deltas are attributed and applied to neighboring block or sub-block positions via a predetermined order.
14 . The method of claim 12 , further comprising parsing the video bitstream to extract a signaling identifying neighboring block or sub-block positions corresponding to the at least one of the set of reference motion vectors for the one or more motion vector deltas to be applied.
15 . The method of claim 12 , wherein the one or more motion vector deltas are signaled in the video bitstream using a same syntax and context as a motion vector difference (MVD) signaling or an adaptive-resolution MVD signaling.
16 . The method of claim 1 , the sub-block motion compensation mode is signaled only when a skip mode is enabled for the video block.
17 . The method of claim 1 , wherein the sub-block motion compensation mode comprises a smooth sub-block motion compensation mode.
18 . An electronic device, comprising a memory for storing instructions, and a processor for executing the stored instructions to:
to determine that a video block in a current frame is to be inter-predicted under a sub-block motion compensation mode, wherein the sub-block motion compensation mode is a parallel mode to an affine warp motion mode and an overlapped Block Motion Compensation (OBMC) mode;
determine a plurality of sub-blocks of the video block;
determine a set of reference motion vectors corresponding to a set of neighboring block or sub-block positions of the video block based on available candidate motion vectors including motion information of one or more neighboring blocks of the video block in the current frame and temporal blocks of the video block; and
derive a sub-block motion vector for each of the plurality of sub-blocks by selecting reference motion vectors from the set of reference motion vectors for each of the sub-blocks at a sub-block level and calculating the sub-block motion vector for each of the plurality of sub-blocks based on the reference motion vectors selected at the sub-block level.
19 . A non-transitory computer readable storage medium for storing a video bitstream that is generated by a video encoding method, the video encoding method comprising:
signaling a syntax element in the video bitstream indicative of whether a video block of a current frame is inter-predicted under a sub-block motion compensation mode, wherein the sub-block motion compensation mode is a parallel mode to an affine warp motion mode and an overlapped Block Motion Compensation (OBMC) mode;
determining a plurality of sub-blocks of the video block;
determining a set of reference motion vectors corresponding to a set of neighboring block or sub-block positions of the video block based on available candidate motion vectors including motion information of one or more neighboring blocks of the video block in the current frame and temporal blocks of the video block;
deriving a sub-block motion vector for each of the plurality of sub-blocks of the video block by selecting reference motion vectors from the set of reference motion vectors for each of the sub-blocks at a sub-block level and calculating the sub-block motion vector for each of the plurality of sub-blocks based on the reference motion vectors selected at the sub-block level;
encoding each of the plurality of the sub-blocks based on the sub-block motion vector in the video bitstream.