Method and apparatus for temporal interpolated prediction in video bitstream
A video decoder is provided for the decoding of a video bitstream encoded in a temporal interpolated prediction (TIP) mode. First and second motion vectors pointing to respective reference frames, or reference pictures within those frames, are generated for a block of a current picture. The motion vectors are then refined by application of a decoder-side motion vector refinement (DMVR) process, based on a bilateral matching process, and the refined motion vectors are used to decode the block. The refinement may more specifically involve consideration of candidates for the refined motion vectors, selected by the bilateral matching. The refinement may be applied at both block and sub-block divisions of the current picture.
1. A method for video decoding in a decoder, the method comprising:
receiving a coded video bitstream including a current picture that includes at least one block and a syntax element indicating that the at least one block is to be predicted in a temporal interpolated prediction (TIP) mode;
generating, for the at least one block, a motion field that includes a first motion vector that points to a first reference picture and a second motion vector that points to a second reference picture;
generating a first refined motion vector and a second refined motion vector by using the first motion vector and the second motion vector in a decoder-side motion vector refinement (DMVR) process for the at least one block, based on a bilateral matching process; and
decoding the at least one block using the first refined motion vector and the second refined motion vector,
wherein the bilateral matching process determines a first set of candidate motion vectors surrounding the first motion vector and a second set of candidate motion vectors surrounding the second motion vector,
wherein the first refined motion vector is a candidate motion vector from the first set of candidate motion vectors that has a lowest distortion cost with respect to the first motion vector, and
wherein the second refined motion vector is a candidate motion vector from the second set of candidate motion vectors that has a lowest distortion cost with respect to the second motion vector.
2. The method according to claim 1 , wherein at least one candidate motion vector is determined based on a search range N, wherein N is an integer value corresponding to a number of luma samples.
3. The method according to claim 1 , wherein at least one candidate motion vector is determined based on a search precision K.
4. The method according to claim 3 , wherein the search precision K is one of a fractional value of a number of luma samples and an integer value corresponding to a number of luma samples.
5. The method according to claim 1 , wherein the generating of the first refined motion vector and the second refined motion vector includes:
splitting the at least one block into a plurality of sub-blocks, and
performing the DMVR process on each sub-block of the plurality of sub-blocks.
6. The method according to claim 5 , wherein the generating of the first refined motion vector and the second refined motion vector further includes at least one of:
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block before the DMVR process is performed on the sub-block, and
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block after the DMVR process is performed on the sub-block.
7. The method according to claim 5 , wherein the generating of the first refined motion vector and the second refined motion vector includes: performing the DMVR process on the at least one block, splitting the at least one block into a plurality of sub-blocks after the DMVR process is performed on the at least one block; and performing the DMVR process on each sub-block in the plurality of sub-blocks.
8. A method of video encoding, the method comprising:
receiving a video data including a current picture; and
encoding the video data such that the encoded video data includes at least one block and a syntax element indicating that the at least one block is to be predicted in a temporal interpolated prediction (TIP) mode, wherein
the encoded video data corresponds to, for the at least one block, a motion field is to be generated that includes a first motion vector that points to a first reference picture and a second motion vector that points to a second reference picture;
the encoded video data further corresponds to a first refined motion vector and a second refined motion vector to be generated by using the first motion vector and the second motion vector in a decoder-side motion vector refinement (DMVR) process for the at least one block, based on a bilateral matching process; and
wherein the bilateral matching process determines a first set of candidate motion vectors surrounding the first motion vector and a second set of candidate motion vectors surrounding the second motion vector,
wherein the first refined motion vector is a candidate motion vector from the first set of candidate motion vectors that has a lowest distortion cost with respect to the first motion vector, and
wherein the second refined motion vector is a candidate motion vector from the second set of candidate motion vectors that has a lowest distortion cost with respect to the second motion vector.
9. The method according to claim 8 , wherein at least one candidate motion vector is determined based on a search range N, wherein N is an integer value corresponding to a number of luma samples.
10. The method according to claim 8 , wherein at least one candidate motion vector is determined based on a search precision K.
11. The method according to claim 10 , wherein the search precision K is one of a fractional value of a number of luma samples and an integer value corresponding to a number of luma samples.
12. The method according to claim 8 , wherein generating of the first refined motion vector and the second refined motion vector includes:
splitting the at least one block into a plurality of sub-blocks, and
performing the DMVR process on each sub-block of the plurality of sub-blocks.
13. The method according to claim 12 , wherein generating of the first refined motion vector and the second refined motion vector further includes at least one of:
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block before the DMVR process is performed on the sub-block, and
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block after the DMVR process is performed on the sub-block.
14. The method according to claim 12 , wherein generating of the first refined motion vector and the second refined motion vector includes:
performing the DMVR process on the at least one block,
splitting the at least one block into a plurality of sub-blocks after the DMVR process is performed on the at least one block; and
performing the DMVR process on each sub-block in the plurality of sub-blocks.
15. A method of processing visual media data, the method comprising:
performing a conversion between a visual media file and a bitstream of a visual media data according to a format rule indicating that:
video data includes at least one block and a syntax element indicating that the at least one block is to be predicted in a temporal interpolated prediction (TIP) mode, wherein
the video data corresponds to, for the at least one block, a motion field is to be generated that includes a first motion vector that points to a first reference picture and a second motion vector that points to a second reference picture;
the video data further corresponds to a first refined motion vector and a second refined motion vector to be generated by using the first motion vector and the second motion vector in a decoder-side motion vector refinement (DMVR) process for the at least one block, based on a bilateral matching process; and
wherein the bilateral matching process determines a first set of candidate motion vectors surrounding the first motion vector and a second set of candidate motion vectors surrounding the second motion vector,
wherein the first refined motion vector is a candidate motion vector from the first set of candidate motion vectors that has a lowest distortion cost with respect to the first motion vector, and
wherein the second refined motion vector is a candidate motion vector from the second set of candidate motion vectors that has a lowest distortion cost with respect to the second motion vector.
16. The method according to claim 15 , wherein at least one candidate motion vector is determined based on a search range N, wherein N is an integer value corresponding to a number of luma samples.
17. The method according to claim 15 , wherein at least one candidate motion vector is determined based on a search precision K.
18. The method according to claim 17 , wherein the search precision K is one of a fractional value of a number of luma samples and an integer value corresponding to a number of luma samples.
19. The method according to claim 15 , wherein generating of the first refined motion vector and the second refined motion vector includes:
splitting the at least one block into a plurality of sub-blocks, and
performing the DMVR process on each sub-block of the plurality of sub-blocks.
20. The method according to claim 19 , wherein generating of the first refined motion vector and the second refined motion vector further includes at least one of:
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block before the DMVR process is performed on the sub-block,
for each sub-block of the plurality of sub-blocks, performing an optical flow refinement process on the sub-block after the DMVR process is performed on the sub-block,
performing the DMVR process on the at least one block,
splitting the at least one block into a plurality of sub-blocks after the DMVR process is performed on the at least one block; and
performing the DMVR process on each sub-block in the plurality of sub-blocks.