IP Library › Granted Patent US 11,109,059
Granted Patent B2
US 11,109,059 · App. 17/098,962 · Granted Aug 31, 2021

Clipping of updated MV or derived MV

Inventors: Hongbin Liu (Beijing, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Yue Wang (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/52H04N19/105H04N19/132H04N19/137H04N19/176H04N19/184H04N19/56H04N19/70H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,109,059
App. No.
17/098,962
Granted
Aug 31, 2021
Kind
B2
Abstract

Devices, systems and methods for digital video coding, which includes motion prediction based on updated motion vectors generated using two-step inter-prediction, are described. Methods for clipping of updated MV or derived MV are described. An exemplary method for video coding includes: determining original motion information for a current block; updating original motion vectors of the original motion information for the current block based on a refining method; clipping the updated motion vectors to be within a range; and performing a conversion between the current block and a bitstream representation of a video including the current block, based on the clipped updated motion vectors.

Claims (60)

1. A video processing method, comprising:

determining original motion information for a current block;

updating original motion vectors of the original motion information based on a decoder-side motion vector refinement (DMVR) method;

clipping the updated motion vectors to be within a first range; and

performing a conversion between the current block and a bitstream representation of a video including the current block, based on the clipped updated motion vectors,

wherein the performing a conversion comprises: decoding the current block from the bitstream representation or encoding the current block into the bitstream representation,

wherein the updating original motion vectors and the clipping the updated motion vectors are performed for each sub-block of the current block, and

wherein in case that a width and/or a height of the video block, to which the sub-block belongs, is larger than a fourth threshold, the current block is split into multiple sub-blocks.

2. The method of claim 1 , wherein the first range is same as an allowed range of the original motion vectors.

3. The method of claim 1 , wherein the performing a conversion comprises:

determining reference samples based on the updated motion vectors; and

adjusting the reference samples based on a motion vector field (v x , v y ) derived during a bi-directional optical flow refinement;

wherein in the bi-directional optical flow refinement, at least one of v x and v y in the motion vector field is clipped to a second range.

4. The method of claim 3 , wherein at least one of v x and v y is constrained to the second range as:

− M x <v x <N x , and/or − M y <v y <N y ,

wherein M x , N x , M y , N y are none-negative integers.

5. The method of claim 3 , wherein the motion vector field (v x , v y ) is determined based on at least one gradient value corresponding to the reference sample.

6. The method of claim 1 , wherein the method is not applied in case that the current block meets a certain condition, wherein the certain condition specifies at least one of: a size of the current block, a slice type of the current block, a picture type of the current block and a tile type of the current block.

7. The method of claim 6 , wherein the certain condition specifies at least one of:

a sample number that the current block contains is smaller than a first threshold,

a width of the current block is smaller than or no larger than a second threshold, or

a height of the current block is smaller than or no larger than a third threshold.

8. The method of claim 1 , wherein each of the multiple sub-blocks undergoes decoder-side motion vector refinement or/and bi-directional optical flow in the same way as a normal video block with size equal to the sub-block size.

9. The method of claim 7 , wherein at least one of the first threshold, the second threshold, or the third threshold is pre-defined.

10. The method of claim 1 , wherein the determining original motion information comprises:

constructing a candidate list for the current block; and

determining the original motion information from the candidate list.

11. The method of claim 1 , wherein the performing a conversion comprises:

decoding the current block from the bitstream representation.

12. The method of claim 1 , wherein the performing a conversion comprises:

encoding the current block into the bitstream representation.

13. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine original motion information for a current block;

update original motion vectors of the original motion information based on a decoder-side motion vector refinement (DMVR) method;

clip the updated motion vectors to be within a first range; and

perform a conversion between the current block and a bitstream representation of a video including the current block, based on the clipped updated motion vectors,

wherein the performing a conversion comprises: decoding the current block from the bitstream representation or encoding the current block into the bitstream representation,

wherein the updating original motion vectors and the clipping the updated motion vectors are performed for each sub-block of the current block, and

wherein in case that a width and/or a height of the video block, to which the sub-block belongs, is larger than a fourth threshold, the current block is split into multiple sub-blocks.

14. The apparatus of claim 13 , wherein the first range is same as an allowed range of the original motion vectors.

15. The apparatus of claim 13 , wherein the instructions upon execution by the processor further cause the processor to:

decode the current block from the bitstream representation.

16. The apparatus of claim 13 , wherein the instructions upon execution by the processor further cause the processor to:

encode the current block into the bitstream representation.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine original motion information for a current block;

update original motion vectors of the original motion information based on a decoder-side motion vector refinement (DMVR) method;

clip the updated motion vectors to be within a first range; and

perform a conversion between the current block and a bitstream representation of a video including the current block, based on the clipped updated motion vectors,

wherein the performing a conversion comprises: decoding the current block from the bitstream representation or encoding the current block into the bitstream representation,

wherein the updating original motion vectors and the clipping the updated motion vectors are performed for each sub-block of the current block, and

wherein in case that a width and/or a height of the video block, to which the sub-block belongs, is larger than a fourth threshold, the current block is split into multiple sub-blocks.

18. A non-transitory computer-readable recording medium storing a bitstream representation which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining original motion information for a current block;

updating original motion vectors of the original motion information based on a decoder-side motion vector refinement (DMVR) method;

clipping the updated motion vectors to be within a first range; and

encoding the current block into the bitstream representation of a video including the current block, based on the clipped updated motion vectors,

wherein the performing a conversion comprises: decoding the current block from the bitstream representation or encoding the current block into the bitstream representation,

wherein the updating original motion vectors and the clipping the updated motion vectors are performed for each sub-block of the current block, and

wherein in case that a width and/or a height of the video block, to which the sub-block belongs, is larger than a fourth threshold, the current block is split into multiple sub-blocks.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2020
From: ZHANG, LI; ZHANG, KAI
To: BYTEDANCE INC.
Reel/Frame 054381/0925 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2020
From: LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 054381/0983 →
Priority Claims (2)
WO PCT/CN2018/098691 · Aug 4, 2018 · international
WO PCT/CN2018/109250 · Oct 6, 2018 · international
Continuity (2)
Continuation PCTIB2019056641 · Aug 5, 2019
Related Publication 20210067783A1 · Mar 4, 2021