IP Library › Granted Patent US 11,240,531
Granted Patent B2
US 11,240,531 · App. 17/324,432 · Granted Feb 1, 2022

Size selective application of decoder side refining tools

Inventors: Hongbin Liu (Beijing, CN); Li Zhang (San Diego, CA); Kai Zhang (San Diego, CA); Jizheng Xu (San Diego, CA); Yue Wang (Beijing, CN)
Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.
H04N19/52H04N19/139H04N19/176H04N19/184H04N19/186H04N19/577
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,240,531
App. No.
17/324,432
Granted
Feb 1, 2022
Kind
B2
Abstract

One example method of video processing includes implementing, by a processor, a decoder-side motion vector derivation (DMVD) scheme for motion vector refinement during a conversion between a current video block and a bitstream representation of the current video block by deriving parameters based on a deriving rule. The conversion may include compressing the current video block into the bitstream representation or uncompressing the bitstream representation into pixel values of the current video block.

Claims (44)

1. A video processing method, comprising:

making, based on a rule that uses a block dimension of a current video block of a video, a first determination on whether a first scheme is enabled or disabled for a conversion between the current video block and a bitstream of the video; wherein the first scheme is Decoder-side Motion Vector Refinement (DMVR) scheme which is used to derive a refined motion information based on a cost between at least one reference sample of list 0 and at least one reference sample of list 1;

making, based on the same rule, a second determination on whether a second scheme is enabled or disabled for the conversion, wherein the second scheme is a bi-directional optical flow scheme; and

performing the conversion based on the first determination and the second determination,

wherein the first scheme and the second scheme are determined to be disabled in response to at least one of:

a first number of samples included in the current video block being less than a second number of samples, or

a width of the current video block being less than a value or a height of the current video block being less than the value.

2. The method of claim 1 , wherein the value is 8.

3. The method of claim 1 , wherein the first scheme and the second scheme are determined to be enabled based on (W*H)>=T1 and H>=T2, where W and H are respectively width and height of the current video block and T1 and T2 are rational numbers.

4. The method of claim 3 , wherein T2 is 8.

5. The method of claim 1 , wherein the first scheme and the second scheme are performed in a same sub-block level for the current video block.

6. The method of claim 5 , wherein a determination of the sub-block size is same for the first scheme and the second scheme.

7. The method of claim 5 , wherein the current video block is split vertically in response to a width of the current video block is greater than a value, and wherein a width of a sub-block of the current video block is less than or equal to the value; or the current video block is split horizontally in response to a height of the current video block is greater than a value, and wherein a height of a sub-block of the current video block is less than or equal to the value.

8. The method of claim 5 , wherein the sub-block is processed by the first scheme and the second scheme in a same way as a coding block with size equal to a sub-block size.

9. The method of claim 7 , wherein the value is 16.

10. The method of claim 1 , wherein the current video block is a luma video block and the determining whether to enable or disable the first scheme or the second scheme is performed based on information of the luma video block and without using information of associated chroma video blocks.

11. The method of claim 5 , wherein and the first scheme is a decoder-side motion vector refinement scheme and the refined motion information is obtained by applying a bilateral matching in the decoder-side motion vector refinement scheme at the sub-block level for the current video block.

12. The method of claim 5 , wherein the second scheme is determined to be enabled and the refined motion information is obtained by performing a sample-wise refinement of the motion information performed at the sub-block level of the current video block in the second coding scheme.

13. The method of claim 5 , wherein the second scheme is determined to be enabled or disabled at a sub-block level for the current video block.

14. The method of claim 1 , wherein the conversion comprises encoding the current video block into the bitstream.

15. The method of claim 1 , wherein the conversion comprises decoding the current video block from the bitstream.

16. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

make, based on a rule that uses a block dimension of a current video block of a video, a first determination on whether a first scheme is enabled or disabled for a conversion between the current video block and a bitstream of the video; wherein the first scheme is Decoder-side Motion Vector Refinement (DMVR) scheme which is used to derive a refined motion information based on a cost between at least one reference sample of list 0 and at least one reference sample of list 1;

make, based on the same rule, a second determination on whether a second scheme is enabled or disabled for the conversion, wherein the second scheme is a bi-directional optical flow scheme; and

perform the conversion based on the first determination and the second determination,

wherein the first scheme and the second scheme are determined to be disabled in response to at least one of:

a first number of samples included in the current video block being less than a second number of samples, or

a width of the current video block being less than a value or a height of the current video block being less than the value.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

make, based on a rule that uses a block dimension of a current video block of a video, a first determination on whether a first scheme is enabled or disabled for a conversion between the current video block and a bitstream of the video; wherein the first scheme is Decoder-side Motion Vector Refinement (DMVR) scheme which is used to derive a refined motion information based on a cost between at least one reference sample of list 0 and at least one reference sample of list 1;

make, based on the same rule, a second determination on whether a second scheme is enabled or disabled for the conversion, wherein the second scheme is a bi-directional optical flow scheme; and

perform the conversion based on the first determination and the second determination,

wherein the first scheme and the second scheme are determined to be disabled in response to at least one of:

a first number of samples included in the current video block being less than a second number of samples, or

a width of the current video block being less than a value or a height of the current video block being less than the value.

18. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

making, based on a rule that uses a block dimension of a current video block of a video, a first determination on whether a first scheme is enabled or disabled for a conversion between the current video block and a bitstream of the video; wherein the first scheme is Decoder-side Motion Vector Refinement (DMVR) scheme which is used to derive a refined motion information based on a cost between at least one reference sample of list 0 and at least one reference sample of list 1;

making, based on the same rule, a second determination on whether a second scheme is enabled or disabled for the conversion, wherein the second scheme is a bi-directional optical flow scheme; and

generating the bitstream based on the first determination and the second determination,

wherein the first scheme and the second scheme are determined to be disabled in response to at least one of:

a first number of samples included in the current video block being less than a second number of samples, or

a width of the current video block being less than a value or a height of the current video block being less than the value.

19. The apparatus of claim 16 , wherein the first scheme and the second scheme are performed in a same sub-block level for the current video block.

20. The apparatus of claim 19 , wherein a determination of the sub-block size is same for the first scheme and the second scheme.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2021
From: ZHANG, LI; ZHANG, KAI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 056287/0674 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2021
From: LIU, HONGBIN; WANG, YUE
To: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.
Reel/Frame 056287/0804 →
Priority Claims (2)
WO PCT/CN2019/075068 · Feb 14, 2019 · international
WO PCT/CN2019/082585 · Apr 13, 2019 · international
Continuity (2)
Continuation PCTCN2020075235 · Feb 14, 2020
Related Publication 20210274211A1 · Sep 2, 2021
Cited By (7)
US 12,348,706 US 12,382,085 US 12,395,661 US 12,407,812 US 12,483,691 US 12,615,394 US 12,621,481