IP Library Granted Patent US 11,949,902
Granted Patent B2
US 11,949,902 · App. 17/709,008 · Granted Apr 2, 2024

Method and apparatus for video coding

Inventors: Guichun Li (Milpitas, CA); Xiang Li (Saratoga, CA); Xiaozhong Xu (State College, PA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/513H04N19/176H04N19/184
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,949,902
App. No.
17/709,008
Granted
Apr 2, 2024
Kind
B2
Abstract

A method for video decoding in a decoder is provided. Coding information of a current block (CB) from a coded video bitstream is decoded. The coding information includes weighted prediction information that indicates a weighted prediction for the CB. A determination is made as to whether to apply a prediction refinement with optical flow (PROF) on the CB based on the weighted prediction information. The CB is reconstructed based on the weighted prediction and whether the PROF is determined to be applied on the CB.

Claims (75)

1. A method for video encoding, comprising:

determining an inter prediction mode for a current block;

generating initial predictions for samples in a subblock of a plurality of subblocks in the current block based on the inter prediction mode; and

for one of the samples in the subblock,

determining a motion vector (MV) refinement (MVR), an absolute value of a first component of the MVR being one of: (i) 2 N1 and (ii) a sum or a difference of 2 N1 and 2 M1 , N1 and M1 being integers, the first component of the MVR being in a first direction that is one of an x direction and a y direction;

generating a prediction refinement based on the MVR and spatial gradients for the one of the samples, the spatial gradients being based on the initial predictions for the samples; and

encoding the current block in a bitstream based on the initial prediction of the one of the samples and the prediction refinement.

2. The method of claim 1 , wherein

the spatial gradients include a first spatial gradient in the first direction and a second spatial gradient in a second direction that is another one of the x direction and the y direction;

the MVR includes a second component that is in the second direction, an absolute value of the second component of the MVR being one of: (i) 2 N2 and (ii) a sum or a difference of 2 N2 and 2 M2 , N2 and M2 being integers; and

the generating the prediction refinement further includes:

determining a first refinement based on one of: (i) shifting the first spatial gradient by |N1| bits and (ii) separately shifting the first spatial gradient by |N1| and |M1| bits respectively and combining the shifted first spatial gradients;

determining a second refinement based on one of: (i) shifting the second spatial gradient by |N2| bits and (ii) separately shifting the second spatial gradient by |N2| and |M2| bits respectively and combining the shifted second spatial gradients; and

generating the prediction refinement based on the first refinement and the second refinement.

3. The method of claim 1 , wherein the determining the MVR further comprises:

determining an initial MVR for the one of the samples in the subblock based on the inter prediction mode; and

rounding a first component of the initial MVR to generate the first component of the MVR.

4. The method of claim 3 , wherein

the inter prediction mode is a subblock-based affine motion model with a prediction refinement with optical flow (PROF); and

for the one of the samples in the subblock, the determining the initial MVR further includes:

determining a sample MV of the one of the samples and a subblock MV of the subblock based on the subblock-based affine motion model; and

determining the initial MVR based the sample MV and the subblock MV.

5. The method of claim 3 , wherein the rounding the first component of the initial MVR further comprises:

rounding the first component of the initial MVR based on a position p of a most significant bit (MSB) of the first component of the initial MVR.

6. The method of claim 5 , wherein the rounding the first component of the initial MVR comprises:

left-shifting a bit value of 1 by p bits to obtain the absolute value of the first component of the MVR as 2 P , N1 being equal to p.

7. The method of claim 5 , wherein the rounding the first component of the initial MVR comprises:

left-shifting a bit value of 1 by p or (p+1) bits to round an absolute value of the first component of the initial MVR to a nearest power of 2 that is one of: 2 P and 2 P+1 .

8. The method of claim 3 , wherein

the first component of the initial MVR is a ratio of a first value over a second value, the first value and the second value being non-zero integers; and

the rounding the first component of the initial MVR further includes rounding the first component of the initial MVR based on a first position p1 of a MSB of the first value and a second position p2 of a MSB of the second value.

9. The method of claim 1 , wherein the first component of the MVR is 2 N1 or −2 N1 .

10. The method of claim 1 , wherein the first component of the MVR is (2 N1 +2 M1 ) or −(2 N1 −2 M1 ).

11. An apparatus for video encoding, comprising processing circuitry configured to:

determine an inter prediction mode for a current block;

generate initial predictions for samples in a subblock of a plurality of subblocks in the current block based on the inter prediction mode; and

for one of the samples in the subblock,

determine a motion vector (MV) refinement (MVR), an absolute value of a first component of the MVR being one of: (i) 2 N1 and (ii) a sum or a difference of 2 N1 and 2 M1 , N1 and M1 being integers, the first component of the MVR being in a first direction that is one of an x direction and a y direction;

generate a prediction refinement based on the MVR and spatial gradients for the one of the samples, the spatial gradients being based on the initial predictions for the samples; and

encode the current block in a bitstream based on the initial prediction of the one of the samples and the prediction refinement.

12. The apparatus of claim 11 , wherein

the spatial gradients include a first spatial gradient in the first direction and a second spatial gradient in a second direction that is another one of the x direction and the y direction;

the MVR includes a second component that is in the second direction, an absolute value of the second component of the MVR being one of: (i) 2 N2 and (ii) a sum or a difference of 2 N2 and 2 M2 , N2 and M2 being integers; and

the processing circuitry is further configured to:

determine a first refinement based on one of: (i) shifting the first spatial gradient by |N 1 | bits and (ii) separately shifting the first spatial gradient by |N 1 | and |M 1 | bits respectively and combining the shifted first spatial gradients;

determine a second refinement based on one of: (i) shifting the second spatial gradient by |N 2 | bits and (ii) separately shifting the second spatial gradient by |N 2 | and |M 2 | bits respectively and combining the shifted second spatial gradients; and

generate the prediction refinement based on the first refinement and the second refinement.

13. The apparatus of claim 11 , wherein the processing circuitry is further configured to:

determine an initial MVR for the one of the samples in the subblock based on the inter prediction mode; and

round a first component of the initial MVR to generate the first component of the MVR.

14. The apparatus of claim 13 , wherein

the inter prediction mode is a subblock-based affine motion model with a prediction refinement with optical flow (PROF); and

for the one of the samples in the subblock, the processing circuitry is configured to:

determine a sample MV of the one of the samples and a subblock MV of the subblock based on the subblock-based affine motion model; and

determine the initial MVR based the sample MV and the subblock MV.

15. The apparatus of claim 13 , wherein the processing circuitry is further configured to:

round the first component of the initial MVR based on a position p of a most significant bit (MSB) of the first component of the initial MVR.

16. The apparatus of claim 13 , wherein

the first component of the initial MVR is a ratio of a first value over a second value, the first value and the second value being non-zero integers; and

the processing circuitry is configured to round the first component of the initial MVR based on a first position p1 of a MSB of the first value and a second position p2 of a MSB of the second value.

17. The apparatus of claim 11 , wherein the first component of the MVR is 2 N1 or −2 N1 .

18. The apparatus of claim 11 , wherein the first component of the MVR is (2 N1 +2 M1 ) or −(2 N1 +2 M1 ).

19. A non-transitory computer-readable storage medium storing instructions which when executed by a processor cause the processor to perform:

determining an inter prediction mode for a current block;

generating initial predictions for samples in a subblock of a plurality of subblocks in the current block based on the inter prediction mode; and

for one of the samples in the subblock,

determining a motion vector (MV) refinement (MVR), an absolute value of a first component of the MVR being one of: (i) 2 N1 and (ii) a sum or a difference of 2 N1 and 2 M1 N1 and M1 being integers, the first component of the MVR being in a first direction that is one of an x direction and a y direction;

generating a prediction refinement based on the MVR and spatial gradients for the one of the samples, the spatial gradients being based on the initial predictions for the samples; and

encoding the current block in a bitstream based on the initial prediction of the one of the samples and the prediction refinement.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the spatial gradients include a first spatial gradient in the first direction and a second spatial gradient in a second direction that is another one of the x direction and the y direction;

the MVR includes a second component that is in the second direction, an absolute value of the second component of the MVR being one of: (i) 2 N2 and (ii) a sum or a difference of 2 N2 and 2 M2 , N2 and M2 being integers; and

the generating the prediction refinement further includes:

determining a first refinement based on one of: (i) shifting the first spatial gradient by IN 11 bits and (ii) separately shifting the first spatial gradient by |N 1 | and |M 1 | bits respectively and combining the shifted first spatial gradients;

determining a second refinement based on one of: (i) shifting the second spatial gradient by |N 2 | bits and (ii) separately shifting the second spatial gradient by |N 2 | and |M 2 | bits respectively and combining the shifted second spatial gradients; and

generating the prediction refinement based on the first refinement and the second refinement.

Continuity (3)
Continuation 16898059 · Jun 10, 2020
Provisional Application 62868488 · Jun 28, 2019
Related Publication 20220232244A1 · Jul 21, 2022