Method and apparatus for improvements of affine prof

1. A method of video decoding performed in a video decoder, the method comprising:

receiving a coded video bitstream including a current block that is divided into a plurality of sub-blocks;

performing sub-block based affine motion compensation on the current block to generate a sub-block prediction for each pixel in each sub-block of the current block;

determining one or more spatial gradients for each sub-block prediction;

performing, for each sub-block prediction, prediction refinement with an optical flow process using a respective determined one or more spatial gradients, a motion vector (MV) difference of a MV of the pixel corresponding to the respective sub-block prediction, and bit-depth constraints included in the coded video bitstream; and

adding, for each sub-block prediction, an output of a respective prediction refinement to a respective sub-block prediction to generate a final prediction for each pixel in each sub-block of the current block,

wherein the bit-depth constraints restrict the one or more spatial gradients to a number of bits that is less than or equal to K and the MV difference to a number of bits that is less than or equal to M in the prediction refinement with the optical flow process in which the restricted one or more spatial gradients of the respective sub-block prediction are multiplied by the MV difference of the MV of the pixel corresponding to the respective sub-block prediction, and

wherein K and M are integers.

2. The method of claim 1 , wherein a sum of K and N is less than or equal to 16.

3. The method of claim 1 , wherein K is equal to a min (11, Dep+1) bits, wherein Dep represents a bit-depth of an input video signal.

4. The method of claim 3 , wherein in response to a determination that Dep is greater than or equal to 10, the one or more spatial gradients are right shifted by (Dep+10).

5. The method of claim 1 , wherein the bit-depth constraints further specify that a precision of the MV difference is one bit higher than a precision of a MV precision used in motion compensated interpolation.

6. The method of claim 1 , wherein the bit-depth constraints further specify that a precision of the one or more spatial gradients is kept the same as interpolated pixels at fractional sample positions.

7. The method of claim 1 , wherein the determination of the one or more spatial gradients and performance of the prediction refinement for the current block are performed in accordance with a prediction refinement with optical flow (PROF) process, wherein another block in a same current picture as the current block is decoded in accordance with a bi-directional optical flow (BDOF) process, and wherein the bit-depth constraints further specify that the PROF process for the current block and the BDOF process for the another block share a same process for determining one or more spatial gradient values for the current block and the another block.

8. The method of claim 7 , wherein the PROF process for the current block and the BDOF process for the another block use a same height and weight to determine the one or more spatial gradient values for the current block and the another block.

9. The method of claim 7 , wherein the PROF process for the current block and the BDOF process for the another block use a same filter kernel to determine the one or more spatial gradient values for the current block and the another block.

10. The method of claim 1 , wherein each sub-block has a width W and a height H, and wherein a sub-block center position of each sub-block is ((W−1)/2, (H−1)/2) to derive each MV of each sub-block.

11. The method of claim 10 , where the MV difference is calculated for each pixel sample relative to the MV of position ((W−1)/2, (H−1)/2).

12. The method of claim 10 , wherein in response to a determination that a MV derived using the position ((W−1)/2, (H−1)/2) is a fractional MV, the fractional MV is rounded to a nearest integer.

13. The method of claim 12 , wherein the fractional MV is rounded to a nearest integer towards infinity.

14. The method of claim 1 , wherein each sub-block has a width W and a height H, and wherein a block size used for the prediction refinement is M×N, wherein M is a multiple of W and M>W, and wherein N is a multiple of H, and N is greater than H.

15. The method of claim 14 , wherein a padding process is applied at boundaries of each M×N block.

16. The method of claim 15 , wherein M and N are included in the coded video bitstream.

17. A video decoder for video decoding, comprising:

processing circuitry configured to:

receive a coded video bitstream including a current block that is divided into a plurality of sub-blocks,

perform sub-block based affine motion compensation on the current block to generate a sub-block prediction for each pixel in each sub-block of the current block,

determine one or more spatial gradients for each sub-block prediction,

perform, for each sub-block prediction, prediction refinement with an optical flow process using a respective determined one or more spatial gradients, a motion vector difference of a MV of the pixel corresponding to the respective sub-block prediction, and bit-depth constraints included in the coded video bitstream, and

add, for each sub-block prediction, an output of a respective prediction refinement to a respective sub-block prediction to generate a final prediction for each pixel in each sub-block of the current block,

wherein the bit-depth constraints restrict the one or more spatial gradients to a number of bits that is less than or equal to K and the MV difference to a number of bits that is less than or equal to M in the prediction refinement with the optical flow process in which the restricted one or more spatial gradients of the respective sub-block prediction are multiplied by the MV difference of the MV of the pixel corresponding to the respective sub-block prediction, and

wherein K and M are integers.

18. A non-transitory computer readable medium having instructions stored therein, which when executed by a processor in a video decoder causes the processor to perform a method comprising:

receiving a coded video bitstream including a current block that is divided into a plurality of sub-blocks;

performing sub-block based affine motion compensation on the current block to generate a sub-block prediction for each pixel in each sub-block of the current block;

determining one or more spatial gradients for each sub-block prediction;

performing, for each sub-block prediction, prediction refinement with an optical flow process using a respective determined one or more spatial gradients, a motion vector difference of a MV of the pixel corresponding to the respective sub-block prediction, and bit-depth constraints included in the coded video bitstream; and

adding, for each sub-block prediction, an output of a respective prediction refinement to a respective sub-block prediction to generate a final prediction for each pixel in each sub-block of the current block,

wherein the bit-depth constraints restrict the one or more spatial gradients to a number of bits that is less than or equal to K and the MV difference to a number of bits that is less than or equal to M in the prediction refinement with the optical flow process in which the restricted one or more spatial gradients of the respective sub-block prediction are multiplied by the MV difference of the MV of the pixel corresponding to the respective sub-block prediction, and

wherein K and M are integers.