Methods and apparatuses for prediction refinement with optical flow
Methods, apparatuses, and non-transitory computer-readable storage mediums are provided for coding a video signal. A method includes obtaining a first reference picture and a second reference picture associated with a video block; obtaining first prediction samples of the video block from the first reference picture; obtaining second prediction samples of the video block from the second reference picture; obtaining padded prediction samples, and obtaining horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the padded prediction samples; obtaining motion refinements for samples in the video block based on the horizontal and vertical gradient values; and obtaining bi-prediction samples of the video block based on the motion refinements.
1. A method for coding a video signal, comprising:
obtaining a first reference picture and a second reference picture associated with a video block;
obtaining first prediction samples of the video block from the first reference picture;
obtaining second prediction samples of the video block from the second reference picture;
obtaining padded prediction samples, and obtaining horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the padded prediction samples, comprising:
deriving rows and columns of prediction samples outside the video block for the first prediction samples;
deriving rows and columns of prediction samples outside the video block for the second prediction samples; and
obtaining horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the derived rows and columns of prediction samples,
wherein bit-depths of the horizontal and vertical gradient values are controlled by performing a shift operation according to shift values, wherein the shift values comprise a first shift value for calculation of gradient values that are used in a prediction refinement with optical flow (PROF) process, and
wherein deriving the rows and columns of the prediction samples further comprises:
deriving a first part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a horizontal direction; and
deriving a second part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a vertical direction;
obtaining motion refinements for samples in the video block based on the horizontal and vertical gradient values; and
obtaining bi-prediction samples of the video block based on the motion refinements.
2. The method of claim 1 , wherein the rows and columns of prediction samples are copied from the corresponding integer reference samples.
3. A computing device for coding a video signal, comprising:
one or more processors;
a non-transitory computer-readable storage medium storing instructions executable by the one or more processors, wherein the one or more processors are configured to:
obtain a first reference picture and a second reference picture associated with a video block;
obtain first prediction samples of the video block from the first reference picture;
obtain second prediction samples of the video block from the second reference picture;
obtain padded prediction samples, and obtain horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the padded prediction samples, comprising:
derive rows and columns of prediction samples outside the video block for the first prediction samples;
derive rows and columns of prediction samples outside the video block for the second prediction samples; and
obtain horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the derived rows and columns of prediction samples,
wherein bit-depths of the horizontal and vertical gradient values are controlled by performing a shift operation according to shift values, wherein the shift values comprise a first shift value for calculation of gradient values that are used in a prediction refinement with optical flow (PROF) process, and
wherein derive the rows and columns of the prediction samples further comprises:
derive a first part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a horizontal direction; and
derive a second part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a vertical direction;
obtain motion refinements for samples in the video block based on the horizontal and vertical gradient values; and
obtain bi-prediction samples of the video block based on the motion refinements.
4. The computing device of claim 3 , wherein the rows and columns of prediction samples are copied from the corresponding integer reference samples.
5. A non-transitory computer readable storage medium storing a bitstream to be decoded by a coding method comprising:
obtaining a first reference picture and a second reference picture associated with a video block;
obtaining first prediction samples of the video block from the first reference picture;
obtaining second prediction samples of the video block from the second reference picture;
obtaining padded prediction samples, and obtaining horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the padded prediction samples, comprising:
deriving rows and columns of prediction samples outside the video block for the first prediction samples;
deriving rows and columns of prediction samples outside the video block for the second prediction samples; and
obtaining horizontal and vertical gradient values of the first prediction samples and the second prediction samples based on the derived rows and columns of prediction samples,
wherein bit-depths of the horizontal and vertical gradient values are controlled by performing a shift operation according to shift values, wherein the shift values comprise a first shift value for calculation of gradient values that are used in a prediction refinement with optical flow (PROF) process, and
wherein deriving the rows and columns of the prediction samples further comprises:
deriving a first part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a horizontal direction; and
deriving a second part of prediction samples from integer reference samples in the reference picture that are closest to a fractional sample position in a vertical direction;
obtaining motion refinements for samples in the video block based on the horizontal and vertical gradient values; and
obtaining bi-prediction samples of the video block based on the motion refinements.
6. The non-transitory computer readable storage medium of claim 5 , wherein the rows and columns of prediction samples are copied from the corresponding integer reference samples.