Granted Patent B2

US 12,267,484 · App. 18/074,562 · Granted Apr 1, 2025

Inventors: Mohammed Golam Sarwer (San Jose, CA); Rachel Barker (Cambridge, GB); Jianle Chen (San Diego, CA); Debargha Mukherjee (Cupertino, CA)

Assignee: GOOGLE LLC

H04N19/105H04N19/176H04N19/182H04N19/46

Abstract

Claims (164)

1. A method comprising:

obtaining a translational dynamic reference list for encoding a current block from a current frame from an input video;

obtaining a warped reference list for encoding the current block;

obtaining an output bitstream including:

an index value corresponding to optimal predicted warped model parameters from the warped reference list,

encoded block data obtained by encoding the current block using optimal warped model parameters, and

differential warped model parameters, wherein the differential warped model parameters indicate a difference between the optimal warped model parameters and the optimal predicted warped model parameters; and

outputting the output bitstream.

2. The method of claim 1 , wherein obtaining the warped reference list includes:

identifying a translational motion vector prediction block corresponding to a translational motion vector for the current block;

in response to a determination that the translational motion vector prediction block corresponds to a context block previously encoded using warped motion prediction, including warped motion parameters from the context block in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that:

at least one index location is available in the warped reference list,

a sequentially minimal unevaluated context block previously encoded using warped motion prediction with reference to a current reference frame indicated by a current reference frame index, wherein encoding the current block includes encoding the current block with reference to the current reference frame, is available among a defined sequence of context blocks, and

warped motion parameters from the sequentially minimal unevaluated context block differ from warped motion parameters in the warped reference list,

including warped motion parameters from the sequentially minimal unevaluated context block in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that:

at least one index location is available in the warped reference list, and

warped motion parameters that differ from the warped motion parameters in the warped reference list are available from a warped motion parameter bank for a current tile, wherein the current tile includes the current block,

including the warped motion parameters from the warped motion parameter bank in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that at least one index location is available in the warped reference list, including global warped motion parameters associated with the current reference frame that differ from the warped motion parameters in the warped reference list in the warped reference list at a sequentially minimal index location available in the warped reference list; and

in response to a determination that at least one index location is available in the warped reference list, including defined warped motion parameters in the warped reference list at a sequentially minimal index location available in the warped reference list.

3. The method of claim 1 , wherein obtaining the output bitstream includes:

determining whether to omit the differential warped model parameters from the output bitstream; and

in response to a determination to omit the differential warped model parameters from the output bitstream, omitting the differential warped model parameters from the output bitstream.

4. The method of claim 3 , wherein determining whether to omit the differential warped model parameters from the output bitstream includes:

in response to a determination that the index value is greater than or equal to a defined threshold, determining to omit the differential warped model parameters from the output bitstream.

5. The method of claim 1 , wherein:

a current pixel from the current block has a current horizontal position in the current frame and a current vertical position in the current frame;

the optimal predicted warped model parameters include:

a horizontal translational motion parameter;

a vertical translational motion parameter;

a horizontal scaling parameter;

a vertical scaling parameter;

a first rotation parameter; and

a second rotation parameter;

a horizontal displacement for encoding the current block is a result of adding:

a result of multiplying the horizontal scaling parameter by the current horizontal position,

a result of multiplying the first rotation parameter by the current vertical position, and

the horizontal translational motion parameter; and

a vertical displacement for encoding the current block is a result of adding:

a result of multiplying the vertical scaling parameter by the current horizontal position,

a result of multiplying the second rotation parameter by the current vertical position, and

the vertical translational motion parameter.

6. The method of claim 5 , wherein obtaining the output bitstream includes:

including, in the output bitstream, a flag indicating that the current block is encoded using a six-parameter warped motion model.

7. The method of claim 1 , wherein:

a current pixel from the current block has a current horizontal position in the current frame and a current vertical position in the current frame;

the optimal predicted warped model parameters include:

a horizontal translational motion parameter;

a vertical translational motion parameter;

a horizontal scaling parameter; and

a rotation parameter;

a horizontal displacement for encoding the current block is a result of:

adding:

a result of subtracting:

a result of multiplying the rotation parameter by the current vertical position, from

a result of multiplying the horizontal scaling parameter by the current horizontal position, and

the horizontal translational motion parameter; and

a vertical displacement for encoding the current block is a result of adding:

a result of multiplying the rotation parameter by the current horizontal position,

a result of multiplying the horizontal scaling parameter by the current vertical position, and

the vertical translational motion parameter.

8. The method of claim 7 , wherein obtaining the output bitstream includes:

including, in the output bitstream, a flag indicating that the current block is encoded using a four-parameter warped motion model.

9. The method of claim 1 , wherein obtaining the warped reference list includes:

determining a maximum cardinality of the warped reference list in accordance with a prediction mode for encoding the current block.

10. The method of claim 9 , wherein:

in response to a determination that the prediction mode is new motion vector mode, the maximum cardinality is four; and

in response to a determination that the prediction mode is other than new motion vector mode, the maximum cardinality is one.

11. The method of claim 1 , wherein obtaining the output bitstream includes:

including, in the output bitstream, a maximum cardinality of the warped reference list for the current frame.

12. The method of claim 1 , wherein:

the input video includes a current sequence of frames that includes the current frame; and

obtaining the output bitstream includes:

including, in the output bitstream, a maximum cardinality of the warped reference list for the current sequence of frames.

13. A method comprising:

obtaining an encoded bitstream;

generating a reconstructed frame, wherein generating the reconstructed frame includes:

obtaining a translational dynamic reference list for decoding a current block for the reconstructed frame;

obtaining a warped reference list for decoding the current block;

decoding a warped reference list index value from the encoded bitstream;

obtaining optimal predicted warped model parameters from the warped reference list in accordance with the warped reference list index value;

decoding differential warped model parameters for the current block from the encoded bitstream;

obtaining, as optimal warped model parameters for the current block, a result of adding the optimal predicted warped model parameters and the differential warped model parameters;

obtaining predicted block data for the current block in accordance with the optimal warped model parameters;

decoding residual block data for the current block from the encoded bitstream;

obtaining, as decoded block data for the current block, a result of adding the residual block data and the predicted block data;

including the decoded block data in a reconstructed block of the reconstructed frame; and

outputting the reconstructed frame.

14. The method of claim 13 , wherein obtaining the warped reference list includes:

identifying a translational motion vector prediction block corresponding to a translational motion vector for the current block;

in response to a determination that the translational motion vector prediction block corresponds to a context block previously decoded using warped motion prediction, including warped motion parameters from the context block in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that:

at least one index location is available in the warped reference list,

a sequentially minimal unevaluated context block previously decoded using warped motion prediction with reference to a current reference frame indicated by a current reference frame index, wherein decoded the current block includes decoded the current block with reference to the current reference frame, is available among a defined sequence of context blocks, and

warped motion parameters from the sequentially minimal unevaluated context block differ from warped motion parameters in the warped reference list,

including warped motion parameters from the sequentially minimal unevaluated context block in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that:

at least one index location is available in the warped reference list, and

warped motion parameters that differ from the warped motion parameters in the warped reference list are available from a warped motion parameter bank for a current tile, wherein the current tile includes the current block,

including the warped motion parameters from the warped motion parameter bank in the warped reference list at a sequentially minimal index location available in the warped reference list;

in response to a determination that at least one index location is available in the warped reference list, including global warped motion parameters associated with the current reference frame that differ from the warped motion parameters in the warped reference list in the warped reference list at a sequentially minimal index location available in the warped reference list; and

in response to a determination that at least one index location is available in the warped reference list, including defined warped motion parameters in the warped reference list at a sequentially minimal index location available in the warped reference list.

15. The method of claim 13 , wherein obtaining the optimal warped model parameters includes:

in response to a determination that the differential warped model parameters are absent from the encoded bitstream, using the optimal predicted warped model parameters as the optimal warped model parameters.

16. The method of claim 13 , wherein:

generating the reconstructed frame includes obtaining, from the encoded bitstream, a flag indicating that the current block is encoded using a six-parameter warped motion model;

a current pixel from the current block has a current horizontal position in the reconstructed frame and a current vertical position in the reconstructed frame;

the optimal predicted warped model parameters include:

a horizontal translational motion parameter;

a vertical translational motion parameter;

a horizontal scaling parameter;

a vertical scaling parameter;

a first rotation parameter; and

a second rotation parameter;

a horizontal displacement for decoding the current block is a result of adding:

a result of multiplying the horizontal scaling parameter by the current horizontal position,

a result of multiplying the first rotation parameter by the current vertical position, and

the horizontal translational motion parameter; and

a vertical displacement for decoding the current block is a result of adding:

a result of multiplying the vertical scaling parameter by the current horizontal position,

a result of multiplying the second rotation parameter by the current vertical position, and

the vertical translational motion parameter.

17. The method of claim 13 , wherein:

generating the reconstructed frame includes obtaining, from the encoded bitstream, a flag indicating that the current block is encoded using a four-parameter warped motion model;

a current pixel from the current block has a current horizontal position in the reconstructed frame and a current vertical position in the reconstructed frame;

the optimal predicted warped model parameters include:

a horizontal translational motion parameter;

a vertical translational motion parameter;

a horizontal scaling parameter; and

a rotation parameter;

a horizontal displacement for decoding the current block is a result of:

adding:

a result of subtracting:

a result of multiplying the rotation parameter by the current vertical position, from

a result of multiplying the horizontal scaling parameter by the current horizontal position, and

the horizontal translational motion parameter; and

a vertical displacement for decoding the current block is a result of adding:

a result of multiplying the rotation parameter by the current horizontal position,

a result of multiplying the horizontal scaling parameter by the current vertical position, and

the vertical translational motion parameter.

18. The method of claim 13 , wherein obtaining the warped reference list includes:

determining a maximum cardinality of the warped reference list in accordance with a prediction mode for decoding the current block, wherein:

in response to a determination that the prediction mode is new motion vector mode, the maximum cardinality is four; and

in response to a determination that the prediction mode is other than new motion vector mode, the maximum cardinality is one.

19. The method of claim 13 , wherein generating the reconstructed frame includes:

obtaining, from the encoded bitstream, a maximum cardinality of the warped reference list for the reconstructed frame or for a current sequence of frames that includes the reconstructed frame.

20. An apparatus comprising:

a memory including computer executable instructions for decoding an encoded video stream; and

a processor that executes the instructions to:

obtain the encoded video stream;

generate a reconstructed frame, wherein to generate the reconstructed frame the processor executes the instructions to:

obtain a translational dynamic reference list for decoding a current block for the reconstructed frame;

obtain a warped reference list for decoding the current block;

decode a warped reference list index value from the encoded video stream;

obtain optimal predicted warped model parameters from the warped reference list in accordance with the warped reference list index value;

decode differential warped model parameters for the current block from the encoded video stream;

obtain, as optimal warped model parameters for the current block, a result of adding the optimal predicted warped model parameters and the differential warped model parameters;

obtain predicted block data for the current block in accordance with the optimal warped model parameters;

decode residual block data for the current block from the encoded video stream;

obtain, as decoded block data for the current block, a result of adding the residual block data and the predicted block data;

include the decoded block data in a reconstructed block of the reconstructed frame; and

output the reconstructed frame.

Assignments (2)

Continuity (1)

References Cited (16)