IP Library › Granted Patent US 12666074
Granted Patent B2
US 12666074 · App. 18/911,319 · Granted Jun 23, 2026

Systems, apparatus and methods for inter prediction refinement with optical flow

Inventors: Jiancong Luo (Skillman, NJ); Yuwen He (San Diego, CA)
Assignee: InterDigital VC Holdings, Inc.
H04N19/52H04N19/105H04N19/132H04N19/139H04N19/167H04N19/176H04N19/55
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12666074
App. No.
18/911,319
Granted
Jun 23, 2026
Kind
B2
Abstract

Method, apparatus and systems are disclosed. In one embodiment, a method of decoding includes obtaining a sub-block based motion prediction signal for a current block of the video; obtaining one or more spatial gradients of the sub-block based motion prediction signal or one or more motion vector difference values; obtaining a refinement signal for the current block based on the one or more obtained spatial gradients or the one or more obtained motion vector difference values; obtaining a refined motion prediction signal for the current block based on the sub-block based motion prediction signal and the refinement signal; and decoding the current block based on the refined motion prediction signal.

Claims (42)

1 . A method of decoding a video, comprising:

generating a sub-block-based motion prediction signal for a sub-block of a block of a picture based on an affine motion model associated with said block;

determining a set of pixel-level motion vector difference values for said sub-block using said affine motion model associated with said block;

determining spatial gradients of said sub-block-based motion prediction signal, for each sample location of said sub-block, wherein an extended sub-block is formed to include said sub-block-based motion prediction signal and a plurality of samples surrounding said sub-block, and wherein each of said plurality of samples surrounding said sub-block is obtained based on integer motion compensation;

determining a motion prediction refinement signal for said sub-block, based on said determined set of pixel-level motion vector difference values and said determined spatial gradients;

combining said motion prediction signal and said motion prediction refinement signal to produce a refined motion prediction signal for said sub-block; and

decoding the video using the refined motion prediction signal.

2 . The method of claim 1 , wherein said integer motion compensation is based on an integer part of a motion vector of said sub-block.

3 . The method of claim 1 , wherein said integer motion compensation is based on a nearest integer motion vector of a motion vector of said sub-block.

4 . The method of claim 1 , wherein said sub-block is extended by one sample in each direction to form said extended sub-block.

5 . A computer readable storage medium having stored thereon instructions for decoding video data according to the method of claim 1 .

6 . A method of encoding a video, comprising:

generating a sub-block-based motion prediction signal for a sub-block of a block of a picture based on an affine motion model associated with said block;

determining a set of pixel-level motion vector difference values for said sub-block using said affine motion model associated with said block;

determining spatial gradients of said sub-block-based motion prediction signal, for each sample location of said sub-block, wherein an extended sub-block is formed to include said sub-block-based motion prediction signal and a plurality of samples surrounding said sub-block, and wherein each of said plurality of samples surrounding said sub-block is obtained based on integer motion compensation;

determining a motion prediction refinement signal for said sub-block, based on said determined set of pixel-level motion vector difference values and said determined spatial gradients;

combining said motion prediction signal and said motion prediction refinement signal to produce a refined motion prediction signal for said sub-block; and

encoding the video using the refined motion prediction signal.

7 . The method of claim 6 , wherein said integer motion compensation is based on an integer part of a motion vector of said sub-block.

8 . The method of claim 6 , wherein said integer motion compensation is based on a nearest integer motion vector of a motion vector of said sub-block.

9 . The method of claim 6 , wherein said sub-block is extended by one sample in each direction to form said extended sub-block.

10 . A computer readable storage medium having stored thereon instructions for encoding video data according to the method of claim 6 .

11 . An apparatus for decoding a video, comprising a processor configured to:

generate a sub-block-based motion prediction signal for a sub-block of a block of a picture based on an affine motion model associated with said block;

determine a set of pixel-level motion vector difference values for said sub-block using said affine motion model associated with said block;

determine spatial gradients of said sub-block-based motion prediction signal, for each sample location of said sub-block, wherein an extended sub-block is formed to include said sub-block-based motion prediction signal and a plurality of samples surrounding said sub-block, and wherein each of said plurality of samples surrounding said sub-block is obtained based on integer motion compensation;

determine a motion prediction refinement signal for said sub-block, based on said determined set of pixel-level motion vector difference values and said determined spatial gradients;

combine said motion prediction signal and said motion prediction refinement signal to produce a refined motion prediction signal for said sub-block; and

decode the video using the refined motion prediction signal.

12 . The apparatus of claim 11 , wherein said integer motion compensation is based on an integer part of a motion vector of said sub-block.

13 . The apparatus of claim 11 , wherein said integer motion compensation is based on a nearest integer motion vector of a motion vector of said sub-block.

14 . The apparatus of claim 11 , wherein said sub-block is extended by one sample in each direction to form said extended sub-block.

15 . An apparatus for encoding a video, comprising a processor configured to:

generate a sub-block-based motion prediction signal for a sub-block of a block of a picture based on an affine motion model associated with said block;

determine a set of pixel-level motion vector difference values for said sub-block using said affine motion model associated with said block;

determine spatial gradients of said sub-block-based motion prediction signal, for each sample location of said sub-block, wherein an extended sub-block is formed to include said sub-block-based motion prediction signal and a plurality of samples surrounding said sub-block, and wherein each of said plurality of samples surrounding said sub-block is obtained based on integer motion compensation;

determine a motion prediction refinement signal for said sub-block, based on said determined set of pixel-level motion vector difference values and said determined spatial gradients;

combine said motion prediction signal and said motion prediction refinement signal to produce a refined motion prediction signal for said sub-block; and

encode the video using the refined motion prediction signal.

16 . The apparatus of claim 15 , wherein said integer motion compensation is based on an integer part of a motion vector of said sub-block.

17 . The apparatus of claim 15 , wherein said integer motion compensation is based on a nearest integer motion vector of a motion vector of said sub-block.

18 . The apparatus of claim 15 , wherein said sub-block is extended by one sample in each direction to form said extended sub-block.