IP Library Granted Patent US 11,627,323
Granted Patent B2
US 11,627,323 · App. 17/410,371 · Granted Apr 11, 2023

DMVR-based inter-prediction method and device

Inventors: Naeri Park (Seoul, KR); Junghak Nam (Seoul, KR)
Assignee: LG ELECTRONICS INC.
H04N19/139H04N19/159H04N19/176H04N19/513
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,627,323
App. No.
17/410,371
Granted
Apr 11, 2023
Kind
B2
Abstract

An image decoding method comprises the steps of: determining whether or not an application condition of decoder-side motion vector refinement (DMVR) for applying motion vector refinement for a current block is satisfied; deriving a minimum sum of absolute differences (SAD) on the basis of L0 and L1 motion vectors of the current block if the application condition of the DMVR is satisfied; deriving refined L0 and L1 motion vectors of the current block based on the minimum SAD; deriving prediction samples of the current block based on the refined L0 and L1 motion vectors; and generating reconstructed samples of the current block based on the prediction samples. With respect to whether or not the application condition of the DMVR is satisfied, the application condition of the DMVR is determined to be satisfied if a prediction mode, in which inter-prediction and intra-prediction are combined, is not applied to the current block.

Claims (47)

1. An image decoding method performed by a decoding apparatus, the image decoding method comprising:

determining whether an application condition of a decoder-side motion vector refinement (DMVR) for applying a motion vector refinement to a current block is satisfied;

deriving a minimum sum of absolute differences (SAD) based on an L0 motion vector and an L1 motion vector of the current block based on a case where the application condition of the DMVR is satisfied;

deriving a refined L0 motion vector and a refined L1 motion vector for the current block based on the minimum SAD;

deriving prediction samples for the current block based on the refined L0 motion vector and the refined L1 motion vector; and

generating reconstructed samples for the current block based on the prediction samples,

wherein the determining of whether the application condition of the DMVR is satisfied determines based on whether a combined inter and intra prediction (CIIP) mode in which an inter prediction and an intra prediction are combined is applied to the current block, wherein the DMVR is applied to the current block based on a case in which the CIIP mode is not applied to the current block,

wherein the deriving of the prediction samples includes determining whether an application condition of a bi-directional optical flow (BDOF) for applying a refinement to the prediction samples is satisfied, and

wherein the determining of whether the application condition of the BDOF is satisfied determines based on whether the CIIP mode is applied to the current block, wherein the BDOF is applied to the prediction samples based on a case in which the CIIP mode is not applied to the current block.

2. The image decoding method of claim 1 , further comprising obtaining, from a bitstream, combined inter and intra prediction (CIIP) flag information related to representing whether the CIIP mode is applied to the current block,

wherein the determining of whether the application condition of the DMVR is satisfied determines that the application condition of the DMVR is satisfied based on a value of the CIIP flag information being equal to 0.

3. The image decoding method of claim 1 , wherein the determining of whether the application condition of the DMVR is satisfied determines that 1) an inter bi-prediction based on the DMVR is enabled, 2) a merge mode is applied to the current block, 3) a merge mode with motion vector difference (MMVD) mode is not applied to the current block, 4) a bi-prediction being performed based on an L0 reference picture and an L1 reference picture is applied to the current block, 5) a distance from the L0 reference picture and a distance from the L1 reference picture are equal to each other based on a current picture, 6) a value of bi-prediction weight index information of the current block is equal to 0, 7) a height of the current block is equal to or larger than 8, or 8) the height x width of the current block is larger than 8×8, and

wherein the determining of whether the application condition of the DMVR is satisfied determines that the application condition of the DMVR is satisfied in case that the 1) to 8) are all satisfied.

4. The image decoding method of claim 2 , wherein the determining of whether the application condition of the BDOF is satisfied determines that the application condition of the BDOF is satisfied based on the value of the CIIP flag information being equal to 0.

5. The image decoding method of claim 1 , wherein the determining of whether the application condition of the BDOF is satisfied determines that 1) an inter prediction based on the BDOF is enabled, 2) a subblock-based merge mode is not applied to the current block, 3) a bi-prediction being performed based on an L0 reference picture and an L1 reference picture is applied to the current block, 4) a distance from the L0 reference picture and a distance from the L1 reference picture are equal to each other based on a current picture, 5) a value of bi-prediction weight index information of the current block is equal to 0, 6) an affine mode is not applied to the current block, 7) a height of the current block is equal to or larger than 8, or 8) the height x width of the current block is larger than 8×8, and

wherein the determining of whether the application condition of the BDOF is satisfied determines that the application condition of the BDOF is satisfied in case that the 1) to 8) are all satisfied.

6. The image decoding method of claim 1 , wherein the deriving of the minimum SAD comprises:

calculating a sample area having the minimum SAD between prediction samples in an L0 reference picture being derived based on the L0 motion vector and prediction samples in an L1 reference picture being derived based on the L1 motion vector; and

performing a refinement for the L0 motion vector and the L1 motion vector based on the sample area having the minimum SAD.

7. The image decoding method of claim 6 , further comprising determining whether a merge mode is applied to the current block,

wherein in case that the merge mode is applied to the current block, the L0 motion vector and the L1 motion vector are derived based on a motion vector of a neighboring block indicated by a merge index among neighboring blocks of the current block, and

the refined L0 motion vector and the refined L1 motion vector are derived in case that a distance between the L0 reference picture and the current picture and a distance between the L1 reference picture and the current picture are equal to each other.

8. The image decoding method of claim 7 , further comprising determining whether a bi-prediction is applied to the neighboring block indicated by the merge index in case that the merge mode is applied to the current block,

wherein in case that the bi-prediction is applied to the neighboring block indicated by the merge index, the motion vector of the neighboring block indicated by the merge index includes a motion vector in an L0 prediction direction and a motion vector in an L1 prediction direction.

9. An image encoding method performed by an encoding apparatus, the image encoding method comprising:

determining whether an application condition of a decoder-side motion vector refinement (DMVR) for applying a motion vector refinement to a current block is satisfied;

deriving a minimum sum of absolute differences (SAD) based on an L0 motion vector and an L1 motion vector of the current block based on a case where the application condition of the DMVR is satisfied;

deriving a refined L0 motion vector and a refined L1 motion vector for the current block based on the minimum SAD;

deriving prediction samples for the current block based on the refined L0 motion vector and the refined L1 motion vector;

deriving residual samples based on the prediction samples; and

encoding image information including information on the residual samples,

wherein the determining of whether the application condition of the DMVR is satisfied determines based on whether a combined inter and intra prediction (CIIP) mode in which an inter prediction and an intra prediction are combined is applied to the current block, wherein the DMVR is applied to the current block based on a case in which the CIIP mode is not applied to the current block,

wherein the deriving of the prediction samples includes determining whether an application condition of a bi-directional optical flow (BDOF) for applying a refinement to the prediction samples is satisfied, and

wherein the determining of whether the application condition of the BDOF is satisfied determines based on whether the CIIP mode is applied to the current block, wherein the BDOF is applied to the prediction samples based on a case in which the CIIP mode is not applied to the current block.

10. The image encoding method of claim 9 , wherein the image information includes combined inter and intra prediction (CIIP) flag information related to representing whether the CIIP mode is applied to the current block, and is encoded.

11. The image encoding method of claim 10 , wherein the determining of whether the application condition of the DMVR is satisfied determines that the application condition of the DMVR is satisfied based on a value of the CIIP flag information being equal to 0.

12. The image encoding method of claim 10 , wherein the determining of whether the application condition of the BDOF is satisfied determines that the application condition of the BDOF is satisfied based on a value of CIIP flag information being equal to 0.

13. A non-transitory computer-readable digital storage medium storing a bitstream generated by a method, the method comprising:

determining whether an application condition of a decoder-side motion vector refinement (DMVR) for applying a motion vector refinement to a current block is satisfied;

deriving a minimum sum of absolute differences (SAD) based on an L0 motion vector and an L1 motion vector of the current block based on a case where the application condition of the DMVR is satisfied;

deriving a refined L0 motion vector and a refined L1 motion vector for the current block based on the minimum SAD;

deriving prediction samples for the current block based on the refined L0 motion vector and the refined L1 motion vector;

deriving residual samples based on the prediction samples; and

encoding image information including information on the residual samples to generate the bitstream,

wherein the determining of whether the application condition of the DMVR is satisfied determines based on whether a combined inter and intra prediction (CIIP) mode in which an inter prediction and an intra prediction are combined is applied to the current block, wherein the DMVR is applied to the current block based on a case in which the CIIP mode is not applied to the current block,

wherein the deriving of the prediction samples includes determining whether an application condition of a bi-directional optical flow (BDOF) for applying a refinement to the prediction samples is satisfied, and

wherein the determining of whether the application condition of the BDOF is satisfied determines based on whether the CIIP mode is applied to the current block, wherein the BDOF is applied to the prediction samples based on a case in which the CIIP mode is not applied to the current block.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S COUNTRY OF RECORD ON COVER SHEET PREVIOUSLY RECORDED AT REEL: 057306 FRAME: 0655. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 1, 2021
From: PARK, NAERI; NAM, JUNGHAK
To: LG ELECTRONICS INC.
Reel/Frame 058295/0441 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2021
From: PARK, NAERI; NAM, JUNGHAK
To: LG ELECTRONICS INC.
Reel/Frame 057306/0655 →
Continuity (3)
Continuation PCTKR2020001785 · Feb 7, 2020
Provisional Application 62809722 · Feb 24, 2019
Related Publication 20220224911A1 · Jul 14, 2022