IP Library › Granted Patent US 12,483,717
Granted Patent B2
US 12,483,717 · App. 18/435,982 · Granted Nov 25, 2025

Low-latency and high-throughput motion vector refinement with template matching

Inventor: Minhua Zhou (San Diego, CA)
Assignee: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
H04N19/44H04N19/105H04N19/132H04N19/159H04N19/176H04N19/513
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,483,717
App. No.
18/435,982
Granted
Nov 25, 2025
Kind
B2
Abstract

A video decoder is provided that includes memory and a processor coupled to the memory. The processor may be configured to convert a bitstream into inter-prediction parameters and reconstruct motion data based on the inter-prediction parameters. The processor may further be configured to refine the motion data based on finding a match between a current template of a current picture and a reference template of a reference picture and perform a motion compensation operation with the refined motion data and a reference block to generate an inter-prediction block. The processor may be configured to add the inter-prediction block to an inter-residual block to produce a reconstructed block. The motion data may be reconstructed without refined motion data associated with a previous prediction unit.

Claims (56)

1 . A video decoder, comprising:

memory; and

a processor coupled to the memory and configured to:

convert a bitstream into inter-prediction parameters;

reconstruct motion data based on the inter-prediction parameters;

perform a first motion compensation operation with the reconstructed motion data and a reference block to generate a first inter-prediction block;

refine the reconstructed motion data based on finding a match between a current template of a current picture and a reference template of a reference picture;

perform a second motion compensation operation with the refined motion data and the reference block to generate a second inter-prediction block; and

add the second inter-prediction block to an inter-residual block to produce a reconstructed block,

wherein the motion data is reconstructed without refined motion data associated with a previous prediction unit.

2 . The video decoder of claim 1 , wherein the processor is further configured to:

fetch a first set of reference blocks from the memory based on the reconstructed motion data;

determine a second set of reference blocks to be accessed in the second motion compensation operation with the refined motion data includes at least one reference block not in the first set of reference blocks; and

use padding for the at least one reference block in the second motion compensation operation.

3 . The video decoder of claim 1 , wherein the processor is further configured to:

pass inter-residual blocks and intra-residual blocks to an inter-prediction stage; and

pass the intra-residual blocks from the inter-prediction stage to an intra-prediction stage.

4 . The video decoder of claim 3 , wherein the inter-prediction block is added to the inter-residual block to produce the reconstructed block in the inter-prediction stage.

5 . The video decoder of claim 1 , wherein the current template of the current picture comprises inter-prediction samples from neighboring prediction units, and

wherein the processor is further configured to exclude from the current template samples from neighboring prediction units that are intra-coded.

6 . The video decoder of claim 5 , wherein the inter-prediction samples from neighboring prediction units are generated using the reconstructed motion data.

7 . The video decoder of claim 5 , wherein the processor is further configured to:

determine all of the samples in the current template are from neighboring prediction units that are intra-coded;

disable refining the motion data; and

perform the second motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

8 . The video decoder of claim 1 , wherein the processor is further configured to:

determine a size of a current prediction unit does not meet a threshold;

disable refining the motion data; and

perform the second motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

9 . A method, comprising:

converting a bitstream into inter-prediction parameters;

reconstructing motion data based on the inter-prediction parameters;

performing a first motion compensation operation with the reconstructed motion data and a reference block to generate a first inter-prediction block;

refining the motion data based on finding a match between a current template of a current picture and a reference template of a reference picture;

performing a second motion compensation operation with the refined motion data and the reference block to generate a second inter-prediction block; and

adding the second inter-prediction block to an inter-residual block to produce a reconstructed block,

wherein the motion data is reconstructed without refined motion data associated with a previous prediction unit.

10 . The method of claim 9 , further comprising:

fetching a first set of reference blocks from the memory based on the reconstructed motion data;

determining a second set of reference blocks to be accessed in the second motion compensation operation with the refined motion data includes at least one reference block not in the first set of reference blocks; and

using padding for the at least one reference block in the second motion compensation operation.

11 . The method of claim 9 , further comprising:

passing inter-residual blocks and intra-residual blocks to an inter-prediction stage; and

passing the intra-residual blocks from the inter-prediction stage to an intra-prediction stage.

12 . The method of claim 11 , wherein the inter-prediction block is added to the inter-residual block to produce the reconstructed block in the inter-prediction stage.

13 . The method of claim 9 , wherein the current template of the current picture comprises inter-prediction samples from neighboring prediction units, and

wherein the method further comprises excluding from the current template samples from neighboring prediction units that are intra-coded.

14 . The method of claim 13 , wherein the inter-prediction samples from neighboring prediction units are generated using the reconstructed motion data.

15 . The method of claim 13 , further comprising:

determining all of the samples in the current template are from neighboring prediction units that are intra-coded;

disabling refining the motion data; and

performing the second motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

16 . The method of claim 9 , further comprising:

determining a size of a current prediction unit does not meet a threshold;

disabling refining the motion data; and

performing the second motion compensation operation with the reconstructed motion data and the reference block to generate the inter-prediction block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2024
From: ZHOU, MINHUA
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 066422/0411 →
Continuity (2)
Continuation 17488094 · Sep 28, 2021
Related Publication 20240179335A1 · May 30, 2024
References Cited (11)
US 11297340B2 · Chien et al. · 2022 [cited by applicant]
US 11917176B2 · Zhou · 2024 [cited by examiner]
US 20190238883A1 · Chen et al. · 2019 [cited by applicant]
US 20190394483A1 · Zhou et al. · 2019 [cited by applicant]
US 20200092545A1 · Xu et al. · 2020 [cited by applicant]
US 20210274213A1 · Xiu et al. · 2021 [cited by applicant]
US 20230007238A1 · Chen · 2023 [cited by examiner]
WO 2020063598A1 · 2020 [cited by applicant]
Xiaoyu Xiu et al, “CE9-2. 2: Disabled decoder-side motion refinement (DMVR) and bidirectional optical flow (BDOF) for combined inter and intra prediction (CIIP)”, 15. JVET Meeting; Jul. 3, 2019-Jul. 12, 2019; Gothenburg… [cited by applicant]
Office Action, Chinese Patent Application No. 202211174864.6, Jul. 18, 2024. [cited by applicant]
Office Action, Chinese Patent Application No. 202211174864.6, Dec. 30, 2024. [cited by applicant]