IP Library Granted Patent US 11,310,526
Granted Patent B2
US 11,310,526 · App. 16/248,452 · Granted Apr 19, 2022

Hardware friendly constrained motion vector refinement

Inventors: Chun-Chia Chen (Hsinchu, TW); Zhen-Yen Lai (Hsinchu, TW); Chih-Wei Hsu (Hsinchu, TW); Tzu-Der Chuang (Hsinchu, TW); Ching-Yeh Chen (Hsinchu, TW)
Assignee: MEDIATEK INC.
H04N19/521H04N19/105H04N19/139H04N19/176H04N19/573H04N19/147H04N19/567
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,310,526
App. No.
16/248,452
Filed
Jan 15, 2019
Granted
Apr 19, 2022
Kind
B2
Art Unit
2488
USPC
375/240.16
Abstract

A video codec that encodes or decodes video sequences using decoder-side motion vector refinement is provided. The video codec identifies a first motion vector and a second motion vector for coding a current block of pixels of a current video frame in the video sequence. The video codec determines whether to perform motion vector refinement for the current block of pixels based on a comparison between a linear dimension of the current block of pixels and a threshold. When motion vector refinement is performed, the video codec refines the first and second motion vectors to minimize a distortion cost and codes the current block of pixels by using the refined first and second motion vectors. When motion vector refinement is not performed, the video codec codes the current block of pixels by using the identified first and second motion vectors.

Claims (46)

1. A method for encoding or decoding a video sequence, the method comprising:

determining whether to partition the current block of pixels for motion vector refinement based on whether the current block of pixels meet a particular condition;

when the current block of pixels is partitioned for motion vector refinement:

partitioning the current block of pixels into a plurality of partitioned units; and

for every partitioned unit among the plurality of partitioned units;

identifying a first motion vector and a second motion vector;

refining the first and second motion vectors;

computing distortion costs of the refined first motion vector and the refined second motion vector based on a difference between a first reference pixel set in a first reference frame indicated by the refined first motion vector and a second reference pixel set in a second reference frame indicated by the refined second motion vector, respectively, wherein the pixel sets are sub-sampling pixel data of a subset of the pixel data in only odd or even numbered lines of the pixel data according to the refined first and second motion vectors,

wherein, when a plurality of distortion costs are calculated for the partitioned unit, deriving a minimized distortion cost based on the plurality of distortion costs, and obtaining a corresponding refined first and second motion vectors based on the minimized distortion cost; and

coding the plurality of partitioned units individually by using the corresponding refined first and second motion vectors;

when the current block of pixels is not partitioned for motion vector refinement:

identifying a pair of motion vectors for coding the current block of pixels; and

coding the current block of pixels by using the identified pair of motion vectors without refining the pair of motion vectors.

2. The method of claim 1 , wherein the current block of pixels is partitioned for motion vector refinement when a linear dimension of the current block of pixels is greater than the threshold.

3. The method of claim 1 , wherein the current block of pixels is partitioned for motion vector refinement when an area of the current block of pixels is greater than the threshold.

4. The method of claim 1 , wherein the current block of pixels is not partitioned for motion vector refinement when the current block of pixels is contained by a predetermined grid unit.

5. The method of claim 1 , wherein the distortion cost is computed based on the sub-sampled pixel data pointed at by the first and second motion vectors.

6. The method of claim 1 , wherein:

the first motion vector is indicative of a first reference pixel set in a first reference frame and the second motion vector is indicative of a second reference pixel set in a second reference frame, and

the distortion cost is computed based on a set of neighboring pixels of the current block of pixels, a set of neighboring pixels of the first reference pixel set, and a set of neighboring pixels of the second reference pixel set.

7. The method of claim 1 , wherein the refined first motion vector and the refined second motion vector are based on opposite search positions relative to a central search position.

8. The method of claim 1 , further comprising:

determining whether to perform motion vector refinement for the current block of pixels based on a comparison between a size of the current block of pixels and a threshold;

When motion vector refinement is performed:

coding the current block of pixels by using the refined first and second motion vectors; and

when motion vector refinement is not performed:

coding the current block of pixels by using the identified first and second motion vectors.

9. The method of claim 1 , further comprising:

calculating an initial cost that corresponds to central points pointed at by the first and second motion vectors;

calculating additional costs corresponding to pixel positions in plus sign shape via predictions located at opposite sides of a central point;

determining a last point based on a gradient of previous calculated costs; and

outputting the first and second refined motion vectors corresponding to the minimal cost.

10. An electronic apparatus comprising:

circuitry configured to:

determine whether to partition the current block of pixels for motion vector refinement based on whether the current block of pixels meet a particular condition;

when the current block of pixels is partitioned for motion vector refinement, the circuitry is further configured to:

partition the current block of pixels into a plurality of partitioned units; and

for every partitioned unit among the plurality of partitioned units:

identify a first motion vector and a second motion vector;

refine the first and second motion vectors;

compute distortion costs of the refined first motion vector and the refined second motion vector based on a difference between a first reference pixel set in a first reference frame indicated by the refined first motion vector and a second reference pixel set in a second reference frame indicated by the refined second motion vector, respectively, wherein the pixel sets are sub-sampling pixel data of a subset of the pixel data in only odd or even numbered lines of the pixel data according to the first and second motion vectors,

wherein, when a plurality of distortion costs are calculated for the partitioned unit, the circuitry is further configured to derive a minimized distortion cost based on the plurality of distortion costs, and obtain a corresponding refined first and second motion vectors based on the minimized distortion cost; and

code the plurality of partitioned units individually by using the corresponding refined first and second motion vectors;

when the current block of pixels is not partitioned for motion vector refinement, the circuitry is further configured to:

identify a pair of motion vectors for coding the current block of pixels, and

code the current block of pixels by using the identified pair of motion vectors without refining the pair of motion vectors.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2022
From: MEDIATEK INC.
To: HFI INNOVATION INC.
Reel/Frame 059369/0481 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2019
From: CHEN, CHUN-CHIA; LAI, ZHEN-YEN; HSU, CHIH-WEI; CHUANG, TZU-DER; CHEN, CHING-YEH
To: MEDIATEK INC.
Reel/Frame 048366/0082 →
Continuity (4)
Provisional Application 62740470 · Oct 3, 2018
Provisional Application 62723014 · Oct 3, 2018
Provisional Application 62622234 · Jan 6, 2018
Related Publication 20190238883A1 · Aug 1, 2019
Cited By (1)
US 12,348,731