IP Library › Granted Patent US 10,951,912

Granted Patent B2

US 10,951,912 · App. 15/713,112 · Granted Mar 16, 2021

Systems and methods for adaptive selection of weights for video coding

Inventors: Hsiao-Chiang Chuang (San Diego, CA); Jianle Chen (San Diego, CA); Xiang Li (San Diego, CA); Kai Zhang (San Diego, CA); Done Bugdayci Sansli (Tampere, FI); Marta Karczewicz (San Diego, CA); Wei-Jung Chien (San Diego, CA); Yu-Chen Sun (San Diego, CA)

Assignee: Qualcomm Incorporated

H04N19/52 H04N19/105 H04N19/11 H04N19/126 H04N19/137 H04N19/139 H04N19/159 H04N19/176 H04N19/186 H04N19/198 H04N19/43 H04N19/44 H04N19/51 H04N19/553 H04N19/577 H04N19/61 H04N19/625 H04N19/119

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,951,912
App. No.: 15/713,112
Granted: Mar 16, 2021
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Techniques and systems are provided for processing video data. For example, a current block of a picture of the video data can be obtained for processing by an encoding device or a decoding device. A pre-defined set of weights for template matching based motion compensation are also obtained. A plurality of metrics associated with one or more spatially neighboring samples of the current block and one or more spatially neighboring samples of at least one reference frame are determined. A set of weights are selected from the pre-defined set of weights to use for the template matching based motion compensation. The set of weights is determined based on the plurality of metrics. The template matching based motion compensation is performed for the current block using the selected set of weights.

Claims (84)

1. A method of processing video data, the method comprising:

obtaining a current block of a picture of the video data;

obtaining a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights;

applying a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value;

applying a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value;

based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determining a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame;

applying a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value;

applying a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value;

based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determining a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame;

comparing the first metric and the second metric;

determining the first metric is smaller than the second metric;

selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and

generating a template matching based motion compensation prediction for the current block based on the selected pair of weights.

2. The method of claim 1 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

wherein the second metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame.

3. The method of claim 1 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

wherein the second metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame.

4. The method of claim 1 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

wherein the second metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the second reference frame.

5. The method of claim 1 , wherein no weighting parameter is signaled with the video data.

6. The method of claim 1 , wherein the template matching based motion compensation includes local illumination compensation (LIC).

7. The method of claim 1 , wherein the template matching based motion compensation includes weighted prediction (WP).

8. An apparatus comprising:

a memory configured to store video data; and

a processor configured to:

obtain a current block of a picture of the video data;

obtain a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights;

apply a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value;

apply a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value;

based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame;

apply a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value;

apply a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value;

based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame;

compare the first metric and the second metric;

determine the first metric is smaller than the second metric;

selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and

generate a template matching based motion compensation prediction for the current block based on the template matching based motion compensation for the current block using the selected pair of weights.

9. The apparatus of claim 8 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

10. The apparatus of claim 8 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

11. The apparatus of claim 8 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

12. The apparatus of claim 8 , wherein no weighting parameter is signaled with the video data.

13. The apparatus of claim 8 , wherein the template matching based motion compensation includes local illumination compensation (LIC).

14. The apparatus of claim 8 , wherein the template matching based motion compensation includes weighted prediction (WP).

15. The apparatus of claim 8 , further comprising:

a display for displaying the video data.

16. The apparatus of claim 8 , wherein the apparatus comprises a mobile device with a camera for capturing pictures.

17. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

obtain a current block of a picture of video data;

based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame;

based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, determine a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame;

compare the first metric and the second metric;

determine the first metric is smaller than the second metric;

select the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and

generate a template matching based motion compensation prediction for the current block based on the selected pair of weights.

18. The non-transitory computer-readable medium of claim 17 , wherein the first metric includes a sum of absolute differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

19. The non-transitory computer-readable medium of claim 17 , wherein the first metric includes a sum of absolute transformed differences between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

20. The non-transitory computer-readable medium of claim 17 , wherein the first metric includes a sum of squared errors of prediction between the one or more spatially neighboring samples of the current block and the one or more spatially neighboring samples of at least the first reference frame; and

21. The non-transitory computer-readable medium of claim 17 , wherein no weighting parameter is signaled with the video data.

22. The non-transitory computer-readable medium of claim 17 , wherein the template matching based motion compensation includes local illumination compensation (LIC).

23. The non-transitory computer-readable medium of claim 17 , wherein the template matching based motion compensation includes weighted prediction (WP).

24. An apparatus for processing video data, comprising:

means for obtaining a current block of a picture of the video data;

means for obtaining a pre-defined set of weights including pre-defined values for template matching based motion compensation, the pre-defined set of weights comprising at least a first pair of weights and a second pair of weights;

means for applying a first weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a first reference frame, the first weight of the first pair of weights having a first pre-defined value;

means for applying a second weight of the first pair of weights of the pre-defined set of weights to one or more spatially neighboring samples of a second reference frame, the second weight of the first pair of weights having a second pre-defined value;

means for determining, based on applying the first weight of the first pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the first pair of weights to the one or more spatially neighboring samples of the second reference frame, a first metric associated with one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of a first reference frame, and the one or more spatially neighboring samples of a second reference frame;

means for applying a first weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the first reference frame, the first weight of the second pair of weights having a third pre-defined value;

means for applying a second weight of the second pair of weights of the pre-defined set of weights to the one or more spatially neighboring samples of the second reference frame, the second weight of the second pair of weights having a fourth pre-defined value;

means for determining, based on applying the first weight of the second pair of weights to the one or more spatially neighboring samples of the first reference frame and applying the second weight of the second pair of weights to the one or more spatially neighboring samples of the second reference frame, a second metric associated with the one or more spatially neighboring samples of the current block, the one or more spatially neighboring samples of the first reference frame, and the one or more spatially neighboring samples of the second reference frame;

means for comparing the first metric and the second metric;

means for determining the first metric is smaller than the second metric;

means for selecting the first pair of weights as a selected pair of weights based on the first metric being smaller than the second metric; and

means for generating a template matching based motion compensation prediction for the current block based on the selected pair of weights the template matching based motion compensation for the current block using the selected pair of weights.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 6, 2017

From: CHUANG, HSIAO-CHIANG; CHEN, JIANLE; LI, XIANG; ZHANG, KAI; BUGDAYCI SANSLI, DONE; KARCZEWICZ, MARTA; CHIEN, WEI-JUNG; SUN, YU-CHEN

To: QUALCOMM INCORPORATED

Reel/Frame 044044/0625 →

Continuity (2)

Provisional Application 62404715 · Oct 5, 2016

Related Publication 20180098070A1 · Apr 5, 2018