IP Library Granted Patent US 11,172,229
Granted Patent B2
US 11,172,229 · App. 16/244,843 · Granted Nov 9, 2021

Affine motion compensation with low bandwidth

Inventors: Kai Zhang (San Diego, CA); Wei-Jung Chien (San Diego, CA); Li Zhang (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: Qualcomm Incorporated
H04N19/615H04N19/105H04N19/119H04N19/139H04N19/176H04N19/433H04N19/51H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,172,229
App. No.
16/244,843
Granted
Nov 9, 2021
Kind
B2
Abstract

An example device for coding video data is configured to determine that a block of the video data includes a plurality of sub-blocks, each of the sub-blocks having respective motion information referring to respective reference blocks in a reference picture in a memory, determine a single reference block of the reference picture, the single reference block including each of the respective reference blocks, wherein determining the single reference block comprises: determine four corner sub-blocks of the block included in the plurality of sub-blocks; and determine the single reference block according to the respective motion information for the four corner sub-blocks such that corners of the single reference block correspond to corners of the respective reference blocks of the four corner sub-blocks, retrieve data of the single reference block from the reference picture, and predict the sub-blocks from the respective reference blocks using the data of the single reference block.

Claims (68)

1. A method of coding video data, the method comprising:

determining that a block of video data includes a plurality of sub-blocks, each of the sub-blocks having respective motion information;

determining a respective sub-block, within the block that includes the plurality of sub-blocks, at each respective corner of the block, the determined sub-blocks being four corner sub-blocks;

determining, within a reference picture, a single reference block according to the respective motion information of each of the four corner sub-blocks such that corners of the single reference block correspond to corners of respective reference blocks, within the single reference block, referred to by the respective motion information of the four corner sub-blocks;

retrieving data of the single reference block from the reference picture, the single reference block including each of the respective reference blocks; and

predicting the sub-blocks from the respective reference blocks using the data of the single reference block.

2. The method of claim 1 , wherein determining the single reference block comprises determining the single reference block to have a smallest size possible while also including each of the respective reference blocks.

3. The method of claim 1 , wherein determining the single reference block comprises:

determining an upper edge of the single reference block to be equal to a highest upper vertex of the reference blocks for the four corner sub-blocks;

determining a lower edge of the single reference block to be equal to a lowest lower vertex of the reference blocks for the four corner sub-blocks;

determining a left edge of the single reference block to be equal to a leftmost vertex of the reference blocks for the four corner sub-blocks; and

determining a right edge of the single reference block to be equal to a rightmost vertex of the reference blocks for the four corner sub-blocks.

4. The method of claim 1 , wherein determining the single reference block comprises determining the single reference block to include additional samples of the reference picture beyond vertices of at least one of the reference blocks for the four corner sub-blocks.

5. The method of claim 1 , wherein determining the single reference block comprises determining the single reference block to be no larger than an allowable maximum size, wherein the allowable maximum size is predetermined, signaled in a parameter set, or determined based on characteristics of the block of video data.

6. The method of claim 1 , wherein predicting the sub-blocks comprises forming prediction blocks for the sub-blocks from the respective reference blocks, the method further comprising decoding the sub-blocks using the prediction blocks.

7. The method of claim 1 , wherein predicting the sub-blocks comprises forming prediction blocks for the sub-blocks from the respective reference blocks, the method further comprising encoding the sub-blocks using the prediction blocks.

8. A device for coding video data, the device comprising:

a memory configured to store video data; and

a processing unit implemented in circuitry and configured to:

determine that a block of the video data includes a plurality of sub-blocks, each of the sub-blocks having respective motion information;

determine a respective sub-block, within the bloc that includes the plurality of sub-blocks, at each respective corner of the block, the determined sub-blocks being four corner sub-blocks;

determine, within a reference picture stored in the memory, a single reference block according to the respective motion information of each of the four corner sub-blocks such that corners of the single reference block correspond to corners of respective reference blocks, within the single reference block, referred to by the respective motion information of the four corner sub-blocks;

retrieve data of the single reference block from the reference picture stored in the memory, the single reference block including each of the respective reference blocks; and

predict the sub-blocks from the respective reference blocks using the data of the single reference block.

9. The device of claim 8 , wherein the processing unit is configured to determine the single reference block to have a smallest size possible while also including each of the respective reference blocks.

10. The device of claim 8 , wherein to determine the single reference block, the processing unit is configured to:

determine an upper edge of the single reference block to be equal to a highest upper vertex of the reference blocks for the four corner sub-blocks;

determine a lower edge of the single reference block to be equal to a lowest lower vertex of the reference blocks for the four corner sub-blocks;

determine a left edge of the single reference block to be equal to a leftmost vertex of the reference blocks for the four corner sub-blocks; and

determine a right edge of the single reference block to be equal to a rightmost vertex of the reference blocks for the four corner sub-blocks.

11. The device of claim 8 , wherein the processing unit is configured to determine the single reference block to include additional samples of the reference picture beyond vertices of at least one of the reference blocks for the four corner sub-blocks.

12. The device of claim 8 , wherein the single processing unit is configured to determine the single reference block to be no larger than an allowable maximum size, wherein the allowable maximum size is predetermined, signaled in a parameter set, or determined based on characteristics of the block of video data.

13. The device of claim 8 , wherein to predict the sub-blocks, the processing unit is configured to form prediction blocks for the sub-blocks from the respective reference blocks, wherein the processing unit is further configured to decode the sub-blocks using the prediction blocks.

14. The device of claim 8 , wherein to predict the sub-blocks, the processing unit is configured to form prediction blocks for the sub-blocks from the respective reference blocks, wherein the processing unit is further configured to encode the sub-blocks using the prediction blocks.

15. The device of claim 8 , further comprising a display configured to display the video data.

16. The device of claim 8 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

17. A device for coding video data, the device comprising:

means for determining that a block of video data includes a plurality of sub-blocks, each of the sub-blocks having respective motion information;

means for determining a respective sub-block, within the block that includes the plurality of sub-blocks, at each respective corner of the block, the determined sub-blocks being four corner sub-blocks; and

means for determining, within a reference picture, a single reference block according to the respective motion information of each of the four corner sub-blocks such that corners of the single reference block correspond to corners of respective reference blocks, within the single reference block, referred to by the respective motion information of the four corner sub-blocks;

means for retrieving data of the single reference block from the reference picture, the single reference block including each of the respective reference blocks; and

means for predicting the sub-blocks from the respective reference blocks using the data of the single reference block.

18. The device of claim 17 , wherein the means for determining the single reference block comprises means for determining the single reference block to have a smallest size possible while also including each of the respective reference blocks.

19. The device of claim 17 , wherein the means for determining the single reference block comprises:

means for determining an upper edge of the single reference block to be equal to a highest upper vertex of the reference blocks for the four corner sub-blocks;

means for determining a lower edge of the single reference block to be equal to a lowest lower vertex of the reference blocks for the four corner sub-blocks;

means for determining a left edge of the single reference block to be equal to a leftmost vertex of the reference blocks for the four corner sub-blocks; and

means for determining a right edge of the single reference block to be equal to a rightmost vertex of the reference blocks for the four corner sub-blocks.

20. The device of claim 17 , wherein the means for determining the single reference block comprises means for determining the single reference block to include additional samples of the reference picture beyond vertices of at least one of the reference blocks for the four corner sub-blocks.

21. The device of claim 17 , wherein the means for determining the single reference block comprises means for determining the single reference block to be no larger than an allowable maximum size, wherein the allowable maximum size is predetermined, signaled in a parameter set, or determined based on characteristics of the block of video data.

22. The device of claim 17 , wherein the means for predicting the sub-blocks comprises means for forming prediction blocks for the sub-blocks from the respective reference blocks, further comprising means for decoding the sub-blocks using the prediction blocks.

23. The device of claim 17 , wherein the means for predicting the sub-blocks comprises means for forming prediction blocks for the sub-blocks from the respective reference blocks, the method further comprising means for encoding the sub-blocks using the prediction blocks.

24. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a device for decoding video data to:

determine that a block of video data includes a plurality of sub-blocks, each of the sub-blocks having respective motion information;

determine a respective sub-block, within the block that includes the plurality of sub-blocks, at each respective corner of the block, the determined sub-blocks being four corner sub-blocks;

determine, within a reference picture, a single reference block according to the respective motion information of each of the four corner sub-blocks such that corners of the single reference block correspond to corners of respective reference blocks, within the single reference block, referred to by the respective motion information of the four corner sub-blocks;

retrieve data of the single reference block from the reference picture, the single reference block including each of the respective reference blocks; and

predict the sub-blocks from the respective reference blocks using the data of the single reference block.

25. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to determine the single reference block comprise instructions that cause the processor to determine the single reference block to have a smallest size possible while also including each of the respective reference blocks.

26. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to determine the single reference block comprise instructions that cause the processor to:

determine an upper edge of the single reference block to be equal to a highest upper vertex of the reference blocks for the four corner sub-blocks;

determine a lower edge of the single reference block to be equal to a lowest lower vertex of the reference blocks for the four corner sub-blocks;

determine a left edge of the single reference block to be equal to a leftmost vertex of the reference blocks for the four corner sub-blocks; and

determine a right edge of the single reference block to be equal to a rightmost vertex of the reference blocks for the four corner sub-blocks.

27. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to determine the single reference block comprise instructions that cause the processor to determine the single reference block to include additional samples of the reference picture beyond vertices of at least one of the reference blocks for the four corner sub-blocks.

28. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to determine the single reference block comprise instructions that cause the processor to determine the single reference block to be no larger than an allowable maximum size, wherein the allowable maximum size is predetermined, signaled in a parameter set, or determined based on characteristics of the block of video data.

29. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to predict the sub-blocks comprise instructions that cause the processor to form prediction blocks for the sub-blocks from the respective reference blocks, further comprising instructions that cause the processor to decode the sub-blocks using the prediction blocks.

30. The computer-readable storage medium of claim 24 , wherein the instructions that cause the processor to predict the sub-blocks comprise instructions that cause the processor to form prediction blocks for the sub-blocks from the respective reference blocks, further comprising instructions that cause the processor to encode the sub-blocks using the prediction blocks.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2019
From: ZHANG, KAI; CHIEN, WEI-JUNG; ZHANG, LI; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 048927/0514 →
Continuity (2)
Provisional Application 62617005 · Jan 12, 2018
Related Publication 20190222865A1 · Jul 18, 2019
Cited By (1)
US 12,294,722