IP Library Granted Patent US 9,912,944
Granted Patent B2
US 9,912,944 · App. 13/863,266 · Granted Mar 6, 2018

Simplified non-square quadtree transforms for video coding

Inventors: Liwei Guo (San Diego, CA); Xianglin Wang (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N19/00781H04N19/119H04N19/122H04N19/159H04N19/619H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,912,944
App. No.
13/863,266
Granted
Mar 6, 2018
Kind
B2
Abstract

In an example, a method of decoding video data includes determining a prediction partitioning structure for predicting pixel values associated with a block of video data. The method also includes determining a transform partitioning structure for applying one or more transforms to the predicted pixel values. Determining the transform split structure includes splitting a parent transform unit, upon determining the transform partitioning structure comprises splitting the parent transform unit into one or more square transforms, determining one or more square transforms such that each of the one or more square transforms correspond to exactly one prediction partition, and upon determining the transform partitioning structure comprises splitting the parent transform unit into one or more non-square transforms, determining whether to split the one or more non-square transforms based at least in part on the one or more non-square transforms being non-square.

Claims (98)

1. A method of decoding encoded video data, the method comprising:

determining that residual quadtree splitting is enabled with respect to a block of the encoded video data;

determining a maximum residual quad tree depth with respect to the residual quadtree splitting of the block of the encoded video data;

determining that a prediction partitioning structure for reconstructing pixel values associated with the block of video data includes one or more asymmetric motion partition (AMP) boundaries;

determining a transform partitioning structure for applying one or more inverse transforms to the reconstructed pixel values, wherein determining the transform partitioning structure includes:

performing, in response to the residual quadtree splitting being enabled with respect to the block of the encoded video data and based on the residual quadtree splitting resulting in one or more transforms crossing at least one AMP boundary of the AMP boundaries, forced splitting of a parent transform unit associated with the block of the encoded video data, wherein the parent transform unit has an equal area with respect to the block of video data, wherein the forced splitting of the parent transform unit produces a plurality of square transforms, and wherein each square transform of the plurality of square transforms is at the maximum residual quad tree depth associated with the residual quadtree splitting; and

upon performing the forced splitting of the parent transform unit to form the plurality of square transforms, determining each respective square transform of the plurality of square transforms based on the one or more AMP boundaries such that each respective square transform of the plurality of square transforms corresponds to a maximum transform size that does not cross any AMP boundary of the one or more AMP boundaries.

2. The method of claim 1 , wherein performing the forced splitting of the parent transform unit to form the plurality of square transforms comprises splitting at least one transform that is applied across a respective AMP boundary of the one or more AMP boundaries and without splitting any transforms that are not applied across the respective AMP boundary.

3. The method of claim 1 , further comprising adaptively enabling the determination of the plurality of square transforms based on a coding characteristic associated with the block of the encoded video data, the coding characteristic comprising one of a block size, a picture type, a picture size, a prediction mode, a motion vector amplitude, or a reference index associated with the block of the encoded video data.

4. The method of claim 1 , further comprising applying differently sized transforms to the residual values.

5. The method of claim 4 , wherein the differently sized transforms include at least one transform having a size smaller than a smallest transform size indicated by a residual quad-tree structure associated with the forced splitting.

6. The method of claim 1 , wherein the prediction partitioning structure comprises prediction units sized one of nL×2N, nR×2N, 2N×nU, or 2N×nD.

7. The method of claim 1 , further comprising determining that each respective square transform of the plurality of square transforms can only be further split into further non-square transforms.

8. The method of claim 7 , wherein the parent transform unit comprises a parent luminance transform unit and wherein the further non-square transforms comprise one or more non-square luminance transforms, the method further comprising:

splitting a parent chrominance transform unit into a plurality of non-square chrominance transforms, wherein the parent chrominance transform unit corresponds to the parent luminance transform unit; and

splitting the one or more non-square chrominance transforms.

9. The method of claim 1 , further comprising applying one or more transform splitting restrictions based at least in part on one or more of a size of a coding unit corresponding to the parent transform unit, a picture type of a picture including the parent transform unit, a pixel resolution of the picture including the parent transform unit, and a block type for the coding unit corresponding to the parent transform unit.

10. The method of claim 1 , the method being executable on a wireless communication device, wherein the wireless communication device comprises:

data storage medium configured to store the block of video data;

a processor configured to execute instructions to process the block of video data stored in the data storage medium; and

a receiver to receive the block of video data.

11. The method of claim 10 , wherein the wireless communication device is a cellular telephone and the block of video data is received by the receiver and modulated according to a communication standard.

12. A method of encoding video data, the method comprising:

determining that a prediction partitioning structure for residual pixel values associated with a block of video data includes one or more asymmetric motion partition (AMP) boundaries;

determining a transform partitioning structure for applying one or more transforms to the residual pixel values, wherein determining the transform partitioning structure includes:

determining that a residual quadtree splitting of the block results in one or more transforms crossing at least one AMP boundary of the AMP boundaries;

performing, based on the transform splitting of the block resulting in the one or more transforms crossing the at least one AMP boundary of the AMP boundaries, forced splitting of a parent transform unit associated with the block of video data, wherein the parent transform unit has an equal area with respect to the block of video data, wherein the forced splitting of the parent transform unit produces a plurality of square transforms, and wherein each square transform of the plurality of square transforms is at a maximum residual quad tree depth associated with the residual quadtree splitting;

upon performing the forced splitting of the parent transform unit to form the plurality of square transforms, determining each respective square transform of the plurality of square transforms based on the one or more AMP boundaries such that each respective square transform of the plurality of square transforms corresponds to a maximum transform size that does not cross any AMP boundary of the one or more AMP boundaries;

encoding an indication that the transform splitting is enabled with respect to the block of video data;

encoding an indication of the maximum residual quad tree depth with respect to the transform splitting of the block of video data; and

encoding the residual pixel values based on the determined transform partitioning structure.

13. The method of claim 12 , wherein performing the forced splitting to form the plurality of square transforms comprises splitting at least one transform that is applied across a respective AMP boundary of the one or more AMP boundaries and without splitting any transforms that are not applied across the respective AMP boundary of the one or more AMP boundaries.

14. The method of claim 12 , further comprising adaptively enabling the determination of the plurality of square transforms based on a coding characteristic associated with the block of video data, the coding characteristic comprising one of a block size, a picture type, a picture size, a prediction mode, a motion vector amplitude, or a reference index associated with the block of video data.

15. The method of claim 12 , further comprising applying differently sized transforms to the residual values.

16. The method of claim 15 , wherein the differently sized transforms include at least one transform having a size smaller than a smallest transform size indicated by a residual quad-tree structure associated with the forced splitting.

17. The method of claim 12 , wherein the prediction partitioning structure comprises prediction units sized one of nL×2N, nR×2N, 2N×nU, or 2N×nD.

18. The method of claim 12 , wherein further comprising determining that each respective square transform of the plurality of square transforms can only be further split into further non-square transforms.

19. The method of claim 18 , wherein the parent transform unit comprises a parent luminance transform unit and wherein the further non-square transforms comprise one or more non-square luminance transforms, the method further comprising:

splitting a parent chrominance transform unit into a plurality of non-square chrominance transforms, wherein the parent chrominance transform unit corresponds to the parent luminance transform unit; and

splitting the one or more non-square chrominance transforms.

20. The method of claim 11 , further comprising applying one or more transform splitting restrictions based at least in part on one or more of a size of a coding unit corresponding to the parent transform unit, a picture type of a picture including the parent transform unit, a pixel resolution of the picture including the parent transform unit, and a block type for the coding unit corresponding to the parent transform unit.

21. An apparatus for decoding encoded video data comprising:

a memory configured to store a the encoded video data; and

one or more processors coupled to the memory, the one or more processors being configured to:

determine that residual quadtree splitting is enabled with respect to a block of the encoded video data stored to the memory;

determine a maximum residual quad tree depth with respect to the residual quadtree splitting of the block of the encoded video data stored to the memory;

determine that a prediction partitioning structure for reconstructing pixel values associated with the block of the encoded video data stored to the memory includes one or more asymmetric motion partition (AMP) boundaries;

determine a transform partitioning structure for applying one or more inverse transforms to the reconstructed pixel values, wherein to determine the transform partition structure, the one or more processors are configured to:

perform, in response to the residual quadtree splitting being enabled with respect to the block of video data and based on the transform splitting resulting in one or more transforms crossing at least one AMP boundary of the AMP boundaries, forced splitting of a parent transform unit associated with the block of the encoded video data, wherein the parent transform unit has an equal area with respect to the block of video data, wherein the forced splitting of the parent transform unit produces a plurality of square transforms, and wherein each square transform of the plurality of square transforms is at the maximum residual quad tree depth associated with the residual quadtree splitting;

upon performing the forced splitting of the parent transform unit to form the plurality of square transforms, determine each respective square transform of the plurality of square transforms based on one or more AMP boundaries of the one or more AMP boundaries such that each square transform of the one or more square transforms corresponds to a maximum transform size that does not cross any AMP boundary of the one or more AMP boundaries based on the splitting of the parent transform unit into the square transforms; and

upon determining to split the parent transform unit into a plurality of non-square transforms, determine that the a plurality of non-square transforms are not to be further split based at least in part on each of the a plurality of non-square transforms being non-square.

22. The apparatus of claim 21 , wherein to perform the forced splitting of the parent transform unit to form the plurality of square transforms, the one or more processors are configured to split at least one transform that is applied across a respective AMP boundary of the one or more AMP boundaries and without splitting any transforms that are not applied across the respective AMP boundary.

23. The apparatus of claim 21 , wherein the one or more processors are further configured to adaptively enable the determination of the plurality of square transforms based on a coding characteristic associated with the block of the encoded video data, the coding characteristic comprising one of a block size, a picture type, a picture size, a prediction mode, a motion vector amplitude, or a reference index associated with the block of the encoded video data.

24. The apparatus of claim 21 , wherein the one or more processors are further configured to apply differently sized transforms to the residual values.

25. The apparatus of claim 24 , wherein the differently sized transforms include at least one transform having a size smaller than a smallest transform size indicated by a residual quad-tree structure associated with the forced splitting.

26. The apparatus of claim 21 , wherein the prediction partitioning structure comprises prediction units sized one of nL×2N, nR×2N, 2N×nU, or 2N×nD.

27. The apparatus of claim 21 , wherein the one or more processors are further configured to determine that each respective square transform of the plurality of transforms can only be further split into further non-square transforms.

28. The apparatus of claim 27 , wherein the parent transform unit comprises a parent luminance transform unit, wherein the further non-square transforms comprise one or more non-square luminance transforms, and wherein the one or more processors are further configured to:

split a parent chrominance transform unit into a plurality of chrominance non-square transforms, wherein the parent chrominance transform unit corresponds to the parent luminance transform unit; and

split the one or more non-square chrominance transforms.

29. The apparatus of claim 21 , wherein the one or more processors are further configured to apply one or more transform splitting instructions based at least in part on one or more of a size of a coding unit corresponding to the parent transform unit, a picture type of a picture including the parent transform unit, a pixel resolution of the picture including the parent transform unit, and a block type for the coding unit corresponding to the parent transform unit.

30. The apparatus of claim 21 , wherein to decode the predicted pixel values, the one or more processors are configured to obtain an indication of the predicted pixel values from an encoded bitstream.

31. The apparatus of claim 21 , wherein the apparatus is a wireless communication device, further comprises a receiver configured to receive the block of video data.

32. The apparatus of claim 31 , wherein the wireless communication device is a cellular telephone and the block of video data is received by the receiver and modulated according to a communication standard.

33. An apparatus for decode encoded video data, the apparatus comprising:

means for determining that residual quadtree splitting is enabled with respect to a block of the encoded video data;

means for determining a maximum residual quad tree depth with respect to the residual quadtree splitting of the block of the encoded video data;

means for determining that a prediction partitioning structure for reconstructing pixel values associated with the block of video data includes one or more asymmetric motion partition (AMP) boundaries;

means for determining a transform partitioning structure for applying one or more inverse transforms to the reconstructed pixel values, wherein the means for determining the transform partitioning structure comprises:

means for performing, in response to the residual quadtree splitting being enabled with respect to the block of the encoded video data and based on the residual quadtree splitting resulting in one or more transforms crossing at least one AMP boundary of the AMP boundaries, forced splitting of a parent transform unit associated with the block of the encoded video data, wherein the parent transform unit has an equal area with respect to the block of video data, wherein the forced splitting of the parent transform unit produces a plurality of square transforms, and wherein each square transform of the plurality of square transforms is at the maximum residual quad tree depth associated with the residual quadtree splitting; and

means for determining, upon performing the forced splitting of the parent transform unit to form the plurality of square transforms, each respective square transform of the plurality of square transforms based on the one or more AMP boundaries such that each respective square transform of the plurality of square transforms corresponds to a maximum transform size that does not cross any AMP boundary of the one or more AMP boundaries.

34. The apparatus of claim 33 , wherein the means for performing the forced splitting of the parent transform unit to form the plurality of square transforms comprises means for splitting at least one transform that is applied across a respective AMP boundary of the one or more AMP boundaries and without splitting any transforms that are not applied across the respective AMP boundary.

35. The apparatus of claim 33 , further comprising means for adaptively enabling the determination of the plurality of square transforms based on a coding characteristic associated with the block of the encoded video data, the coding characteristic comprising one of a block size, a picture type, a picture size, a prediction mode, a motion vector amplitude, or a reference index associated with the block of the encoded video data.

36. The apparatus of claim 33 , further comprising means for applying differently sized transforms to the residual values.

37. The apparatus of claim 36 , wherein the differently sized transforms include at least one transform having a size smaller than a smallest transform size indicated by a residual quad-tree structure associated with the forced splitting.

38. The apparatus of claim 33 , wherein the prediction partitioning structure comprises prediction units sized one of nL×2N, nR×2N, 2N×nU, or 2N×nD.

39. The apparatus of claim 33 , further comprising means for determining that each respective square transform of the plurality of square transforms can only be further split into further non-square transforms.

40. The apparatus of claim 39 , wherein the parent transform unit comprises a parent luminance transform unit and wherein the further non-square transforms comprise one or more non-square luminance transforms, the apparatus further comprising:

means for splitting a parent chrominance transform unit into a plurality of non-square chrominance transforms, wherein the parent chrominance transform unit corresponds to the parent luminance transform unit; and

means for splitting the one or more non-square chrominance transforms.

41. The apparatus of claim 33 , further comprising means for applying one or more transform splitting restrictions based at least in part on one or more of a size of a coding unit corresponding to the parent transform unit, a picture type of a picture including the parent transform unit, a pixel resolution of the picture including the parent transform unit, and a block type for the coding unit corresponding to the parent transform unit.

42. A non-transitory computer-readable medium storing instructions thereon that, when executed, cause one or more processors to:

determine that residual quadtree splitting is enabled with respect to a block of encoded video data;

determine a maximum residual quad tree depth with respect to the residual quadtree splitting of the block of the encoded video data;

determine that a prediction partitioning structure for reconstructing pixel values associated with the block of video data includes one or more asymmetric motion partition (AMP) boundaries;

determine a transform partitioning structure for applying one or more inverse transforms to the reconstructed pixel values, wherein to determine the transform partitioning structure, the instructions cause the one or more processors to:

perform, in response to the residual quadtree splitting being enabled with respect to the block of the encoded video data and based on the residual quadtree splitting resulting in one or more transforms crossing at least one AMP boundary of the AMP boundaries of the AMP boundaries, forced splitting of a parent transform unit associated with the block of the encoded video data, wherein the parent transform unit has an equal area with respect to the block of video data, wherein the forced splitting of the parent transform unit produces a plurality of square transforms, and wherein each square transform of the plurality of square transforms is at the maximum residual quad tree depth associated with the residual quadtree splitting; and

upon performing the forced splitting of the parent transform unit to form the plurality of square transforms, determine each respective square transform of the plurality of square transforms based on the one or more AMP boundaries such that each respective square transform of the plurality of square transforms corresponds to a maximum transform size that does not cross any AMP boundary of the one or more AMP boundaries.

43. The non-transitory computer-readable medium of claim 42 , further comprising instructions that cause the one or more processors to split at least one transform that is applied across a respective AMP boundary of the one or more AMP boundaries and without splitting any transforms that are not applied across the respective AMP boundary.

44. The non-transitory computer-readable medium of claim 42 , further comprising instructions that cause the one or more processors to adaptively enable the determination of the plurality of square transforms based on a coding characteristic associated with the block of the encoded video data, the coding characteristic comprising one of a block size, a picture type, a picture size, a prediction mode, a motion vector amplitude, or a reference index associated with the block of the encoded video data.

45. The non-transitory computer-readable medium of claim 42 , further comprising instructions that cause the one or more processors to apply differently sized transforms to the residual values.

46. The non-transitory computer-readable medium of claim 45 , wherein the differently sized transforms include at least one transform having a size smaller than a smallest transform size indicated by a residual quad-tree structure associated with the forced splitting.

47. The non-transitory computer-readable medium of claim 42 , wherein the prediction partitioning structure comprises prediction units sized one of nL×2N, nR×2N, 2N×nU, or 2N×nD.

48. The non-transitory computer-readable medium of claim 42 , further comprising instructions that cause the one or more processors to determine that each respective square transform of the plurality of square transforms can only be further split into further non-square transforms.

49. The non-transitory computer-readable medium of claim 48 , wherein the parent transform unit comprises a parent luminance transform unit and wherein the further non-square transforms comprise one or more non-square luminance transforms, the one or more processors further configured to:

split a parent chrominance transform unit into a plurality of non-square chrominance transforms, wherein the parent chrominance transform unit corresponds to the parent luminance transform unit; and

split the one or more non-square chrominance transforms.

50. The non-transitory computer-readable medium of claim 42 , further comprising instructions that cause the one or more processors to apply one or more transform splitting instructions based at least in part on one or more of a size of a coding unit corresponding to the parent transform unit, a picture type of a picture including the parent transform unit, a pixel resolution of the picture including the parent transform unit, and a block type for the coding unit corresponding to the parent transform unit.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2013
From: GUO, LIWEI; WANG, XIANGLIN; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 031047/0818 →
Continuity (3)
Provisional Application 61625038 · Apr 16, 2012
Provisional Application 61637220 · Apr 23, 2012
Related Publication 20130272381A1 · Oct 17, 2013