IP Library Granted Patent US 9,756,359
Granted Patent B2
US 9,756,359 · App. 14/569,126 · Granted Sep 5, 2017

Large blocks and depth modeling modes (DMM'S) in 3D video coding

Inventors: Xin Zhao (San Diego, CA); Ying Chen (San Diego, CA); Li Zhang (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N19/597H04N19/11H04N19/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,756,359
App. No.
14/569,126
Granted
Sep 5, 2017
Kind
B2
Abstract

In some examples, a method of decoding depth data in a video coding process includes defining a depth prediction unit (PU) of a size greater than 32×32 within a depth coding unit (CU) and generating one or more partitions of the depth PU. The method also includes obtaining residual data for each of the partitions; obtaining prediction data for each of the partitions; and reconstructing each of the partitions based on the residual data and the prediction data for the respective partitions.

Claims (120)

1. A method of decoding depth data in a video coding process, the method comprising:

defining a depth prediction unit (PU) of a size greater than 32×32 within a depth coding unit (CU);

generating one or more partitions of the depth PU, wherein generating the one or more partitions of the depth PU comprises:

upsampling an N×N partition pattern where N is 32 or less;

storing a pattern list comprising a plurality of different, partition patterns, wherein each respective partition pattern is identified by a respective one of a plurality of partition pattern indexes;

selecting, based on one of the partition pattern indexes, a partition pattern from the pattern list; and

applying the selected partition pattern to the depth PU to generate the one or more partitions;

obtaining residual data for each of the partitions;

obtaining prediction data for each of the partitions; and

reconstructing each of the partitions based on the residual data and the prediction data for the respective partitions.

2. The method of claim 1 , wherein the depth PU is of size 64×64.

3. The method of claim 1 , wherein the plurality of different, partition patterns includes at least one Wedgelet partition pattern, wherein storing the pattern list comprising the plurality of partition patterns further comprises:

determining the plurality of different, partition patterns based on at least two different start locations and at least two different end locations of the depth PU;

associating each respective partition pattern of the plurality of different, partition patterns with a respective index value; and

storing, in the pattern list, a respective association between each respective partition pattern and corresponding, respective index value.

4. The method of claim 3 , wherein determining the plurality of different, partition patterns based on the at least two different start locations and the at least two different end locations of the depth PU further comprises:

connecting at least one of the two different start locations and at least one of the two different end locations using a resolution of half-sample, full-sample, two samples, three samples or four samples.

5. The method of claim 1 , wherein generating one or more partitions of the depth PU further comprises:

receiving information identifying a sub-PU within the depth PU, wherein a size of the sub-PU is less than a size of the depth PU;

extending a linear line of the partition pattern for the sub-PU to boundaries of the depth PU, wherein a resulting linear line defines the partition pattern for the depth PU; and

determining the one or more partitions based on the partition pattern.

6. The method of claim 1 , wherein generating one or more partitions of the depth PU further comprises:

generating one or more partitions of the depth PU based only on at least one of a horizontal partition pattern or a vertical partition pattern.

7. The method of claim 1 , wherein the residual data for at least one of the partitions includes a single DC residual value generated for the at least one of the partitions using segment-wise DC coding (SDC).

8. The method of claim 1 , wherein generating one or more partitions of the depth PU further comprises:

determining an average value of samples in a texture block, the texture block included at a first region of a texture picture, and the first region being co-located with a second region of the depth PU included in a depth map; and

determining for each depth value of the depth map, based on comparing a respective sample of the texture block to the average value of samples, a partition of the one or more partitions to assign the depth value; and

assigning the depth value to the partition of the one or more partitions.

9. The method of claim 8 , wherein assigning the depth value to the partition of the one or more partitions further comprises:

determining a first location of the respective depth value within the depth map;

determining a second location of the respective sample in the texture block, wherein the first location in the depth PU is co-located with the second location in the texture block; and

responsive to determining that the respective sample is greater than the average value of samples, assigning the depth value of the first location to a first partition of the one or more partitions.

10. A device for decoding video data comprising:

a video data memory that stores a plurality of different, partition patterns; and

one or more processors configured to:

define a depth prediction unit (PU) of a size greater than 32×32 within a depth coding unit (CU);

generate one or more partitions of the depth PU, wherein to generate the one or more partitions of the depth PU the one or more processors are configured to:

upsample an N×N partition pattern where N is 32 or less;

store a pattern list comprising the different, partition patterns, wherein each respective partition pattern is identified by a respective one of a plurality of partition pattern indexes;

select, based on one of the partition pattern indexes, a partition pattern from the pattern list; and

apply the selected partition pattern to the depth PU to generate the one or more partitions;

determine residual data for each of the partitions;

determine prediction data for each of the partitions; and

reconstruct each of the partitions based on the residual data and the prediction data for the respective partitions.

11. The device for decoding video data of claim 10 ,

wherein the one or more processors are configured to:

determine the plurality of different, partition patterns based on at least two different start locations and at least two different end locations of the depth PU;

associate each respective partition pattern of the plurality of different, partition patterns with a respective index value; and

store, in the pattern list, a respective association between each respective partition pattern and corresponding, respective index value.

12. The device for decoding video data of claim 10 ,

wherein the one or more processors are configured to:

determine an average value of samples in a texture block, the texture block included at a first region of a texture picture, and the first region being co-located with a second region of the depth PU included in a depth map; and

determine for each depth value of the depth map, based on comparing a respective sample of the texture block to the average value of samples, a partition of the one or more partitions to assign the depth value; and

assign the depth value to the partition of the one or more partitions.

13. The device for coding decoding video data of claim 12 ,

wherein the one or more processors are configured to:

determine a first location of the respective depth value within the depth map;

determine a second location of the respective sample in the texture block, wherein the first location in the depth PU is co-located with the second location in the texture block; and

responsive to determining that the respective sample is greater than the average value of samples, assign the depth value of the first location to a first partition of the one or more partitions.

14. The device for decoding video data of claim 10 ,

wherein the one or more processors are configured to:

determine an average value of samples in a texture block, the texture block included at a first region of a texture picture, and the first region being co-located with a second region of the depth PU included in a depth map; and

determine for each depth value of the depth map, based on comparing a respective sample of the texture block to the average value of samples, a partition of the one or more partitions to assign the depth value; and

assign the depth value to the partition of the one or more partitions.

15. The device for decoding video data of claim 14 ,

wherein the one or more processors are configured to:

determine a first location of the respective depth value within the depth map;

determine a second location of the respective sample in the texture block, wherein the first location in the depth PU is co-located with the second location in the texture block; and

responsive to determining that the respective sample is greater than the average value of samples, assign the depth value of the first location to a first partition of the one or more partitions.

16. The device of claim 10 , wherein the device is selected from the group consisting of:

an integrated circuit (IC);

a micro-controller; and

a wireless device.

17. The device of claim 10 , wherein the device comprises a wireless communication device, the device further comprising a receiver configured to receive encoded video data comprising the plurality of different, partition patterns.

18. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a device for coding video data to:

define a depth prediction unit (PU) of a size greater than 32×32 within a depth coding unit (CU);

generate one or more partitions of the depth PU, wherein to generate the one or more partitions of the depth PU, the non-transitory computer-readable storage medium stores instructions that cause the one or more processors to:

upsample an N×N partition pattern where N is 32 or less;

store a pattern list comprising a plurality of different, partition patterns, wherein each respective partition pattern is identified by a respective one of a plurality of partition pattern indexes;

select, based on one of the partition pattern indexes, a partition pattern from the pattern list; and

apply the selected partition pattern to the depth PU to generate the one or more partitions;

determine residual data for each of the partitions;

determine prediction data for each of the partitions; and

reconstruct each of the partitions based on the residual data and the prediction data for the respective partitions.

19. The non-transitory computer-readable storage medium of claim 18 having stored thereon instructions that, when executed, cause the one or more processors of the device for coding video data to:

determine the plurality of different, partition patterns based on at least two different start locations and at least two different end locations of the depth PU;

associate each respective partition pattern of the plurality of different, partition patterns with a respective index value; and

store, in the pattern list, a respective association between each respective partition pattern and corresponding, respective index value.

20. The non-transitory computer-readable storage medium of claim 18 having stored thereon instructions that, when executed, cause the one or more processors of the device for coding video data to:

determine an average value of samples in a texture block, the texture block included at a first region of a texture picture, and the first region being co-located with a second region of the depth PU included in a depth map; and

determine for each depth value of the depth map, based on comparing a respective sample of the texture block to the average value of samples, a partition of the one or more partitions to assign the depth value; and

assign the depth value to the partition of the one or more partitions.

21. A device for decoding video data comprising:

a video data memory that stores one or more partition patterns; and

one or more processors configured to:

define a depth prediction unit (PU) of a size greater than 32×32 within a depth coding unit (CU);

generate one or more partitions of the depth PU, wherein to generate the one or more partitions of the depth PU, the one or more processors are configured to:

upsample an N×N partition pattern where N is 32 or less;

determine an average value of samples in a texture block, the texture block included at a first region of a texture picture, and the first region being co-located with a second region of the depth PU included in a depth map; and

determine for each depth value of the depth map, based on comparing a respective sample of the texture block to the average value of samples, a partition of the one or more partitions to assign the respective depth value; and

assign the respective depth value to the determined partition of the one or more partitions, wherein to assign the respective depth value to the determined partition of the one or more partitions further, the one or more processors are configured to:

determine a first location of the respective depth value within the depth map;

determine a second location of the respective sample in the texture block, wherein the first location in the depth PU is co-located with the second location in the texture block; and

responsive to determining that the respective sample is greater than the average value of samples, assign the respective depth value of the first location to a first partition of the one or more partitions;

obtain residual data for each of the partitions;

obtain prediction data for each of the partitions; and

reconstruct each of the partitions based on the residual data and the prediction data for the respective partitions.

22. The device of claim 21 , wherein the depth PU is of size 64×64.

23. The device of claim 21 , wherein the residual data for at least one of the partitions includes a single DC residual value generated for the at least one of the partitions using segment-wise DC coding (SDC).

24. The device of claim 21 , wherein the device is selected from the group consisting of:

an integrated circuit (IC);

a micro-controller; and

a wireless device.

25. The method of claim 1 , further comprising:

receiving encoded video data comprising the depth data at a receiver of a wireless communication device;

storing the encoded video data in a memory of the wireless communication device;

processing the encoded video data on one or more processors of the wireless communication device; and

outputting the processed encoded video data.

26. The method of claim 25 , wherein the wireless communication device comprises a telephone handset and wherein receiving the encoded video data at the receiver of the wireless communication device comprises demodulating, according to a wireless communication standard, a signal comprising the encoded video data.

27. The device of claim 17 , wherein the wireless communication device comprises a telephone handset and wherein the receiver is configured to demodulate, according to a wireless communication standard, a signal comprising the encoded video data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2015
From: ZHAO, XIN; CHEN, YING; ZHANG, LI
To: QUALCOMM INCORPORATED
Reel/Frame 034815/0045 →
Continuity (2)
Provisional Application 61916780 · Dec 16, 2013
Related Publication 20150172717A1 · Jun 18, 2015