IP Library › Granted Patent US 9,819,941

Granted Patent B2

US 9,819,941 · App. 14/509,797 · Granted Nov 14, 2017

Signaling for sub-decoded picture buffer (sub-DPB) based DPB operations in video coding

Inventors: Adarsh Krishnan Ramasubramonian (San Diego, CA); Ye-Kui Wang (San Diego, CA)

Assignee: QUALCOMM Incorporated

H04N19/156 H04N19/177 H04N19/196 H04N19/31 H04N19/423 H04N19/44 H04N19/463 H04N19/597 H04N19/70

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 9,819,941
App. No.: 14/509,797
Granted: Nov 14, 2017
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A method of decoding video data including receiving an encoded video bitstream that includes a plurality of pictures and storing the plurality of pictures in one or more sub-DPBs. The method further including receiving a respective set of sub-DPB parameters for each respective operation point of the encoded video bitstream. applying the respective set of sub-DPB parameters to all layers of an output layer set for each respective operation point, and performing a sub-DPB management process on the one or more sub-DPBs in accordance with the received respective single sets of sub-DPB parameters.

Claims (136)

1. A method of decoding video data, the method comprising:

receiving, from an encoded video bitstream, a plurality of output layer sets, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer;

decoding each respective plurality of sub-layers of the plurality of output layer sets;

storing each respective decoded plurality of sub-layers in one or more sub-decoded picture buffers (DPBs);

receiving, from the encoded video bitstream, a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

receiving, from the encoded video bitstream, a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

receiving, from the encoded video bitstream, a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers; and

performing a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received.

2. The method of claim 1 , wherein each respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers includes at least one of:

a respective third syntax element indicative of a maximum number of reorder pictures allowed for the one or more sub-DPBs; or

a respective fourth syntax element indicative of a maximum latency allowed for the one or more sub-DPBs.

3. The method of claim 1 , wherein receiving the respective set of sub-DPB parameters comprises receiving the respective set of sub-DPB parameters in a video parameter set (VPS).

4. The method of claim 1 , further comprising:

applying, based on the first respective syntax element indicating that sub-DPB parameters are not signaled for each sub-layer of a second particular output layer set of the plurality of output layer sets, the respective set of sub-DPB parameters of the second particular output layer set to all sub-layers of the second particular output layer set.

5. The method of claim 1 , further comprising:

receiving each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set when the first respective syntax element indicating that sub-DPB parameters are signaled for each sub-layer of the first particular output layer set.

6. The method of claim 1 , further comprising:

applying, based on the second respective syntax element indicating that sub-DPB parameters are not signaled for a particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a last received respective set of sub-DPB parameters for the particular sub-layer; or

receiving, when the second respective syntax element is indicative that sub-DPB parameters are signaled for the particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a respective set of sub-DPB parameters for the particular sub-layer.

7. The method of claim 1 , further comprising:

applying, based on the second respective syntax element indicating that sub-DPB parameters are not signaled for a particular sub-layer with a sub-layer ID X, a last received respective set of sub-DPB parameters for sub-layers of the one or more sub-layers other than the first sub-layer of the first particular output layer set that have a respective sub-layer ID greater than X, wherein the particular sub-layer with the sub-layer ID X is one of the one or more sub-layers other than the first sub-layer of the first particular output layer set.

8. The method of claim 1 , wherein the sub-DPB management process includes at least one of marking pictures as unused for reference, marking pictures for output, or removing pictures from the sub-DPB, and wherein the performing the sub-DPB management process comprises performing the sub-DPB management process on an access unit level using the received respective sets of sub-DPB parameters.

9. The method of claim 8 , wherein performing the sub-DPB management process on the access unit level comprises:

outputting pictures based on the number of access units in a first sub-DPB of the one or more sub-DPBs that contain at least one picture marked as needed for output;

marking pictures as unused for reference on a per-layer level; and

removing pictures from the first sub-DPB on a per-layer level.

10. An apparatus configured to decode video data, the apparatus comprising:

one or more memory devices comprising one or more sub-decoded picture buffers (DPBs) configured to store video data; and

a video decoder configured to:

receive, from an encoded video bitstream, a plurality of output layer sets, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer;

decode each respective plurality of sub-layers of the plurality of output layer sets;

store each respective decoded plurality of sub-layers in the one or more sub-DPBs;

receive, from the encoded video bitstream, a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

receive, from the encoded video bitstream, a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

receive, from the encoded video bitstream, a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers; and

perform a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received.

11. The apparatus of claim 10 , wherein each respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers includes at least one of:

a respective third syntax element indicative of a maximum number of reorder pictures allowed for the one or more sub-DPBs; or

a respective fourth syntax element indicative of a maximum latency allowed for the one or more sub-DPBs.

12. The apparatus of claim 10 , wherein the video decoder is configured to receive the respective set of sub-DPB parameters in a video parameter set (VPS).

13. The apparatus of claim 10 , wherein the video decoder is further configured to:

apply, based on the first respective syntax element indicating that sub-DPB parameters are not signaled for each sub-layer of a second particular output layer set of the plurality of output layer sets, the respective set of sub-DPB parameters of the second particular output layer set to all sub-layers of the second particular output layer set.

14. The apparatus of claim 10 , wherein the video decoder is further configured to:

receive each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set when the first respective syntax element indicating that sub-DPB parameters are signaled for each sub-layer of the first particular output layer set.

15. The apparatus of claim 10 , wherein the video decoder is further configured to:

apply, based on the second respective syntax element indicating that sub-DPB parameters are not signaled for a particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a last received respective set of sub-DPB parameters for the particular sub-layer; or

receive, when the second respective syntax element is indicative that sub-DPB parameters are signaled for the particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a respective set of sub-DPB parameters for the particular sub-layer.

16. The apparatus of claim 10 , wherein the video decoder is further configured to:

apply, based on the second respective syntax element indicating that sub-DPB parameters are not signaled for a particular sub-layer with a sub-layer ID X, a last received respective set of sub-DPB parameters for sub-layers of the one or more sub-layers other than the first sub-layer of the first particular output layer set that have a respective sub-layer ID greater than X, wherein the particular sub-layer with the sub-layer ID X is one of the one or more sub-layers other than the first sub-layer of the first particular output layer set.

17. The apparatus of claim 10 , wherein the sub-DPB management process includes at least one of marking pictures as unused for reference, marking pictures for output, or removing pictures from the sub-DPB, and wherein the performing the sub-DPB management process comprises performing the sub-DPB management process on an access unit level using the received respective sets of sub-DPB parameters.

18. The apparatus of claim 17 , wherein the video decoder is further configured to:

output pictures based on the number of access units in a first sub-DPB of the one or more sub-DPBs that contain at least one picture marked as needed for output;

mark pictures as unused for reference on a per-layer level; and

remove pictures from the first sub-DPB on a per-layer level.

19. An apparatus configured to decode video data, the apparatus comprising:

means for receiving, from an encoded video bitstream, a plurality of output layer sets, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer;

means for decoding each respective plurality of sub-layers of the plurality of output layer sets;

means for storing each respective decoded plurality of sub-layers in one or more sub-decoded picture buffers (DPBs);

means for receiving, from the encoded video bitstream, a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

means for receiving, from the encoded video bitstream, a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

means for receiving, from the encoded video bitstream, a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers; and

means for performing a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received.

20. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors of a device configured to decode video data to:

decode each respective plurality of sub-layers of the plurality of output layer sets;

store each respective decoded plurality of sub-layers in one or more sub- DPBs;

receive, from the encoded video bitstream, a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers; and

perform a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received.

21. A method of encoding video data, the method comprising:

reconstructing a plurality of output layer sets of an encoded video bitstream, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer, wherein reconstructing the plurality of output layer sets includes reconstructing each respective plurality of sub-layers of the plurality of output layer sets;

storing each respective reconstructed plurality of sub-layers in one or more sub-decoded picture buffers (DPBs);

generating a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

generating a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

generating a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers;

performing a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received; and

generating each respective set of sub-DPB parameters in the encoded video bitstream.

22. The method of claim 21 , wherein each respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers includes at least one of:

a respective third syntax element indicative of a maximum number of reorder pictures allowed for the one or more sub-DPBs; or

a respective fourth syntax element indicative of a maximum latency allowed for the one or more sub-DPBs.

23. The method of claim 21 , wherein generating the respective set of sub-DPB parameters comprises generating the respective set of sub-DPB parameters in a video parameter set (VPS).

24. The method of claim 21 , further comprising:

generating, in the encoded video bitstream, each first respective syntax element for each output layer set of the plurality of output layer sets; and

generating, in the encoded video bitstream, each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets.

25. The method of claim 21 , further comprising:

generating each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set based on the first respective syntax element indicating that sub-DPB parameters are signaled for each sub-layer of the first particular output layer set.

26. The method of claim 21 , further comprising:

generating, based on the second respective syntax element indicating that sub-DPB parameters are signaled for a particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a respective set of sub-DPB parameters for the particular sub-layer.

27. The method of claim 21 , wherein the sub-DPB management process includes at least one of marking pictures as unused for reference, marking pictures for output, or removing pictures from the sub-DPB, and wherein the performing the sub-DPB management process comprises performing the sub-DPB management process on an access unit level using the received respective sets of sub-DPB parameters.

28. The method of claim 27 , wherein performing the sub-DPB management process on the access unit level comprises:

outputting pictures based on the number of access units in a first sub-DPB of the one or more sub-DPBs that contain at least one picture marked as needed for output;

marking pictures as unused for reference on a per-layer level; and

removing pictures from the first sub-DPB on a per-layer level.

29. An apparatus configured to encode video data, the apparatus comprising:

one or more memory devices comprising one or more sub-decoded picture buffers (DPBs) configured to store video data; and

a video encoder configured to:

reconstruct a plurality of output layer sets of an encoded video bitstream, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer, wherein reconstructing the plurality of output layer sets includes reconstructing each respective plurality of sub-layers of the plurality of output layer sets;

store each respective reconstructed plurality of sub-layers in the one or more sub-DPBs;

generate a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

generate a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

generate a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers;

perform a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received; and

generate each respective set of sub-DPB parameters in the encoded video bitstream.

30. The apparatus of claim 29 , wherein each respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers includes at least one of:

a respective third syntax element indicative of a maximum number of reorder pictures allowed for the one or more sub-DPBs; or

a respective fourth syntax element indicative of a maximum latency allowed for the one or more sub-DPBs.

31. The apparatus of claim 29 , wherein the video encoder is configured to generate the respective set of sub-DPB parameters in a video parameter set (VPS).

32. The apparatus of claim 29 , wherein the video encoder is further configured to:

generate, in the encoded video bitstream, each first respective syntax element for each output layer set of the plurality of output layer sets; and

generate, in the encoded video bitstream, each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets.

33. The apparatus of claim 29 , wherein the video encoder is further configured to:

generate each second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set based on the first respective syntax element indicating that sub-DPB parameters are signaled for each sub-layer of the first particular output layer set.

34. The apparatus of claim 29 , wherein the video encoder is further configured to:

generate, based on the second respective syntax element indicating that sub-DPB parameters are signaled for a particular sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set, a respective set of sub-DPB parameters for the particular sub-layer.

35. The apparatus of claim 29 , wherein the sub-DPB management process includes at least one of marking pictures as unused for reference, marking pictures for output, or removing pictures from the sub-DPB, and wherein the performing the sub-DPB management process comprises performing the sub-DPB management process on an access unit level using the received respective sets of sub-DPB parameters.

36. The apparatus of claim 35 , wherein the video encoder is further configured to:

output pictures based on the number of access units in a first sub-DPB of the one or more sub-DPBs that contain at least one picture marked as needed for output;

mark pictures as unused for reference on a per-layer level; and

remove pictures from the first sub-DPB on a per-layer level.

37. An apparatus configured to encode video data, the apparatus comprising:

means for reconstructing a plurality of output layer sets of an encoded video bitstream, wherein each output layer set of the plurality of output layer sets includes a respective plurality of sub-layers, wherein each respective plurality of sub-layers includes a first sub-layer and one or more sub-layers other than the first sub-layer, wherein the first sub-layer of each respective plurality of sub-layers is a base sub-layer, wherein reconstructing the plurality of output layer sets includes reconstructing each respective plurality of sub-layers of the plurality of output layer sets;

means for storing each respective reconstructed plurality of sub-layers in one or more sub-decoded picture buffers (DPBs);

means for generating a first respective syntax element for each output layer set of the plurality of output layer sets, wherein each first respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each sub-layer of each respective output layer set;

means for generating a second respective syntax element for each sub-layer of the one or more sub-layers other than the first sub-layer of a first particular output layer set of the plurality of output layer sets but not for the first sub-layer of the first particular output layer set, wherein each second respective syntax element is indicative of whether or not sub-DPB parameters are signaled for each respective sub-layer of the one or more sub-layers other than the first sub-layer of the first particular output layer set;

means for generating a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers;

means for performing a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received; and

means for generating each respective set of sub-DPB parameters in the encoded video bitstream.

38. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors of a device configured to encode video data to:

store each respective reconstructed plurality of sub-layers in one or more sub-decoded picture buffers (DPBs);

generate a respective set of sub-DPB parameters for the first sub-layer of each respective plurality of sub-layers;

perform a sub-DPB management process on the one or more sub-DPBs in accordance with each respective set of sub-DPB parameters received; and

generate each respective set of sub-DPB parameters in the encoded video bitstream.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 11, 2014

From: RAMASUBRAMONIAN, ADARSH KRISHNAN; WANG, YE-KUI

To: QUALCOMM INCORPORATED

Reel/Frame 034145/0862 →

Continuity (3)

Provisional Application 61889515 · Oct 10, 2013

Provisional Application 61890841 · Oct 14, 2013

Related Publication 20150103884A1 · Apr 16, 2015