Constraints and unit types to simplify video random access
Disclosed herein are innovations for bitstreams having clean random access (CRA) pictures and/or other types of random access point (RAP) pictures. New type definitions and strategic constraints on types of RAP pictures can simplify mapping of units of elementary video stream data to a container format. Such innovations can help improve the ability for video coding systems to more flexibly perform adaptive video delivery, production editing, commercial insertion, and the like.
1. A video decoder system comprising:
a buffer, implemented in memory, configured to receive encoded data in a bitstream for multiple pictures in a video sequence; and
a video decoder, implemented with one or more processing units and memory, configured to perform operations to decode the multiple pictures, wherein each of the multiple pictures is designated as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, and wherein the operations include:
decoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
decoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
2. The video decoder system of claim 1 , wherein the first picture is not an initial picture of the bitstream.
3. The video decoder system of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
4. The video decoder system of claim 3 , wherein the operations further include:
decoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
selectively decoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
5. The video decoder system of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the third picture type, decoding the fourth picture.
6. The video decoder system of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the fifth picture type, skipping the decoding of the fourth picture.
7. The video decoder system of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures, and (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
8. The video decoder system of claim 1 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
9. In a computer system, a method comprising:
receiving encoded data in a bitstream for multiple pictures in a video sequence, each of the multiple pictures being designated in the encoded data as one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture; and
storing, in memory or storage, the encoded data, the encoded data being organized to facilitate decoding of the multiple pictures by operations that include:
decoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
decoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
10. The method of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
11. The method of claim 10 , wherein the operations further include:
decoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
selectively decoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
12. The method of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures, and (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
13. The method of claim 9 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
14. One or more non-transitory computer-readable media having stored thereon computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations comprising:
receiving multiple pictures in a video sequence to be encoded; and
encoding the multiple pictures, thereby generating encoded data in a bitstream for the multiple pictures, wherein each of the multiple pictures is designated in the encoded data as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, and wherein the encoding the multiple pictures includes:
encoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
encoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
15. The one or more computer-readable media of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
16. The one or more computer-readable media of claim 15 , wherein the operations further include:
encoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
encoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
17. The video encoder system of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture.
18. The one or more computer-readable media of claim 17 , wherein the multiple picture types further include (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures.
19. The one or more computer-readable media of claim 18 , wherein the multiple picture types further include (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
20. The one or more computer-readable media of claim 14 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.