Constraints and unit types to simplify video random access
Disclosed herein are innovations for bitstreams having clean random access (CRA) pictures and/or other types of random access point (RAP) pictures. New type definitions and strategic constraints on types of RAP pictures can simplify mapping of units of elementary video stream data to a container format. Such innovations can help improve the ability for video coding systems to more flexibly perform adaptive video delivery, production editing, commercial insertion, and the like.
1. In a computer system that implements a video decoder, a method comprising:
receiving encoded data in a bitstream for multiple pictures in a video sequence; and
decoding the multiple pictures, wherein each of the multiple pictures is designated as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, and wherein the decoding the multiple pictures includes:
decoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
decoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
2. The method of claim 1 , wherein the first picture is not an initial picture of the bitstream.
3. The method of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
4. The method of claim 3 , wherein the decoding the multiple pictures further includes:
decoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
selectively decoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
5. The method of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the third picture type, decoding the fourth picture.
6. The method of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the fifth picture type, skipping the decoding of the fourth picture.
7. The method of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures, and (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
8. The method of claim 1 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
9. One or more computer-readable memory or storage devices having stored thereon encoded data in a bitstream for multiple pictures in a video sequence, each of the multiple pictures being designated in the encoded data as one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, the encoded data being organized to facilitate decoding of the multiple pictures, in a computer system that implements a video decoder, by operations that include:
decoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
decoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
10. The one or more computer-readable memory or storage devices of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
11. The one or more computer-readable memory or storage devices of claim 10 , wherein the decoding the multiple pictures further includes:
decoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
selectively decoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
12. The one or more computer-readable memory or storage devices of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fourth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures; and (f) a fourth picture type indicating an IDR picture that does not have any associated leading pictures.
13. The one or more computer-readable memory or storage devices of claim 9 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
14. A video encoder system comprising:
a buffer, implemented in memory, configured to store multiple pictures in a video sequence to be encoded; and
a video encoder, implemented with one or more processing units and memory, configured to perform operations to encode the multiple pictures, thereby generating encoded data in a bitstream for the multiple pictures, wherein each of the multiple pictures is designated in the encoded data as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, and wherein the operations include:
encoding a first picture among the multiple pictures, the first picture being designated as the first picture type; and
encoding a second picture among the multiple pictures, the second picture being designated as the third picture type.
15. The video encoder system of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
16. The video encoder system of claim 15 , wherein the operations further include:
encoding a third picture among the multiple pictures, the third picture being designated as the fourth picture type; and
encoding a fourth picture among the multiple pictures, the fourth picture being designated as the third picture type or the fifth picture type.
17. The video encoder system of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture.
18. The video encoder system of claim 17 , wherein the multiple picture types further include (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures.
19. The video encoder system of claim 18 , wherein the multiple picture types further include (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
20. The video encoder system of claim 14 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.