Constraints and unit types to simplify video random access
Disclosed herein are innovations for bitstreams having clean random access (CRA) pictures and/or other types of random access point (RAP) pictures. New type definitions and strategic constraints on types of RAP pictures can simplify mapping of units of elementary video stream data to a container format. Such innovations can help improve the ability for video coding systems to more flexibly perform adaptive video delivery, production editing, commercial insertion, and the like.
1. In a computer system that implements a video decoder, a method comprising:
receiving, as part of encoded data in a bitstream, encoded data for a first picture, wherein the first picture is designated, according to network abstraction layer (“NAL”) unit type information, as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture;
decoding the first picture, the first picture being designated as the first picture type;
receiving, as part of the encoded data in the bitstream, encoded data for a second picture, wherein the second picture is designated, according to NAL unit type information, as being one of the multiple picture types; and
decoding the second picture, the second picture being designated as the third picture type.
2. The method of claim 1 , wherein the first picture is not an initial picture of the bitstream.
3. The method of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
4. The method of claim 3 , further comprising:
receiving, as part of the encoded data in the bitstream, encoded data for a third picture, wherein the third picture is designated, according to NAL unit type information, as being one of the multiple picture types;
decoding the third picture, the third picture being designated as the fourth picture type;
receiving, as part of the encoded data in the bitstream, encoded data for a fourth picture, wherein the fourth picture is designated, according to NAL unit type information, as being one of the multiple picture types; and
selectively decoding the fourth picture, the fourth picture being designated as the third picture type or the fifth picture type.
5. The method of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the third picture type, decoding the fourth picture.
6. The method of claim 4 , wherein the selectively decoding the fourth picture includes:
responsive to the fourth picture being designated as the fifth picture type, skipping the decoding of the fourth picture.
7. The method of claim 1 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures, and (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
8. The method of claim 1 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
9. One or more non-transitory computer-readable media having stored thereon encoded data in a bitstream, the encoded data in the bitstream including encoded data for a first picture and encoded data for a second picture, wherein the first picture is designated, according to network abstraction layer (“NAL”) unit type information, as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture, and wherein the second picture is designated, according to NAL unit type information, as being one of the multiple picture types, the encoded data being organized to facilitate decoding, in a computer system that implements a video decoder, by operations that include:
decoding the first picture, the first picture being designated as the first picture type; and
decoding the second picture, the second picture being designated as the third picture type.
10. The one or more computer-readable media of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
11. The one or more computer-readable media of claim 10 , wherein the encoded data in the bitstream further includes encoded data for a third picture and encoded data for a fourth picture, wherein the third picture is designated, according to NAL unit type information, as being one of the multiple picture types, wherein the fourth picture is designated, according to NAL unit type information, as being one of the multiple picture types, and wherein the operations further include:
decoding the third picture, the third picture being designated as the fourth picture type; and
selectively decoding the fourth picture, the fourth picture being designated as the third picture type or the fifth picture type.
12. The one or more computer-readable media of claim 9 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture, (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures, and (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
13. The one or more computer-readable media of claim 9 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.
14. A computer system comprising one or more processing units and memory, wherein the computer system implements a video encoder configured to perform operations comprising:
encoding a first picture, thereby generating encoded data for the first picture, wherein the first picture is designated, according to network abstraction layer (“NAL”) unit type information in the encoded data for the first picture, as being one of multiple picture types, the multiple picture types including (a) a first picture type indicating a broken link access (“BLA”) picture that is capable of being used as a random access point (“RAP”) picture and does not have any associated non-decodable leading pictures, but may have one or more associated decodable leading pictures, (b) a second picture type indicating a BLA picture that is capable of being used as a RAP picture but does not have any associated leading pictures, and (c) a third picture type indicating a random access decodable leading (“RADL”) picture;
outputting, as part of encoded data in a bitstream, the encoded data for the first picture, wherein the first picture is designated as the first picture type in the encoded data for the first picture;
encoding a second picture, thereby generating encoded data for the second picture, wherein the second picture is designated, according to NAL unit type information in the encoded data for the second picture, as being one of the multiple picture types; and
outputting, as part of the encoded data in the bitstream, the encoded data for the second picture, wherein the second picture is designated as the third picture type in the encoded data for the second picture.
15. The computer system of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a BLA picture that is capable of being used as a RAP picture and may have one or more associated decodable or non-decodable leading pictures, and (e) a fifth picture type indicating a random access skipped leading (“RASL”) picture.
16. The computer system of claim 15 , wherein the operations further include:
encoding a third picture, thereby generating encoded data for the third picture, wherein the third picture is designated, according to NAL unit type information in the encoded data for the third picture, as being one of the multiple picture types;
outputting, as part of the encoded data in the bitstream, the encoded data for the third picture, wherein the third picture is designated as the fourth picture type in the encoded data for the third picture;
encoding a fourth picture, thereby generating encoded data for the fourth picture, wherein the fourth picture is designated, according to NAL unit type information in the encoded data for the fourth picture, as being one of the multiple picture types; and
outputting, as part of the encoded data in the bitstream, the encoded data for the fourth picture, wherein the fourth picture is designated as the third picture type or the fifth picture type in the encoded data for the fourth picture.
17. The computer system of claim 14 , wherein the multiple picture types further include (d) a fourth picture type indicating a clean random access (“CRA”) capable of being used as a RAP picture.
18. The computer system of claim 17 , wherein the multiple picture types further include (e) a fifth picture type indicating an instantaneous decoding refresh (“IDR”) picture that may have one or more associated decodable leading pictures.
19. The computer system of claim 18 , wherein the multiple picture types further include (f) a sixth picture type indicating an IDR picture that does not have any associated leading pictures.
20. The computer system of claim 14 , wherein the bitstream is ordered such that any leading pictures, including the second picture, associated with the first picture precede all non-leading pictures associated with the first picture.