Constraints and unit types to simplify video random access
Disclosed herein are innovations for bitstreams having clean random access (CRA) pictures and/or other types of random access point (RAP) pictures. New type definitions and strategic constraints on types of RAP pictures can simplify mapping of units of elementary video stream data to a container format. Such innovations can help improve the ability for video coding systems to more flexibly perform adaptive video delivery, production editing, commercial insertion, and the like.
1 . In a computer system that implements a video encoder, a method comprising:
encoding a random access point (RAP) picture, thereby producing encoded data for the RAP picture, the RAP picture being at a bitstream order position after an initial picture of a bitstream, the encoded data for the RAP picture including an indication signaling that the RAP picture is a picture from which a video decoder can begin decoding;
encoding one or more potentially non-decodable leading pictures associated with the RAP picture, thereby producing encoded data for the one or more potentially non-decodable leading pictures associated with the RAP picture;
encoding one or more non-leading pictures associated with the RAP picture, thereby producing encoded data for the one or more non-leading pictures associated with the RAP picture, wherein the one or more leading pictures associated with the RAP picture and the one or more non-leading pictures associated with the RAP picture are ordered such that all of the one or more leading pictures associated with the RAP picture precede, in decoding order, the one or more non-leading pictures associated with the RAP picture; and
outputting, as part of encoded data in the bitstream, the encoded data for the RAP picture, the encoded data for the one or more potentially non-decodable leading pictures associated with the RAP picture, and the encoded data for the one or more non-leading pictures associated with the RAP picture.
2 . The method of claim 1 , wherein the indication signaling that the RAP picture is a picture from which a video decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types.
3 . The method of claim 2 , wherein the plurality of RAP picture types include a broken link access (BLA) picture type and a clean random access (CRA) picture type.
4 . The method of claim 2 , wherein the syntax element indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RAP picture.
5 . The method of claim 1 , further comprising:
receiving, as part of the bitstream, the encoded data for the RAP picture; and
decoding the RAP picture such that decoding begins at the RAP picture.
6 . The method of claim 1 , wherein the encoded data for the one or more potentially non-decodable leading pictures associated with the RAP picture includes, for each of the one or more potentially non-decodable leading pictures associated with the RAP picture, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture.
7 . The method of claim 6 , wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RASL picture.
8 . The method of claim 1 , wherein the one or more leading pictures associated with the RAP picture are potentially non-decodable when decoding begins at the RAP picture due to missing reference pictures for the one or more leading pictures associated with the RAP picture when decoding begins at the RAP picture.
9 . A computer system comprising one or more processing units and memory, wherein the computer system implements a video decoder configured to perform operations comprising:
receiving, as part of encoded data in a bitstream, encoded data for a random access point (RAP) picture, encoded data for one or more potentially non-decodable leading pictures associated with the RAP picture, and encoded data for one or more non-leading pictures associated with the RAP picture, the RAP picture being at a bitstream order position after an initial picture of a bitstream, the encoded data for the RAP picture including an indication signaling that the RAP picture is a picture from which the video decoder can begin decoding, wherein the one or more leading pictures associated with the RAP picture and the one or more non-leading pictures associated with the RAP picture are ordered such that all of the one or more leading pictures associated with the RAP picture precede, in decoding order, the one or more non-leading pictures associated with the RAP picture;
decoding the RAP picture; and
decoding the one or more non-leading pictures associated with the RAP picture.
10 . The computer system of claim 9 , wherein the indication signaling that the RAP picture is a picture from which the video decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types.
11 . The computer system of claim 10 , wherein the plurality of RAP picture types include a broken link access (BLA) picture type and a clean random access (CRA) picture type.
12 . The computer system of claim 10 , wherein the syntax element indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RAP picture.
13 . The computer system of claim 9 , wherein the operations further comprise:
determining that decoding begins at the RAP picture; and
discarding the one or more potentially non-decodable leading pictures associated with the RAP picture.
14 . The computer system of claim 9 , wherein the operations further comprise:
determining that decoding did not begin at the RAP picture; and
decoding the one or more potentially non-decodable leading pictures associated with the RAP picture.
15 . The computer system of claim 9 , wherein the encoded data for the one or more potentially non-decodable leading pictures associated with the RAP picture includes, for each of the one or more potentially non-decodable leading pictures associated with the RAP picture, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture.
16 . The computer system of claim 15 , wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RASL picture.
17 . The computer system of claim 9 , wherein the one or more leading pictures associated with the RAP picture are potentially non-decodable when decoding begins at the RAP picture due to missing reference pictures for the one or more leading pictures associated with the RAP picture when decoding begins at the RAP picture.
18 . One or more non-transitory computer-readable media having stored thereon encoded data in a bitstream, the encoded data in the bitstream including encoded data for a random access point (RAP) picture, encoded data for one or more potentially non-decodable leading pictures associated with the RAP picture, and encoded data for one or more non-leading pictures associated with the RAP picture, the RAP picture being at a bitstream order position after an initial picture of a bitstream, the encoded data for the RAP picture including an indication signaling that the RAP picture is a picture from which a video decoder can begin decoding, wherein the one or more leading pictures associated with the RAP picture and the one or more non-leading pictures associated with the RAP picture are ordered such that all of the one or more leading pictures associated with the RAP picture precede, in decoding order, the one or more non-leading pictures associated with the RAP picture, the encoded data in the bitstream resulting from processing by operations that include:
encoding the RAP picture, thereby producing the encoded data for the RAP picture;
encoding the one or more potentially non-decodable leading pictures associated with the RAP picture, thereby producing the encoded data for the one or more potentially non-decodable leading pictures associated with the RAP picture; and
encoding the one or more non-leading pictures associated with the RAP picture, thereby producing the encoded data for the one or more non-leading pictures associated with the RAP picture.
19 . The one or more computer-readable media of claim 18 , wherein the indication signaling that the RAP picture is a picture from which a video decoder can begin decoding is a syntax element signaling that the RAP picture is one of a plurality of RAP picture types, the syntax element indicating a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RAP picture.
20 . The one or more computer-readable media of claim 18 , wherein the encoded data for one or more potentially non-decodable leading pictures associated with the RAP picture includes, for each of the one or more potentially non-decodable leading pictures associated with the RAP picture, an indication that the potentially non-decoding leading picture is a random access skipped leading (RASL) picture, and wherein the indication that the potentially non-decoding leading picture is a RASL picture is a syntax element that indicates a network abstraction layer (NAL) unit type value for a NAL unit that includes at least some of the encoded data for the RASL picture.