Syntax and semantics for buffering information to simplify video splicing
Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using the CPBRD value for the current picture.
1. A computer system comprising one or more processing units and memory, wherein the computer system implements a video encoder configured to perform operations comprising:
setting a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture among pictures of a video sequence, the current picture having a first-type supplemental enhancement information (“SEI”) message associated with the current picture;
setting a value of a concatenation flag for the given access unit, wherein:
if the value of the concatenation flag is a first value, a CPBRD value in a second-type SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a first-type SEI message associated with the first preceding picture; and
if the value of the concatenation flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
signaling the CPBRD delta value and the value of the concatenation flag for the given access unit in the first-type SEI message associated with the current picture.
2. The computer system of claim 1 , wherein the CPBRD delta value is signaled in the first-type SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
3. The computer system of claim 1 , wherein the second preceding picture immediately precedes the current picture in decoding order.
4. The computer system of claim 1 , wherein the current picture is a random access picture, and wherein the signaling the CPBRD delta value and the value of the concatenation flag facilitates splicing that starts at the random access picture through a change to the value of the concatenation flag.
5. The computer system of claim 1 , wherein the operations further comprise:
setting the CPBRD value for the given access unit; and
signaling the CPBRD value for the given access unit in the second-type SEI message as a syntax element representing the CPBRD value minus 1.
6. The computer system of claim 1 , wherein, if the value of the concatenation flag is the first value:
if the second preceding picture lacks a first-type SEI message associated with the second preceding picture, the CPBRD delta value indicates a difference between the CPBRD value for the given access unit and a CPBRD value for the second preceding picture; and
if the second preceding picture has a first-type SEI message associated with the second preceding picture, the CPBRD delta value equals the CPBRD value for the given access unit.
7. The computer system of claim 1 , wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.
8. The computer system of claim 1 , wherein:
the second preceding picture lacks a first-type SEI message associated with the second preceding picture, and the first preceding picture and the second preceding picture are different pictures; or
the second preceding picture has a first-type SEI message associated with the second preceding picture, and the first preceding picture is the second preceding picture.
9. In a computer system that implements a video decoder, a method comprising:
receiving a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture among pictures of a video sequence, the current picture having a first-type supplemental enhancement information (“SEI”) message associated with the current picture;
receiving a value of a concatenation flag for the given access unit, wherein:
if the value of the concatenation flag is a first value, a CPBRD value in a second-type SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a first-type SEI message associated with the first preceding picture; and
if the value of the concatenation flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
decoding the current picture, including parsing the CPBRD delta value and the value of the concatenation flag for the given access unit from the first-type SEI message associated with the current picture.
10. The method of claim 9 , wherein the CPBRD delta value is signaled in the first-type SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
11. The method of claim 9 , wherein the current picture is a random access picture.
12. The method of claim 9 , further comprising:
receiving the CPBRD value for the given access unit, signaled in the second-type SEI message for the given access unit as a syntax element representing the CPBRD value minus 1.
13. The method of claim 9 , wherein, for a spliced bitstream, the nominal CPB removal time of the current picture can be computed as a sum of (a) the nominal CPB removal time of the second preceding picture and (b) a product of a clock tick value and the CPBRD delta value for the given access unit, constrained by a constraint that ensures that delay is at least as long as it would be if decoding started at the given access unit.
14. The method of claim 9 , further comprising:
based at least in part on the CPBRD delta value for the given access unit, computing the nominal CPB removal time of the current picture.
15. The method of claim 9 , wherein, if the value of the concatenation flag is the first value:
if the second preceding picture lacks a first-type SEI message associated with the second preceding picture, the CPBRD delta value indicates a difference between the CPBRD value for the given access unit and a CPBRD value for the second preceding picture; and
if the second preceding picture has a first-type SEI message associated with the second preceding picture, the CPBRD delta value equals the CPBRD value for the given access unit.
16. The method of claim 9 , wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.
17. The method of claim 9 , wherein:
the second preceding picture lacks a first-type SEI message associated with the second preceding picture, and the first preceding picture and the second preceding picture are different pictures; or
the second preceding picture has a first-type SEI message associated with the second preceding picture, and the first preceding picture is the second preceding picture.
18. One or more non-transitory computer-readable media having stored thereon encoded data in a bitstream for pictures of a video sequence, the encoded data being organized to facilitate decoding with operations comprising:
receiving a coded picture buffer removal delay (“CPBRD”) delta value for a given access unit for a current picture among the pictures of the video sequence, the current picture having a first-type supplemental enhancement information (“SEI”) message associated with the current picture;
receiving a value of a concatenation flag for the given access unit, wherein:
if the value of the concatenation flag is a first value, a CPBRD value in a second-type SEI message for the given access unit indicates an increment value specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a first-type SEI message associated with the first preceding picture; and
if the value of the concatenation flag is a second value, the nominal CPB removal time of the current picture is indicated, by the CPBRD delta value, as an increment value relative to a nominal CPB removal time of a second preceding picture in decoding order; and
decoding the current picture, including parsing the CPBRD delta value and the value of the concatenation flag for the given access unit from the first-type SEI message associated with the current picture.
19. The one or more computer-readable media of claim 18 , wherein the CPBRD delta value is signaled in the first-type SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
20. The one or more computer-readable media of claim 18 , wherein the operations further comprise receiving the CPBRD value for the given access unit, signaled in the second-type SEI message for the given access unit as a syntax element representing the CPBRD value minus 1.