Syntax and semantics for buffering information to simplify video splicing
Innovations in syntax and semantics of coded picture buffer removal delay (“CPBRD”) values potentially simplify splicing operations. For example, a video encoder sets a CPBRD value for a current picture that indicates an increment value relative to a nominal coded picture buffer removal time of a preceding picture in decoding order, regardless of whether the preceding picture has a buffering period SEI message. The encoder can signal the CPBRD value according to a single-value approach in which a flag indicates how to interpret the CPBRD value, according to a two-value approach in which another CPBRD value (having a different interpretation) is also signaled, or according to a two-value approach that uses a flag and a delta value. A corresponding video decoder receives and parses the CPBRD value for the current picture. A splicing tool can perform simple concatenation operations to splice bitstreams using CPBRD value for the current picture.
1. A computer system comprising:
a buffer configured to receive encoded data for pictures of a video sequence, the encoded data including:
a picture timing supplemental enhancement information (“SEI”) message associated with a current picture among the pictures of the video sequence, the picture timing SEI message including a coded picture buffer removal delay (“CPBRD”) value for the current picture, the CPBRD value for the current picture specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
a buffering period SEI message associated with the current picture, the buffering period SEI message associated with the current picture including a concatenation flag and a CPBRD delta value, wherein the concatenation flag has a given value, and wherein the CPBRD delta value indicates a difference between the CPBRD value for the current picture and a CPBRD value for a second preceding picture in decoding order, the second preceding picture lacking a buffering period SEI message associated with the second preceding picture; and
a video decoder configured to decode the encoded data by performing operations that include parsing the picture timing SEI message associated with the current picture and parsing the buffering period SEI message associated with the current picture, wherein the CPBRD delta value enables a video decoder, using the CPBRD delta value and based on the concatenation flag having the given value, to detect loss of encoded data for the second preceding picture.
2. The computer system of claim 1 , wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
3. The computer system of claim 1 , wherein the CPBRD value for the current picture is signaled in the picture timing SEI message associated with the current picture as a syntax element representing the CPBRD value for the current picture minus 1.
4. The computer system of claim 1 , wherein the current picture is a random access picture.
5. The computer system of claim 1 , wherein the operations further comprise:
determining, based on a check indicating (a) the CPBRD value for the current picture is equal to (b) a sum based on the CPBRD value for the second preceding picture and the CPBRD delta value for the current picture, that the encoded data for the second preceding picture has not been lost.
6. The computer system of claim 1 , wherein the operations further comprise:
determining, based on a check indicating (a) the CPBRD value for the current picture is not equal to (b) a sum based on the CPBRD value for the second preceding picture and the CPBRD delta value for the current picture, that the encoded data for the second preceding picture has been lost.
7. The computer system of claim 1 , wherein the second preceding picture immediately precedes the current picture in decoding order.
8. The computer system of claim 1 , wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.
9. In a computer system, a method comprising:
receiving encoded data for pictures of a video sequence, the encoded data including:
a picture timing supplemental enhancement information (“SEI”) message associated with a current picture among the pictures of the video sequence, the picture timing SEI message including a coded picture buffer removal delay (“CPBRD”) value for the current picture, the CPBRD value for the current picture specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
a buffering period SEI message associated with the current picture, the buffering period SEI message associated with the current picture including a concatenation flag and a CPBRD delta value, wherein the concatenation flag has a given value, and wherein the CPBRD delta value indicates a difference between the CPBRD value for the current picture and a CPBRD value for a second preceding picture in decoding order, the second preceding picture lacking a buffering period SEI message associated with the second preceding picture; and
storing, in memory or storage, the encoded data, the encoded data being organized to facilitate decoding by operations that include parsing the picture timing SEI message associated with the current picture and parsing the buffering period SEI message associated with the current picture, wherein the CPBRD delta value enables a video decoder, using the CPBRD delta value and based on the concatenation flag having the given value, to detect loss of encoded data for the second preceding picture.
10. The method of claim 9 , wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
11. The method of claim 9 , wherein the CPBRD value for the current picture is signaled in the picture timing SEI message associated with the current picture as a syntax element representing the CPBRD value for the current picture minus 1.
12. The method of claim 9 , wherein the current picture is a random access picture.
13. The method of claim 9 , wherein the second preceding picture immediately precedes the current picture in decoding order.
14. The method of claim 9 , wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.
15. One or more computer-readable media having stored thereon computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory, and storage, the operations comprising:
encoding pictures of a video sequence, thereby producing encoded data, wherein the encoding the pictures includes:
encoding a picture timing supplemental enhancement information (“SEI”) message associated with a current picture among the pictures of the video sequence, the picture timing SEI message including a coded picture buffer removal delay (“CPBRD”) value for the current picture, the CPBRD value for the current picture specifying a nominal coded picture buffer (“CPB”) removal time of the current picture relative to a nominal CPB removal time of a first preceding picture in decoding order, the first preceding picture having a buffering period SEI message associated with the first preceding picture; and
encoding a buffering period SEI message associated with the current picture, the buffering period SEI message associated with the current picture including a concatenation flag and a CPBRD delta value, wherein the concatenation flag has a given value, and wherein the CPBRD delta value indicates a difference between the CPBRD value for the current picture and a CPBRD value for a second preceding picture in decoding order, the second preceding picture lacking a buffering period SEI message associated with the second preceding picture; and
outputting the encoded data as part of a bitstream, the encoded data including the picture timing SEI message associated with the current picture and the buffering period SEI message associated with the current picture, thereby enabling a video decoder, using the CPBRD delta value and so long as the concatenation flag has the given value, to detect loss of encoded data for the second preceding picture.
16. The one or more computer-readable media of claim 15 , wherein the CPBRD delta value is signaled in the buffering period SEI message associated with the current picture as a syntax element representing the CPBRD delta value minus 1.
17. The one or more computer-readable media of claim 15 , wherein the CPBRD value for the current picture is signaled in the picture timing SEI message associated with the current picture as a syntax element representing the CPBRD value for the current picture minus 1.
18. The one or more computer-readable media of claim 15 , wherein the current picture is a random access picture.
19. The one or more computer-readable media of claim 15 , wherein the second preceding picture immediately precedes the current picture in decoding order.
20. The one or more computer-readable media of claim 15 , wherein:
the current picture is part of a temporal layer, and the second preceding picture immediately precedes the current picture in decoding order in the temporal layer of the current picture or a lower temporal layer; or
the second preceding picture, among pictures having a temporal identifier of zero, immediately precedes the current picture in decoding order.