Supplemental enhancement information including confidence level and mixed content information
This application relates to video encoding and decoding, and specifically to tools and techniques for using and providing supplemental enhancement information in bitstreams. Among other things, the detailed description presents innovations for bitstreams having supplemental enhancement information (SEI). In particular embodiments, the SEI message includes picture source data (e.g., data indicating whether the associated picture is a progressive scan picture or an interlaced scan picture and/or data indicating whether the associated picture is a duplicate picture). The SEI message can also express a confidence level of the encoder's relative confidence in the accuracy of this picture source data. A decoder can use the confidence level indication to determine whether the decoder should separately identify the picture as progressive or interlaced and/or a duplicate picture or honor the picture source scanning information in the SEI as it is.
1. A method performed by an encoder device, the method comprising:
determining a first flag and a second flag for identifying source scan type of pictures in a sequence, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures in the sequence by a value of a picture-level syntax element that is to be signaled as part of an SEI message or to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive;
outputting the first flag and the second flag as part of encoded data in a bitstream or bitstream portion;
determining, for a given picture of the pictures in the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein, if the picture-level syntax element is not present in the encoded data in the bitstream or bitstream portion, the value of the picture-level syntax element is to be inferred according to rules that include:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
if the picture-level syntax element is to be signaled, outputting the picture-level syntax element as part of the encoded data in the bitstream or bitstream portion;
encoding the given picture; and
outputting, as part of the encoded data in the bitstream or bitstream portion, the encoded given picture.
2. The method of claim 1 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
3. The method of claim 1 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
4. The method of claim 1 , wherein the SEI message is a picture-timing SEI message.
5. The method of claim 1 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures in the sequence by the value of the picture-level syntax element for that picture.
6. The method of claim 5 , wherein the picture-level syntax element for the given picture is not present in the encoded data, and wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is set to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is set to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is set to be interlaced.
7. The method of claim 1 , further comprising:
determining the source scan type of the pictures in the sequence, including analyzing source scan characteristics of the pictures in the sequence.
8. The method of claim 1 , further comprising:
determining the source scan type of the given picture, including analyzing source scan characteristics of the given picture.
9. A tangible computer-readable media storing encoded data for a sequence of pictures in a bitstream or bitstream portion, the encoded data in the bitstream or bitstream portion including a first flag and a second flag for identifying source scan type of the pictures in the sequence, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures in the sequence by a value of a picture-level syntax element that is to be signaled as part of an SEI message or to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive, the encoded data being organized to facilitate decoding by operations that include:
determining, for a given picture of the pictures in the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the value of the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein the determining the value of the picture-level syntax element includes:
determining whether or not the picture-level syntax element is present in the encoded data;
if the picture-level syntax element is present in the encoded data, setting the value of the picture-level syntax element according to the picture-level syntax element in the encoded data; and
otherwise, the picture-level syntax element not being present in the encoded data, inferring the value of the picture-level syntax element, including:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
decoding the given picture; and
processing the given picture in accordance with the source scan type of the given picture.
10. The tangible computer-readable media of claim 9 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
11. The tangible computer-readable media of claim 9 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
12. The tangible computer-readable media of claim 9 , wherein the SEI message is a picture-timing SEI message.
13. The tangible computer-readable media of claim 9 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures in the sequence by the value of the picture-level syntax element for that picture.
14. The tangible computer-readable media of claim 13 , wherein the picture-level syntax element for the given picture is not present in the encoded data, and wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is set to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is set to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is set to be interlaced.
15. A computer system comprising:
a buffer, implemented using memory, configured to receive encoded data for a sequence of pictures in a bitstream or bitstream portion, the encoded data in the bitstream or bitstream portion including a first flag and a second flag for identifying source scan type of the pictures in the sequence, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures in the sequence by a value of a picture-level syntax element that is to be signaled as part of an SEI message or to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive; and
a video decoder, implemented using one or more processors, configured to perform operations comprising:
determining, for a given picture of the pictures in the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the value of the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein the determining the value of the picture-level syntax element includes:
determining that the picture-level syntax element is not present in the encoded data; and
inferring the value of the picture-level syntax element, including:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
decoding the given picture; and
processing the given picture in accordance with the source scan type of the given picture.
16. The computer system of claim 15 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
17. The computer system of claim 15 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
18. The computer system of claim 15 , wherein the SEI message is a picture-timing SEI message.
19. The computer system of claim 15 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures in the sequence by the value of the picture-level syntax element for that picture.
20. The computer system of claim 19 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is set to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is set to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is set to be interlaced.