Supplemental enhancement information including confidence level and mixed content information
This application relates to video encoding and decoding, and specifically to tools and techniques for using and providing supplemental enhancement information in bitstreams. Among other things, the detailed description presents innovations for bitstreams having supplemental enhancement information (SEI). In particular embodiments, the SEI message includes picture source data (e.g., data indicating whether the associated picture is a progressive scan picture or an interlaced scan picture and/or data indicating whether the associated picture is a duplicate picture). The SEI message can also express a confidence level of the encoder's relative confidence in the accuracy of this picture source data. A decoder can use the confidence level indication to determine whether the decoder should separately identify the picture as progressive or interlaced and/or a duplicate picture or honor the picture source scanning information in the SEI as it is.
1. A method performed by a decoder device, comprising:
receiving encoded data for a sequence of pictures in a bitstream or bitstream portion, the encoded data in the bitstream or bitstream portion including a first flag and a second flag for identifying source scan type for the sequence of pictures, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures of the sequence by a value of a picture-level syntax element that is to be signaled as part of an SEI message or to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive; and
determining, for a given picture of the pictures of the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein the determining the value of the picture-level syntax element includes:
determining that the picture-level syntax element is not present in the encoded data; and
inferring the value of the picture-level syntax element, including:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
decoding the pictures; and
processing the decoded pictures in accordance with the source scan type.
2. The method of claim 1 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
3. The method of claim 1 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
4. The method of claim 1 , wherein the SEI message is a picture-timing SEI message.
5. The method of claim 1 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures of the sequence by the value of the picture-level syntax element for that picture.
6. The method of claim 5 , wherein the picture-level syntax element for the given picture is not present in the encoded data, and wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is inferred to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is inferred to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is inferred to be interlaced.
7. A tangible computer-readable media storing computer-executable instructions for causing a computing device to perform operations, the operations comprising:
receiving encoded data for a sequence of pictures in a bitstream or bitstream portion, the encoded data in the bitstream or bitstream portion including a first flag and a second flag for identifying source scan type for the sequence of pictures, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures of the sequence by a value of a picture-level syntax element that is to be signaled as part of an SEI message or to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive; and
determining, for a given picture of the pictures of the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the value of the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein the determining the value of the picture-level syntax element includes:
determining whether or not the picture-level syntax element is present in the encoded data;
if the picture-level syntax element is present in the encoded data, setting the value of the picture-level syntax element according to the picture-level syntax element in the encoded data; and
otherwise, the picture-level syntax element not being present in the encoded data, inferring the value of the picture-level syntax element, including:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
decoding the pictures; and
processing the decoded pictures in accordance with the source scan type.
8. The tangible computer-readable media of claim 7 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
9. The tangible computer-readable media of claim 7 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
10. The tangible computer-readable media of claim 7 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures of the sequence by the value of the picture-level syntax element for that picture.
11. The tangible computer-readable media of claim 10 , wherein the picture-level syntax element in for the given picture is not present in the encoded data, and wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is inferred to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is inferred to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is inferred to be interlaced.
12. A computer system comprising one or more processors, memory, and storage storing computer-executable instructions for causing the one or more processors, when programmed thereby, to perform operations comprising:
receiving encoded data for a sequence of pictures in a bitstream or bitstream portion, the encoded data in the bitstream or bitstream portion including a first flag and a second flag for identifying source scan type for the sequence of pictures, the first flag and the second flag collectively and exclusively indicating one of the following unique states for the sequence of pictures: a state indicating that the source scan type of the pictures in the sequence is interlaced, a state indicating that the source scan type of the pictures in the sequence is progressive, a state indicating that the source scan type of the pictures in the sequence is unknown, and a state indicating that the source scan type is independently indicated for each picture of the pictures of the sequence by a value of a picture-level syntax element is to be signaled as part of an SEI message or is to be inferred, wherein the first flag indicates whether the source scan type of the pictures is interlaced and the second flag is different from the first flag, is a separate syntax element from the first flag, and indicates whether the source scan type of the pictures is progressive; and
determining, for a given picture of the pictures of the sequence, the value of the picture-level syntax element that indicates the source scan type of the given picture, the value of the picture-level syntax element indicating one of the following states: a state indicating that the source scan type of the given picture is interlaced, a state indicating that the source scan type of the given picture is progressive, and a state indicating that the source scan type of the given picture is unknown, wherein the determining the value of the picture-level syntax element includes:
determining that the picture-level syntax element is not present in the encoded data; and
inferring the value of the picture-level syntax element, including:
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is progressive, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is progressive;
if the first flag and the second flag indicate that the source scan type of the pictures in the sequence is interlaced, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is interlaced; and
otherwise, inferring the value of the picture-level syntax element to indicate that the source scan type of the given picture is unknown;
decoding the pictures; and
processing the decoded pictures in accordance with the source scan type.
13. The computer system of claim 12 , wherein the encoded data in the bitstream or bitstream portion further comprises a duplicate picture flag indicating whether one or more of the pictures are duplicate pictures.
14. The computer system of claim 12 , wherein the first flag and the second flag are part of information for a profile, level, and tier in the bitstream or bitstream portion.
15. The computer system of claim 12 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the pictures in the sequence is unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the pictures in the sequence is progressive;
if the first flag is 1 and the second flag is 0, the source scan type of the pictures in the sequence is interlaced; and
if the first flag is 1 and the second flag is 1, the source scan type is independently indicated for each picture of the pictures of the sequence by the value of the picture-level syntax element for that picture.
16. The computer system of claim 15 , wherein:
if the first flag is 0 and the second flag is 0, the source scan type of the given picture is inferred to be unknown;
if the first flag is 0 and the second flag is 1, the source scan type of the given picture is inferred to be progressive; and
if the first flag is 1 and the second flag is 0, the source scan type of the given picture is inferred to be interlaced.