IP Library Granted Patent US 10,979,729
Granted Patent B2
US 10,979,729 · App. 15/826,565 · Granted Apr 13, 2021

Systems and methods for signaling and constraining a high dynamic range (HDR) video system with dynamic metadata

Inventors: Dmytro Rusanovskyy (San Diego, CA); Adarsh Krishnan Ramasubramonian (San Diego, CA); Joel Sole Rojals (Reus, ES)
Assignee: Qualcomm Incorporated
H04N19/50H04N1/64H04N19/12H04N19/136H04N19/17H04N19/176H04N19/18H04N19/186H04N19/46H04N19/70H04N19/39H04N19/52H04N21/4316H04N21/47
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,979,729
App. No.
15/826,565
Granted
Apr 13, 2021
Kind
B2
Abstract

Provided are methods, apparatus, and computer-readable medium for processing video data using a video coding system that implements ST 2094-10 for modifying the color volume of a video signal. In various implementations, metadata blocks that include color volume transform parameters can be included in an encoded bitstream with an encoded video. Mastering display color volume parameters must also be included in the encoded bitstream when the color volume transform parameters are included in the bitstream. When the mastering display color volume parameters are not also included in the bitstream, all or part of the bitstream is non-conforming.

Claims (69)

1. A method for processing video data, comprising:

receiving first video data and second video data, wherein the first video data is associated with a first color volume and the second video data is associated with a second color volume;

obtaining a first set of color volume transform parameters associated with the first video data, the first set of color volume transform parameters including a first transfer characteristic associated with a first transfer function, wherein the first set of color volume transform parameters can be used to transform the first color volume associated with the first video data according to the first transfer function;

obtaining a second set of color volume transform parameters associated with the second video data, the second set of color volume transform parameters including a second transfer characteristic associated with a second transfer function, wherein the second set of color volume transform parameters can be used to transform the second color volume associated with the second video data according to the second transfer function;

obtaining a set of mastering display color volume parameters, wherein the set of mastering display color volume parameters includes values determined based on generation of a master copy of the first video data; and

generating an encoded bitstream for the first video data, wherein generating the encoded bitstream comprises:

determining whether a first value associated with the first transfer characteristic of the first set of color volume transform parameters is a value of 16 associated with a Perceptual Quantizer (PQ) transfer function;

responsive to a determination that the first value associated with the first transfer characteristic of the first set of color volume transform parameters is the value of 16 associated with the PQ transfer function, generating at least one Society of Motion Picture and Television Engineers (SMPTE) ST 2094-10 metadata block for the first set of color volume transform parameters and including, in a first message of the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for the first set of color volume transform parameters, the at least one SMPTE ST 2094-10 metadata block including at least a length value indicating a length of the at least one SMPTE ST 2094-10 metadata block, a payload value indicating a level of payload included in the at least one SMPTE ST 2094-10 metadata block, and payload data corresponding to the level of payload; and

responsive to a determination that the at least one SMPTE ST 2094-10 metadata block for the first set of color volume transform parameters was included within the first message of the encoded bitstream, including, in the encoded bitstream, one or more metadata blocks for the set of mastering display color volume parameters based on a constraint indicating that one or more mastering display color volume messages shall be present in the encoded bitstream when one or more SMPTE ST 2094-10 metadata blocks are included in the encoded bitstream;

determining whether a second value associated with the second transfer characteristic of the second set of color volume transform parameters is the value of 16 associated with the PQ transfer function; and

responsive to a determination that the second value associated with the second transfer characteristic of the second set of color volume transform parameters is a value other than the value of 16 associated with the PQ transfer function, determining not to generate any SMPTE ST 2094-10 metadata blocks for the second set of color volume transform parameters.

2. The method of claim 1 , further comprising:

determining that the first set of color volume transform parameters and the set of mastering display color volume parameters include a same field; and

omitting the field from the at least one metadata block for the first set of color volume transform parameters based on the field being present in the one or more metadata blocks for the set of mastering display color volume parameters.

3. The method of claim 1 , wherein the first video data includes a plurality of processing windows, each processing window describing a respective portion of a display, the method further comprising constraining a number of processing windows included within the encoded bitstream to be a value between one and sixteen.

4. The method of claim 1 , wherein the first video data includes a plurality of content description elements, each content description element providing information associated with a video signal, the method further comprising constraining a number content description elements to be a value of one.

5. The method of claim 1 , wherein the first video data includes a plurality of target display elements, each target display element providing information associated with a display device configured to render a particular video signal, the method further comprising constraining a number of target display elements to be a value between one and sixteen.

6. The method of claim 1 , further comprising, including, in the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for each access unit in the encoded bitstream.

7. The method of claim 1 , wherein values defined as reserved are excluded from the encoded bitstream.

8. The method of claim 1 , further comprising constraining the length value to be a multiple of eight.

9. The method of claim 1 , further comprising constraining the length value to be a value between 0 and 255.

10. The method of claim 1 , further comprising encoding the at least one SMPTE ST 2094-10 metadata block in at least one Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) unit.

11. The method of claim 1 , further comprising encoding the one or more metadata blocks for the set of mastering display color volume parameters in at least one Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) unit.

12. An apparatus for processing video data, comprising:

a memory configured to store video data, wherein the video data is associated with a color volume; and

a processor implemented in circuitry and configured to:

obtain a set of color volume transform parameters associated with the video data, the set of color volume transform parameters including a transfer characteristic associated with a transfer function, wherein the set of color volume transform parameters can be used to transform the color volume associated with the video data according to the transfer function;

obtain a set of mastering display color volume parameters, wherein the set of mastering display color volume parameters includes values determined based on generation of a master copy of the video data; and

generate an encoded bitstream for the video data, such that the processor is further configured to:

determine whether a value associated with the transfer characteristic of the set of color volume transform parameters is a value of 16 associated with a Perceptual Quantizer (PQ) transfer function;

responsive to a determination that the value associated with the transfer characteristic of the set of color volume transform parameters is the value of 16 associated with the PQ transfer function, generate at least one Society of Motion Picture and Television Engineers (SMPTE) ST 2094-10 metadata block for the set of color volume transform parameters and including, in a message of the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for the set of color volume transform parameters, the at least one SMPTE ST 2094-10 metadata block including at least a length value indicating a length of the at least one SMPTE ST 2094-10 metadata block, a payload value indicating a level of payload included in the at least one SMPTE ST 2094-10 metadata block, and payload data corresponding to the level of payload;

responsive to a determination that the value associated with the transfer characteristic of the set of color volume transform parameters is a value other than the value of 16 associated with the PQ transfer function, determine not to generate any SMPTE ST 2094-10 metadata blocks for the second set of color volume transform parameters; and

responsive to a determination that the at least one SMPTE ST 2094-10 metadata block for the set of color volume transform parameters was included within the message of the encoded bitstream, include, in the encoded bitstream, one or more metadata blocks for the set of mastering display color volume parameters based on a constraint indicating that one or more mastering display color volume messages shall be present in the encoded bitstream when one or more SMPTE ST 2094-10 metadata blocks are included in the encoded bitstream.

13. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

receive first video data and second video data, wherein the first video data is associated with a first color volume and the second video data is associated with a second color volume;

obtain a first set of color volume transform parameters associated with the first video data, the first set of color volume transform parameters including a first transfer characteristic associated with a first transfer function, wherein the first set of color volume transform parameters can be used to transform the first color volume associated with the first video data according to the first transfer function;

obtain a second set of color volume transform parameters associated with the second video data, the second set of color volume transform parameters including a second transfer characteristic associated with a second transfer function, wherein the second set of color volume transform parameters can be used to transform the second color volume associated with the second video data according to the second transfer function;

obtain a set of mastering display color volume parameters, wherein the set of mastering display color volume parameters includes values determined based on generation of a master copy of the first video data; and

generate an encoded bitstream for the first video data, wherein the non-transitory computer-readable medium comprises further instructions that cause the one or more processors to:

determine whether a first value associated with the first transfer characteristic of the first set of color volume transform parameters is a value of 16 associated with a Perceptual Quantizer (PQ) transfer function;

responsive to a determination that the first value associated with the first transfer characteristic of the first set of color volume transform parameters is the value of 16 associated with the PQ transfer function, generating at least one Society of Motion Picture and Television Engineers (SMPTE) ST 2094-10 metadata block for the first set of color volume transform parameters and including, in a first message of the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for the first set of color volume transform parameters, the at least one SMPTE ST 2094-10 metadata block including at least a value indicating a length of the at least one SMPTE ST 2094-10 metadata block, a value indicating a level of payload included in the at least one SMPTE ST 2094-10 metadata block, and payload data corresponding to the level of payload; and

responsive to a determination that the at least one SMPTE ST 2094-10 metadata block for the first set of color volume transform parameters was included within the first message of the encoded bitstream, including, in the encoded bitstream, one or more metadata blocks for the set of mastering display color volume parameters based on a constraint indicating that one or more mastering display color volume messages shall be present in the encoded bitstream when one or more SMPTE ST 2094-10 metadata blocks are included in the encoded bitstream;

determine whether a second value associated with the second transfer characteristic of the second set of color volume transform parameters is the value of 16 associated with the PQ transfer function; and

responsive to a determination that the second value associated with the second transfer characteristic of the second set of color volume transform parameters is a value other than the value of 16 associated with the PQ transfer function, determining not to generate any SMPTE ST 2094-10 metadata blocks for the second set of color volume transform parameters.

14. The apparatus of claim 12 , wherein the processor is further configured to:

determine that the set of color volume transform parameters and the set of mastering display color volume parameters include a same field; and

omit the field from the at least one metadata block for the set of color volume transform parameters based on the field being present in the one or more metadata blocks for the set of mastering display color volume parameters.

15. The apparatus of claim 12 , wherein the video data includes a plurality of processing windows, each processing window describing a respective portion of a display, and wherein the processor is further configured to constrain a number of processing windows included within the encoded bitstream to be a value between one and sixteen.

16. The apparatus of claim 12 , wherein the video data includes a plurality of content description elements, each content description element providing information associated with a video signal, and wherein the processor is further configured to constrain a number content description elements to be a value of one.

17. The apparatus of claim 12 , wherein the video data includes a plurality of target display elements, each target display element providing information associated with a display device configured to render a particular video signal, and wherein the processor is further configured to constrain a number of target display elements to be a value between one and sixteen.

18. The apparatus of claim 12 , wherein the processor is further configured to, include, in the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for each access unit in the encoded bitstream.

19. The apparatus of claim 12 , wherein the processor is further configured to exclude values defined as reserved from the encoded bitstream.

20. The apparatus of claim 12 , wherein the processor is further configured to constrain the length value to be a multiple of eight.

21. The apparatus of claim 12 , wherein the processor is further configured to constrain the length value to be a value between 0 and 255.

22. The apparatus of claim 12 , wherein the processor is further configured to encode the at least one SMPTE ST 2094-10 metadata block in at least one Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) unit.

23. The apparatus of claim 12 , wherein the processor is further configured to encode the one or more metadata blocks for the set of mastering display color volume parameters in at least one Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) unit.

24. The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed, cause the one or more processors:

determine that the first set of color volume transform parameters and the set of mastering display color volume parameters include a same field; and

omit the field from the at least one metadata block for the first set of color volume transform parameters based on the field being present in the one or more metadata blocks for the set of mastering display color volume parameters.

25. The non-transitory computer-readable medium of claim 13 , wherein the first video data includes a plurality of processing windows, each processing window describing a respective portion of a display, and wherein the non-transitory computer-readable medium comprises further instructions that cause the one or more processors to constrain a number of processing windows included within the encoded bitstream to be a value between one and sixteen.

26. The non-transitory computer-readable medium of claim 13 , wherein the first video data includes a plurality of content description elements, each content description element providing information associated with a video signal, and wherein the non-transitory computer-readable medium comprises further instructions that cause the one or more processors to constrain a number content description elements to be a value of one.

27. The non-transitory computer-readable medium of claim 13 , wherein the first video data includes a plurality of target display elements, each target display element providing information associated with a display device configured to render a particular video signal, and wherein the non-transitory computer-readable medium comprises further instructions that cause the one or more processors to constrain a number of target display elements to be a value between one and sixteen.

28. The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed, cause the one or more processors to, include, in the encoded bitstream, the at least one SMPTE ST 2094-10 metadata block for each access unit in the encoded bitstream.

29. The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed, cause the one or more processors to constrain the length value to be a multiple of eight.

30. The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed, cause the one or more processors to constrain the length value to be a value between 0 and 255.

31. The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed, cause the one or more processors to encode the one or more metadata blocks for the set of mastering display color volume parameters in at least one Supplemental Enhancement Information (SEI) Network Abstraction Layer (NAL) unit.

32. The method of claim 1 , wherein the first message is a supplemental information enhancement (SEI) message.

33. The apparatus of claim 12 , wherein the message is a supplemental information enhancement (SEI) message.

34. The non-transitory computer-readable medium of claim 13 , wherein the first message is a supplemental information enhancement (SEI) message.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: RUSANOVSKYY, DMYTRO; RAMASUBRAMONIAN, ADARSH KRISHNAN; SOLE ROJALS, JOEL
To: QUALCOMM INCORPORATED
Reel/Frame 044823/0513 →
Continuity (2)
Provisional Application 62428511 · Nov 30, 2016
Related Publication 20180152703A1 · May 31, 2018
Cited By (3)
US 12,244,934 US 12,445,637 US 12,501,064